Hadoop 'No Longer Optional,' Says Forrester - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Data Management // Software Platforms
News
11/4/2014
09:06 AM
Connect Directly
LinkedIn
Twitter
RSS
E-Mail
50%
50%

Hadoop 'No Longer Optional,' Says Forrester

Forrester says economics will make Hadoop "mandatory," predicts new roles, new software sources, and an end to skills shortage by 2015.

HP's 3D Future: Sprout Visual Tour
HP's 3D Future: Sprout Visual Tour
(Click image for larger view and slideshow.)

"Hadooponomics" will make Apache's open-source big data platform a "must have for large enterprises."

This is the first of eight predictions Forrester Research published on Tuesday in "Predictions 2015: Hadoop Will Become A Cornerstone Of Your Business Technology Agenda." Hadooponomics is "the ability to linearly scale both data storage and data processing," the report explains, and it is closely tied to the ability to leverage "pay-per-use cloudonomics."

Many enterprises are just dabbling in Hadoop, the report acknowledges, but leaders including Wal-Mart, Fidelity Investments, Sears, Verizon, USAA, Cardinal Health, Wells Fargo, Proctor & Gamble, Cablevision, Nasdaq, AutoTrader, Netflix, and Yelp have proven, mission-critical Hadoop deployments.

"The remaining minority of dazed and confused CIOs will make Hadoop a priority for 2015," the report states. "Application development and delivery professionals should be ready and waiting with a compelling use case to get started."

[Want more on this big data platform? Read Hadoop: 5 Undeniable Truths.]

The report, which was written by Mike Gualtieri along with half a dozen other Forrester analysts, offers eight predictions about Hadoop for 2015, with the most notable addressing SQL, cloud deployment, roles beyond analytics, and new sources of Hadoop software.

SQL becomes Hadoop's killer app
Fast, ANSI-compliant SQL-on-Hadoop options will "create immediate opportunities for Hadoop to become a useful data platform for enterprises" because they will be familiar to data-management professionals and accessible to existing systems, says Forrester. This will provide a sandbox for analysis of data that is not currently accessible.

Hadoop clusters bloom in the cloud
Storage, compute, and networking capacity needs are rarely in sync, so cloud elasticity will be a key to cost efficiency, the report argues. Thus, adopters will increasingly use Hadoop in the cloud to meet demand for ad hoc analytics, yet on-premises adoption "won't slow down."

Hadoop won't be just for analytics
With Hadoop's YARN management layer, options including HBase, the transactional NoSQL database, and the Apache Spark in-memory platform will turn Hadoop into an application platform that can run code in combination with data. This will "collapse the application stack," and vendors that offer middleware, databases, search engines, integration tools, and other software infrastructure "must move to develop versions that will run natively inside Hadoop," Forrester warns.

New Hadoop sources emerge
Not only does Forrester foresee new Hadoop distributions emerging from the likes of HP, Oracle, SAP, Software AG, and Tibco, it "sees no reason" why Microsoft, Red Hat, VMware, and other operating system vendors can't include Hadoop and make it a configurable option from within their operating systems. "This would be a game-changer because it would mean that every node could be configured as a Hadoop node," Gualtieri writes. "This would disrupt the existing model used by Hadoop distribution vendors that charge $2,000 to $3,000 per node, per year for their current distributions."

Perhaps Forrester's most surprising prediction is that the Hadoop skills shortage will disappear in 2015. Understanding a distributed file system and distributing computing platform is "no big deal," Forrester says, and the Java APIs are "nothing new" to Java application developers.

"CIOs won't have to hire high-priced Hadoop consultants to get projects done," according to the report. "Hadoop projects will get done faster because the enterprise's very own application developers and operations professionals know the data, the integration points, the applications, and the business challenges."

This last prediction seems particularly optimistic, as the collective understanding of Hadoop is not likely to turn on like a light bulb in 2015. What's more, we're guessing that Hadoop won't be perceived as much of a "killer" platform if SQL is the pinnacle of its analysis capabilities. Machine learning, high-scale correlation, streaming analysis, and the discovery of important, predictive patterns across new and old data types is the stuff that will deliver real breakthroughs.

What will you use for your big data platform? A high-scale relational database? NoSQL database? Hadoop? Event-processing technology? One size doesn't fit all. Here's how to decide. Get the new Pick Your Platform For Big Data issue of InformationWeek Tech Digest today. (Free registration required.)

Doug Henschen is Executive Editor of InformationWeek, where he covers the intersection of enterprise applications with information management, business intelligence, big data and analytics. He previously served as editor in chief of Intelligent Enterprise, editor in chief of ... View Full Bio

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Comments
Threaded  |  Newest First  |  Oldest First
David F. Carr
100%
0%
David F. Carr,
User Rank: Author
11/4/2014 | 10:05:54 AM
Too much focus on Hadoop?
Is Hadoop really that unique? Or is it part of a generation of new data technologies that are all vying for attention? I didn't think the focus on Hadoop was so unanimous as Forrester makes it out to be.
D. Henschen
50%
50%
D. Henschen,
User Rank: Author
11/4/2014 | 10:19:16 AM
Re: Too much focus on Hadoop?
No, it's really the case that there's nothing on an equal footing with Hadoop in the pantheon of emerging big-data technologies. There are a variety of NoSQL databases for different use cases and with different degrees of adoption. But only Hadoop is being widely adopted and supported by vendors as the centerpiece data lake/data hub for capturing and economically retaining enterprise data. There are many tools to analyze, report on, and predict against data, but you have to have the data platform in place to drive all those possibilities. That's the foundation.
David F. Carr
50%
50%
David F. Carr,
User Rank: Author
11/4/2014 | 10:37:47 AM
Re: Too much focus on Hadoop?
Data lake? I must be falling behind on my tech jargon.
D. Henschen
50%
50%
D. Henschen,
User Rank: Author
11/4/2014 | 1:03:27 PM
Re: Too much focus on Hadoop?
David, I wouldn't sweat it. If you weren't covering data warehousing, data marts, business intelligence and analytics before this era, you probably won't be spending much time on data lakes/data hubs etc. going forward. The bottom line for business will be data-informed predictions and prescriptive recommendations. Hopefully these can be exposed as services so those building and consuming apps won't have to worry about the data-intensive work going on behind the scenes.
InformationWeek Is Getting an Upgrade!

Find out more about our plans to improve the look, functionality, and performance of the InformationWeek site in the coming months.

Commentary
New Storage Trends Promise to Help Enterprises Handle a Data Avalanche
John Edwards, Technology Journalist & Author,  4/1/2021
Slideshows
11 Things IT Professionals Wish They Knew Earlier in Their Careers
Lisa Morgan, Freelance Writer,  4/6/2021
Commentary
How to Submit a Column to InformationWeek
InformationWeek Staff 4/9/2021
White Papers
Register for InformationWeek Newsletters
Video
Current Issue
Successful Strategies for Digital Transformation
Download this report to learn about the latest technologies and best practices or ensuring a successful transition from outdated business transformation tactics.
Slideshows
Flash Poll