No One Buys Hadoop - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Data Management // Big Data Analytics
Commentary
4/16/2015
10:15 AM
50%
50%

No One Buys Hadoop

In launching a big data initiative the work extends far beyond the acquisition of Hadoop or other technologies.

The first quarter results for Hortonworks after its IPO showed that professional services were growing faster than support. That's unsurprising news for someone like myself who is working in big data professional services.

The truth is that no one buys Hadoop. You have a problem, and you want a solution. Big data is not about Hadoop or NoSQL. It is about a group of problems that require novel technologies and architecture patterns to solve them.

We can observe this with the evolution of the distributions. Initially, it was about large-scale computation and distributed storage with MapReduce and HDFS. There were early adopters who had problems fitting for these technologies that have their roots in crawling and processing the web. The distributions made Hadoop accessible and gave support and peace of mind to business operations. The decreasing storage cost and accessibility of Hadoop pushed it further into more and more use cases.

Today, as big data is becoming ever more watered down as a term it still has a core technology with Hadoop. However, the requirements are now around a multi-purpose enterprise grade large-scale data storage and processing architecture that does so much more than MapReduce and HDFS. As such, the distributions are growing upwards in the value chain and outwards on the feature set. Search, governance, security, compliance, business intelligence, and data warehouse integration with Hadoop are only some prominent examples of this movement. The combination of these result in the data lake idea.

The ugly truth though is that there is no one-click install for an enterprise architecture, and while patterns are emerging that hold true across industries everyone has their hurdles to overcome. The usual suspects are legacy systems, data silos, data warehouse dependencies, and change management issues.

How can an organization break up its data silos, and reimagine its infrastructure including the data warehouse and untangle legacy systems? The people in the organization working with entrenched systems and technologies have no spare time to learn or visibility of most new patterns and technologies. Hiring the experience and talent is nearly impossible. It is expensive and scarce, and due to the novelty senior stakeholders in organizations often don't know what to look exactly for in the new hires.

The consequence is a thriving professional services industry. Yes, Hadoop may be the answer to parts of your problems but choosing technology is not the first problem; it is somewhere in the middle. So you know you need Hadoop and out of desperation ask the vendor for advice but you also know that they clearly will favor their solution, which in itself may or may not be part of your solution. Most stakeholders are desperate to find trustworthy and experienced partners who can help them translate their domain challenges and organizational needs to future-proof, coherent and executable plans leveraging technology. Ideally showing immediate success with small cost to generate buy-in and help change thinking organization-wide.

The product and service strategy is an interesting topic to watch evolve since distributors are expanding both. There is demand for professional services, and it increases with the growing capabilities of the platform. What the successful business model around Hadoop will look like in a few years remains an unanswered question.

Related post

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Slideshows
What Digital Transformation Is (And Isn't)
Cynthia Harvey, Freelance Journalist, InformationWeek,  12/4/2019
Commentary
Watch Out for New Barriers to Faster Software Development
Lisa Morgan, Freelance Writer,  12/3/2019
Commentary
If DevOps Is So Awesome, Why Is Your Initiative Failing?
Guest Commentary, Guest Commentary,  12/2/2019
White Papers
Register for InformationWeek Newsletters
Video
Current Issue
The Cloud Gets Ready for the 20's
This IT Trend Report explores how cloud computing is being shaped for the next phase in its maturation. It will help enterprise IT decision makers and business leaders understand some of the key trends reflected emerging cloud concepts and technologies, and in enterprise cloud usage patterns. Get it today!
Slideshows
Flash Poll