5 Big Wishes For Big Data Deployments

Most organizations that implement big data platforms expect to derive significant value from their investment. But nearly half of these firms aren't achieving the level of value or return on investment (ROI) that they had expected.

According to a new study by Wikibon, an open-source research firm that competes with Gartner and Forrester, the ROI of these big-data projects is proving to be a big letdown for most enterprises.

"In the long term, they expect $3 to $4 return on investment for every dollar. But based on our analysis, the average company right now is getting a return of about 55 cents on the dollar," said Jeffrey F. Kelly, Wikibon principal research contributor, in a phone interview with InformationWeek.

[ Here's what to look for in a big data wrangler. Read 3 Key Skills Of Successful Data Scientists. ]

Wikibon bases its findings on multiple information sources, including conversations with big data vendors and service providers, feedback from the Wikibon community, and results from a survey of nearly 100 "big data practitioners," the firm said.

Forty-six percent of survey respondents reported that they've realized only "partial value" from their big data deployments, while 2% called their deployments "total failures, with no value achieved," the report states.

So what's the problem? Wikibon identified three key reasons for companies' inability to achieve maximum ROI from big data.

1. Lack Of Skilled Big-Data Experts

The data scientist shortage is a well-chronicled phenomenon and one that might persist for some time.

"In terms of the lack of skilled practitioners, I don't see that changing anytime soon," said Kelly.

A company's existing staff, such as a database administrator (DBA) with years of Oracle experience, probably lacks the skills to manage big data technologies like Hadoop, he added. In the short term, this dilemma provides an opportunity for big-data services firms to fill the gap.

2. Immature Technology

Big-data tools are in their infancy. They require refinement for use by a wider range of business workers -- not just highly trained data scientists -- a problem that many software developers are working to solve.

3. Lack Of A Compelling Need

Enterprises often invest in big-data projects without tying these efforts to specific and measurable business applications.

"In such cases, largely driven by IT departments, enterprises begin amassing large volumes of data in Hadoop, which is sometimes made available to data scientists and business analysts for exploratory analysis, but that otherwise sits underutilized," says the Wikibon report.

"A lot of these deployments are driven by IT departments, which sometimes are looking to offload some of the workload from their existing relational systems," said Kelly. "Basically they load in a lot of data, and make it available to their data scientists and analysts to do some exploratory analysis. You've got a lot of experimenting going on, but no real business application tied to it."

To overcome these big-data obstacles, the Wikibon report advises businesses to consider professional services organizations, cloud services or both. It's also important to clearly define a project's goals before you begin.

"Generally we recommend that enterprises start with small, strategic [projects]. Pick a very discrete use case, something that's going to be fairly easy to measure," said Kelly. "Do it in an area that's strategic to your business rather than a peripheral use case."

He added: "Most of the successful projects we've seen are not initiated by IT, but are driven more by line of business departments, either marketing or finance."

Making decisions based on flashy macro trends while ignoring "little data" fundamentals is a recipe for failure. Also in the new, all-digital Blinded By Big Data issue of InformationWeek: How Coke Bottling's CIO manages mobile strategy. (Free registration required.)