Executives Push Big Data Projects, Not Sure Why

Big Data Talent War: 10 Analytics Job Trends

There are a lot of challenges to be overcome before most companies will be able to take full advantage of a big-data facility, or even a single project, according to a recently published study by VAR and reseller-training organization CompTIA. Few companies are run by executives who understand either the technicalities or purpose of big data well enough to help their companies profit from it.

The study includes results from two surveys conducted in July. One polled end users, IT staff, and business executives involved with technical projects. The other was made up of responses from 435 VAR and reseller executives.

Of the business and IT executives responding, only 37% said they were very or mostly familiar with the concept of big data--but not to the extent of being able to design or supervise the building of a big-data facility or even a single project.

At a recent London conference of senior-level, IT-savvy business managers, not one was willing to hazard a genuine guess at how much more profitable their companies could be with free access to 100 times as much data on customers as it already had, according to Harvard Business Review columnist Michael Schrage, who posed the question to the audience.

[ Related: How To Find Strategic Advantage From Big Data. ]

The more the CEOs thought about how to roll out a big-data project, the more likely it seemed that big data would cost a lot of money, take a lot of time and development work--and still face organizational barriers designed to make it easier to manage people and revenue streams than the data they generate, said Schrage.

"Neither the quantity nor quality of data was the issue," Schrage writes in his blog. "What matters is how--and why--vastly more data leads to vastly greater value creation."

Making decisions about how to reinforce, eliminate, or reduce obstacles is the province of boards of directors and top corporate officers, rather than the middle managers and IT managers usually involved with nascent big-data products, noted Schrage. Without help to clear the hierarchy of authority and data ownership, any big-data project is more an experiment in analytics than an effective way to make business decisions, he writes.

Not every company can or should use big-data analytics, according to Tim Herbert, CompTIA's VP of research. Those that want to use it anticipate a range of benefits, but don't seem to know how to build and adapt to the technology necessary to get them. Only a third of the CompTIA survey's business respondents said their companies were exactly where they should be on the big-data learning curve. And just 20% said their companies do a good job of analyzing Web traffic patterns. Fifteen percent said they were good at analyzing email marketing results, and 12% said they were good at analyzing their efforts to use social media.

Although the business-unit managers were slightly less likely to know anything at all about big data, they had a better grasp than other respondents on the kinds of questions big data could answer or trends it could identify--if the tools and organization were created first to allow it to be used.

Fifty-nine percent of respondents wanted to create or improve the ability to see subtle trends in their own data; 67% wanted to learn to understand the relationships between two variables.

Half the companies responding to the survey plan to increase training of existing employees; 32% plan to hire new people with the necessary skills, and to think about whether to add more training.

"Basic work needs to be done before many companies are ready for a big data initiative," Herbert said. That doesn't mean executives who think they don't have the skills in-house to handle big data can just forget about it, he said. Big data is today's cloud, the most-hyped technology in the computer business, one that every company needs to have a position on or risk being brushed off or left behind.

"Not every business will need a big data strategy," Herbert said. "But just about every business will need to effectively aggregate, store, manage, and analyze the data they do have, regardless of its volume, velocity or variety." The evidence that shows benefits, especially in marketing and customer targeting, from big data is clear to the point of being repetitive, according to a special report from Bloomberg BusinessWeek.

Real-world examples of companies successfully using big data are far more thin, though case studies are available describing audience-selection efforts at Spotify and LastFM, live-traffic analysis from satellite navigation providers, and real-time analysis of operational data within European professional soccer, according to TechWeekEurope.

One business sector eager to reap the benefits of big data is retail, where legendarily thin profit margins and high transaction volumes could make good use of any increase in marketing efficiency. Only a third of retailers, however, have succeeded in building even a working big-data proof-of-concept app, according to a study by retail analyst firm Edgell Knowledge Network.

In retail, identified by other analysts as one of the markets that could benefit most dramatically from better handling of its vast stores of data on every item purchased by every consumer, 80% of respondents had heard of big data and knew it was a technology with which they would need to become involved. Only 47%, however, said they understood the kinds of changes their organizations would have to undergo to profit using big data analytics.

Business "priorities, budgets and ROI" are the biggest inhibitors to using big data, according to the Edgell survey. Also, as with the CompTIA survey, the very executives who could approve big-data projects said they did not know enough about big data or what to do with it to feel comfortable giving the go-ahead.

Nevertheless, 60% of companies responding said they are in the midst of at least one major big-data project--whether they understand why or not.

See the future of business technology at Interop New York, Oct. 1-5. It's the best place to learn about next-generation technologies including cloud computing, BYOD, big data, and virtualization.