Big Data Moves From Hype To Reality, CompTIA Finds - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Data Management // Big Data Analytics

Big Data Moves From Hype To Reality, CompTIA Finds

Organizations are moving past the hype and into actual value when it comes to big data and analytics implementation, according to a new survey by CompTIA. But challenges remain, including a skills gap and the struggle to wrangle the growing quantity of data generated.

14 Ways IoT Will Change Big Data And Business Forever
14 Ways IoT Will Change Big Data And Business Forever
(Click image for larger view and slideshow.)

Companies are moving to the next stage with big data -- past the hype and into broader adoption. But new challenges are on the horizon, such as how to master all that data. Those are a couple of several key findings in a recent Computing Technology Industry Association (CompTIA) study.

The industry organization surveyed 402 IT professionals for the report released in December and found that 72% of respondents said big data projects had exceeded their expectations, and about 75% said their businesses would be stronger if they could harness all of their data.

Companies "are trying to consolidate data and understand the structure of their data," said Seth Robinson, senior director for technical analysis at CompTIA, in an interview with InformationWeek. They are finding this out the hard way, because lax data policies in the past allowed departments and work units to "do their own thing" in IT, resulting in data silos. "Companies have to look at data differently now."

Data silos are a problem for 42% of IT professionals responding to the survey, up from 29% two years ago.

More Data Than Ever

Plus, more data is being generated than ever before. The CompTIA report cites another study by Cisco that forecast 2016's IP traffic passing the 1 zettabyte threshold. That is 1 billion TB of data. That number will double again in three years. Helping to drive the data glut, the price of storage has dropped from $1,000 per GB in 1995 to a few pennies today. So, squirreling away 100TB of data is not going to be a big deal, especially now that companies can rent storage in the cloud to suit needs.

[Algorithms are changing the world. But where did they start? Read Algorithms: Turning Data Into Products.]

How Organizations Use Data

Yet 63% of survey respondents said they rely on data for day-to-day operations, 61% said that they are more sensitive to data privacy issues, and 59% said that they want to use data to better understand their customers.

Respondents are evenly split when asked how and where they seek improvement in their operations. For example, 40% said they are doing well with real-time analysis, while 42% seek improvement. A full 39% of respondents want to be able to search all of their data across departmental lines, while 37% say they want to improve their ability to do so.

(Image: matdesign24/iStockphoto)

(Image: matdesign24/iStockphoto)

In 2015, 51% of all respondents said they were undertaking big data projects, while another 36% said they were planning to do so. This is up from 2013, when 42% of respondents said they were undertaking big data projects, and 46% said they were planning them.

Despite the desire to reap the fullest benefit from data, companies undertaking big data projects are not suffering setbacks. "They are not running into headaches or hidden costs," Robinson said. Companies may be meeting modest goals. "The metric may not be that obvious," he said. "Coming up with the right metric is part of being data savvy."

Skills Gap

Still, there is a skills gap that needs to be addressed. A total of 49% of respondents said they have sufficient data skills in-house to take care of big data projects. But 40% admit they are "moderately deficient." The biggest skills gaps are real-time analytics (reported by 42%), relational databases (41%), data security (36%), predictive analysis (35%), distributed storage (33%), and data mining (30%).

"Part of this is technical," Robinson observed. The tools needed to handle big data are new, so there may be insufficient experience to make full use of their potential. There is also an informal skill set needed as well, since data analysts need to "speak" business as well as understand the technology, he added.

Enterprises may turn to outsourcing to help fill this knowledge gap, but only if they can find the right skills there. "We heard companies saying when they go outside, they can't find the knowledge or the skills," Robinson said.

The report noted that third parties are still concentrating their expertise on component services and not end-to-end solutions.

Still, component expertise can become a building block toward providing a big data fix. Data backup, for instance, could lead to "having a conversation about the data," Robinson said, while installing a firewall can be a step toward greater data security.

"People have gotten past thinking it (big data) is hype," Robinson said. Companies are building on what they have learned, which in turn can lead them to more comprehensive big data solutions, he added. "We'll see that play out over this year."

**Elite 100 2016: DEADLINE EXTENDED TO JAN. 15, 2016** There's still time to be a part of the prestigious InformationWeek Elite 100! Submit your company's application by Jan. 15, 2016. You'll find instructions and a submission form here: InformationWeek's Elite 100 2016.

William Terdoslavich is an experienced writer with a working understanding of business, information technology, airlines, politics, government, and history, having worked at Mobile Computing & Communications, Computer Reseller News, Tour and Travel News, and Computer Systems ... View Full Bio

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Ninja
1/15/2016 | 11:58:37 PM
Re: Promising outlook
Kevin, what does "exceed expectations" mean? What were their expectations? Were they very low to begin with? It could mean a lot that three out of four respondents were pleasantly surprised by results, unless it doesn't. We need more info to get excited, right?
User Rank: Moderator
1/15/2016 | 9:48:26 AM
Nice perspective!
William, very nice perspective on Big Data. We are seeing an increase in businesses seeking specialized skills to help address challenges that arose with the era of big data. The open source HPCC Systems platform from LexisNexis helps to fill this gap by allowing data analysts themselves to own the complete data life cycle. Designed by data scientists, the programming language called ECL is declarative and expresses data algorithms across the entire HPCC platform. Their built-in analytics libraries for Machine Learning and BI integration provide a complete integrated solution from data ingestion and data processing to data delivery. HPCC Systems provides proven solutions to handle what are now called Big Data problems, and have been doing so for more than a decade. Their free online introductory courses allow for students, academia and other developers to quickly get started. For more info visit:
User Rank: Moderator
1/13/2016 | 10:54:03 AM
Maturity is steady
Big data as a whole has grown as a result of the hype, which is crucial considering the role analytics will play as we move towards an IoT environment. There is still a strong need for the queries to start coming from the business rather than IT and data scientists. This is going to take time to build the data culture. Peter Fretty, IDG blogger for SAS
User Rank: Author
1/12/2016 | 5:04:41 PM
Re: Promising outlook
We do have to bear in mind that the results here are based on the responses of only 402 people. It's possible that's not a large enough sample to really gauge big data trends for businesses in general. 
User Rank: Author
1/12/2016 | 5:02:51 PM
Re: It's already done.
@Ilya If you say it is, in fact, structured and then quote IBM saying it can process unstructured data, are you saying that IBM misrepresents itself?
User Rank: Apprentice
1/12/2016 | 2:17:52 PM
Promising outlook
Very useful findings here.  The data point that 72% of respondents said big data projects had exceeded their expectations implies we might be futher along the maturity curve than is commonly thought.  Using Gartner's model, this suggests we are moving past the trough of disillusionment toward the plateau of productivity.

10 Trends Accelerating Edge Computing
Cynthia Harvey, Freelance Journalist, InformationWeek,  10/8/2020
Is Cloud Migration a Path to Carbon Footprint Reduction?
Joao-Pierre S. Ruth, Senior Writer,  10/5/2020
IT Spending, Priorities, Projects: What's Ahead in 2021
Jessica Davis, Senior Editor, Enterprise Apps,  10/2/2020
White Papers
Register for InformationWeek Newsletters
Current Issue
[Special Report] Edge Computing: An IT Platform for the New Enterprise
Edge computing is poised to make a major splash within the next generation of corporate IT architectures. Here's what you need to know!
Flash Poll