10 Big Predictions About Big Data

Big Data Talent War: 10 Analytics Job Trends

Will big data be a force for good or evil within a decade? Will humanity find new and innovative ways to analyze, visualize, and extract value from massive and growing data sets, or will we become overwhelmed by information that's simply too abundant to manage effectively?

These are just a few of the questions the Pew Internet Center's Internet & American Life Project and Elon University asked more than 1,000 Internet "experts," including educators, business executives, pundits, scientists, and other tech industry observers. "The Future of Big Data" survey posed a series of thought-provoking questions centered on one main theme: How will big data influence our lives in 2020?

The issue concerns government leaders as well. The White House Office of Science and Technology Policy announced in March the Big Data Research and Development Initiative, a plan by six federal agencies to spend more than $200 million to develop new tools to access, structure, and pull meaning from massive volumes of data.

With as murky a term as "big data" is, it's no surprise the responses to the Pew survey were varied. Optimists and pessimists offered their thoughts on the state of data analysis within a decade.

[ Will big data be good or bad for your company? Read one point of view: Why Big Is Bad When It Comes To Data. ]

We've posted some of the more thought-provoking responses below. Respondents were free to agree or disagree with each statement, and explain why. The full set of survey predictions is available here.

Pew survey statement: "By 2020, the use of Big Data will improve our understanding of ourselves and the world."

Sean Mead, director of analytics at Mead, Mead & Clark, Interbrand, believes big data may be the next tech boom: "Large, publicly available data sets, easier tools, wider distribution of analytics skills, and early stage artificial intelligence software will lead to a burst of economic activity and increased productivity comparable to that of the Internet and PC revolutions of the mid to late 1990s."

"Big data is the new oil," wrote Bryan Trogdon, an entrepreneur and user-experience professional. "The companies, governments, and organizations that are able to mine this resource will have an enormous advantage over those that don't."

Survey statement: "Nowcasting, real-time data analysis, and pattern recognition will surely get better."

Google chief economist Hal Varian agrees that real-time forecasting has a bright future: "I'm a big believer in nowcasting," he wrote. "Nearly every large company has a real-time data warehouse and has more timely data on the economy than our government agencies. In the next decade we will see a public/private partnership that allows the government to take advantage of some of these private-sector data stores. This is likely to lead to a better informed, more pro-active fiscal and monetary policy."

Survey statement: "The good of big data will outweigh the bad. User innovation could lead the way, with "do-it-yourself analytics."

Marjory S. Blumenthal, associate provost at Georgetown University and adjunct staff officer at RAND, sees the pros and cons of advancements in data analysis tools and techniques. "Do-it-yourself analytics will help more people analyze and forecast than ever before. This will have a variety of societal benefits and further innovation. It will also contribute to new kinds of crime," Blumenthal wrote. Survey statement: "In the end, humans just won't be able to keep up."

Jeff Eisenach, managing director of Navigant Economics LLC, a consulting business, and formerly a senior policy expert with the U.S. Federal Trade Commission, agrees: "Big data will not be so big. Most data will remain proprietary, or reside in incompatible formats and inaccessible databases where it cannot be used in 'real time.'"

Survey statement: "Take off the rose-colored glasses: Big data has the potential for significant negative impacts that may be impossible to avoid."

Marcia Richards Suelzer, senior analyst at Wolters Kluwer, sees potential risks in real-time data analysis: "We can now make catastrophic miscalculations in nanoseconds and broadcast them universally. We have lost the balance inherent in 'lag time.'"

[ What are the key issues when it comes to big data? Read Oracle Big Data Study Shows Longtime Pain. ]

Some respondents feared the motives of governments and corporations, organizations with the most data and the greatest incentive to exploit it.

John Pike, director of GlobalSecurity.org, wrote: "The world is too complicated to be usefully encompassed in such an undifferentiated Big Idea. Whose 'Big Data' are we talking about? Wall Street, Google, the NSA? I am small, so generally I do not like Big."

And an anonymous survey respondent offered this bleak, Orwellian view of big data's future: "Data aggregation is growing today for two main purposes: National security apparatus and ever-more-focused marketing (including political) databases. Neither of these are intended for the benefit of individual network users but rather look at users as either potential terrorists or as buyers of goods and services."

Survey statement: "The rich will profit from big data and the poor will not."

"The collection of information is going to benefit the rich, at the expense of the poor," wrote Brian Harvey, a lecturer at the University of California-Berkeley. "I suppose that for a few people that counts as a positive outcome, but your two choices should have been 'will mostly benefit the rich' or 'will mostly benefit the poor,' rather than 'good for society' and 'bad for society.' ... And yes, I know about farmers in Africa using their cell phones to track prices for produce in the big cities. That's great, but it's not enough."

Some respondents offered a more comprehensive view of the future of big data.

Jerry Michalski, founder and president of Sociate and consultant for the Institute for the Future, pointed out big data's potential to feed humanity's dark side:

"So the best-intentioned of humans will try to use big data to solve big problems, but are unlikely to do well at it. Big ideas have driven innumerable bad decisions over time. Think of the Domino Theory, eugenics, and racial superiority theories--even survival of the fittest. These all have led us into mess after mess."

And its bright side:

"There are a few bright spots on the horizon. When crowds of people work openly with one another around real data, they can make real progress. See Wikipedia, OpenStreetMap, CureTogether, PatientsLikeMe, and many other projects that weren't possible pre-Internet. We need small groups empowered by big data, then coordinating with other small groups everywhere to find what works pragmatically."

New innovative products may be a better fit for today's enterprise storage than monolithic systems. Also in the new, all-digital Storage Innovation issue of InformationWeek: Compliance in the cloud era. (Free with registration.)