Big data worries people for two opposite reasons: it's either a big threat or a big lie.
The "big threat theory" was on display during a panel discussion Friday at MIT, where the answer to the question "Is big data turning into bad data?" seemed to be yes.
Noam Chomsky, perhaps America's best-known cranky intellectual, harrumphed about the potential for a Stasi-like surveillance state. "We can be confident that any system of power -- government, Amazon, Google -- will try to use the best available technology to control and dominate and maximize their power. And they'll want to do it in secret," Chomsky said.
By invoking the Stasi (the East German secret police, for those born after 1989), Chomsky clearly isn't all that worried about big data per se. He's worried about powerful people running amok and using big data to help them do it. As for big data itself, he sounds like a typical corporate CEO: "Our problems aren't lack of access to data; it's understanding it."
[ Are human conflicts preventing your organization from using big data to its fullest potential? Read Big Data Has A Trust Problem. ]
Chomsky's fears of the US state using big data to ill effect for ordinary citizens have not been borne out by what's in the Snowden papers, said journalist and author Barton Gellman, who is one of the three people to have access to the Snowden papers and who served on the MIT panel with Chomsky. But Gellman nonetheless says we need to be concerned about the amount of data available to companies and government, and about their own ability to keep things from us.
"We become more and more transparent to government and large institutions and they become more opaque to us," Gellman said. Big data, then, could become "a one-way mirror." Smart companies will want to avoid backlash by making clear what they do with the data they collect, which is in fact the consumer's data.
According to Columbia University professor Saskia Sassen, the winners in the push for big data aren't governments but technology firms. That's pushing toward a different problem for big data: that it's perpetuated by vendor hype.
In fact, Harper Reed, President Obama's former campaign CTO, recently called big data "BS," describing it as just "a plot by vendors to sell more stuff." While he is a fan of what we can do with data, Reed told attendees at a recent education conference, "The 'big' there is purely marketing. This is about you buying expensive servers and whatnot."
There will always be some need for expensive servers "and whatnot." But most businesses don't want to spend on high-end gear and cutting-edge applications. Personally, I hate the 80/20 rule, where 80 percent of the way is good enough -- but for a lot of things in business it rings true. Ask me about the impact of "good enough" on journalism, and I'll rant about the corrosive effects of commoditized content. But most people think such content is good enough. For big data, "good enough" means data, Excel-style.
That's why we're seeing tools like SiSense, an on-chip analytics tool, gain popularity. AT&T introduced a similar tool called Nanocubes. People want to gain access to data. They are going to want access through the tools they know best, and those tend to be the tools that cost least.
Such has been the cycle of computing forever. As Tracy Kidder wrote in The Soul of a New Machine (Back Bay Books, 2000), things start out scarce, large, and expensive: "One big machine... Often it lay behind a plate glass window, people in white gowns attending it, and those who wished to use it did so through intermediaries. Users were like supplicants. The process could be annoying."
That was 1981. He could be writing about Hadoop clusters and big data today. People don't like to be annoyed. At this point, they don't like to feel cut off from information, especially now that they know just how much of their information is being hoarded.
IT doesn't have too many hot new things to offer, but big data could be one of them. That is, if the CIO can turn it into something that's not a big threat -- or a bunch of BS.
There's no single migration path to the next generation of enterprise communications and collaboration systems and services, and Enterprise Connect delivers what you need to evaluate all the options. Register today and learn about the full range of platforms, services and applications that comprise modern communications and collaboration systems. Register with code MPIWK and save $200 on the entire event and Tuesday-Thursday conference passes or for a Free Expo pass. It happens in Orlando, Fla., March 17-19.