4 Big Data Essentials For Startups

Tech Hygiene: 10 Bad Habits To Break

Big-data products are generally targeted at large enterprises, and for good reason. They can be enormously expensive to initiate and operate, and therefore out of reach for the average startup or small business.

That's changing, but fledgling firms need to answer some hard questions before embarking on a data-driven strategy. These include: Do I need a big data system? And what insights do I hope to gain from it?

At last week's Techweek conference in Santa Monica, Calif., a two-day event connecting tech entrepreneurs with investors, Sean Anderson, manager of data services at cloud computing company Rackspace, offered some solid advice in his talk, "How Startups Can Leverage Big Data." Anderson's talk covered a wide range of data-related topics, but we've distilled a few takeaway points for companies debating the merits of a big-data initiative.

1. Explore the three Vs, and don't forget C.
Big data is often defined by the three Vs: volume, velocity, and variety. Anderson recommends adding a fourth letter to the formula: C, for complexity.

Volume, of course, refers to the sheer capacity of data. If the volume of information you're putting into your database or data store is breaking the system, it's a good time to explore a big-data technology. The same wisdom applies to the velocity of incoming data, a growing concern in the era of near-real-time analytics. As for variety, your database should allow you to query multiple data types, including unstructured data such as audio, video, and email text.

[It's time to make your gift-giving list. See 10 Smart Tech Toys For Kids.]

"There are plenty of legacy technologies that can scan and analyze text very, very easily," said Anderson. "But as time goes on... the types of file formats and the things we're trying to analyze and process get increasingly more complex."

That's where complexity comes into play.

"The things we're trying to stick into a database are changing," Anderson said. In this age of social engagement and content sharing, "we want to start understanding and getting insights from stuff that's not just text."

2. Volume is overrated.
Startups mustn't assume that big-data technologies are the exclusive domain of enterprises wrestling with petabytes or exabytes of data.

"The reality is, when [we] talk to enterprise customers -- Fortune 100 customers that are starting their big-data initiatives -- they don't have petabytes of information. Some of them don't have terabytes of information. Some are just trying to get good, reliable insights from gigabytes of information."

Takeaway: Don't use volume as the only factor when deciding if big data technology is right for your company.

3. Big data is hard.
The big data landscape is "changing at a feverish pace," said Anderson. He cited a recent Gartner survey that showed, while 73% of enterprise respondents have invested or plan to invest in big data in the next two years, these investments haven't led to an associated boost in organizations reporting deployed big-data projects. In fact, just 13% of respondents reported having "big data projects deployed to production." That's a notable increase from 8% a year earlier, but it also suggests that much of today's big-data work involves pilot programs, strategy development, and experimental projects, according to Gartner.

Takeaway: It's difficult for the smartest organizations, including startups, to keep up with the latest big-data innovations.

4. If it ain't broke...
Maybe you don't need a big-data initiative at all. If your current database system handles all of your data needs today and in the foreseeable future, why fix what doesn't need fixing?

"We... would be remiss to go to any of you with systems that are functioning properly and tell you to deploy a big-data solution," said Anderson. "It's disruptive to your business, and it may not be the thing that you need."

He added: "There are some great stories out there of companies that spent months evaluating big-data technologies, only to find out that their current technology... does everything they need."

Just 30% of respondents to our new Big Data and Analytics Survey say their companies are very or extremely effective at identifying critical data and analyzing it to make decisions, down from 42% in 2013. What gives? Get the The Trouble With Big Data issue of InformationWeek Tech Digest today. (Free registration required.)