10 Lessons Learned By Big Data Pioneers - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Healthcare // Analytics
News
8/23/2011
12:03 PM
Doug Henschen
Doug Henschen
Slideshows
Connect Directly
Google+
LinkedIn
Twitter
RSS
E-Mail
50%
50%

10 Lessons Learned By Big Data Pioneers

How can you prepare for the big data era? Consider this expert advice from IT pros who have wrestled with the thorny problems, including data growth and unconventional data.
Previous
1 of 11
Next


What does it take to make the most of big data, as in tens, if not hundreds of terabytes of information? That depends on your needs and priorities. Ad-delivery firm Interclick found a fast platform that helps it be more productive while also delivering near-real-time insight. Harvard Medical School learned that data can grow even when obvious measures such as patient counts and years of data studied remain constant. comScore, the digital-media measurement giant, has twelve years of experience taking advantage of data compression by way of a column-store database. In fact, it uses sorting techniques to optimize compression and reduce processing demands.

Yahoo, eHarmony, Facebook, NetFlix, and Twitter have discovered that Hadoop is an ideal, low-cost platform for processing unstructured data. This open-source project is not just for Internet giants, however. JPMorgan Chase and other mainstream businesses are also taking advantage of Hadoop. And as data supplier InfoChimps has discovered, Hadoop is fast maturing, with a growing selection of add-on and helper applications available to support deployments.

Keep in mind that not all big-data deployments are measured by total scale. Linkshare, for instance, only retains a few months worth or data, but each day it loads and must quickly analyze tens of gigabytes, so it's a big deployment measured on an interday scale. Perhaps the most important lesson detailed in this image gallery is to heed Richard Winter's advice to pay attention to all six dimensions of data warehouse scalability. Only then can you formulate an accurate request for proposal, test for the most demanding needs, and make appropriate technology investments that will meet long-term needs.

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Previous
1 of 11
Next
Comment  | 
Print  | 
More Insights
News
IT Spending Forecast: Unfortunately, It's Going to Hurt
Jessica Davis, Senior Editor, Enterprise Apps,  5/15/2020
Commentary
Helping Developers and Enterprises Answer the Skills Dilemma
Joao-Pierre S. Ruth, Senior Writer,  5/19/2020
Slideshows
Top 10 Programming Languages in Demand Right Now
Cynthia Harvey, Freelance Journalist, InformationWeek,  4/28/2020
White Papers
Register for InformationWeek Newsletters
Video
Current Issue
Key to Cloud Success: The Right Management
This IT Trend highlights some of the steps IT teams can take to keep their cloud environments running in a safe, efficient manner.
Slideshows
Flash Poll