10 Lessons Learned By Big Data Pioneers - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Healthcare // Analytics
News
8/23/2011
12:03 PM
Doug Henschen
Doug Henschen
Slideshows
Connect Directly
Google+
LinkedIn
Twitter
RSS
E-Mail
50%
50%

10 Lessons Learned By Big Data Pioneers

How can you prepare for the big data era? Consider this expert advice from IT pros who have wrestled with the thorny problems, including data growth and unconventional data.
Previous
1 of 11
Next


What does it take to make the most of big data, as in tens, if not hundreds of terabytes of information? That depends on your needs and priorities. Ad-delivery firm Interclick found a fast platform that helps it be more productive while also delivering near-real-time insight. Harvard Medical School learned that data can grow even when obvious measures such as patient counts and years of data studied remain constant. comScore, the digital-media measurement giant, has twelve years of experience taking advantage of data compression by way of a column-store database. In fact, it uses sorting techniques to optimize compression and reduce processing demands.

Yahoo, eHarmony, Facebook, NetFlix, and Twitter have discovered that Hadoop is an ideal, low-cost platform for processing unstructured data. This open-source project is not just for Internet giants, however. JPMorgan Chase and other mainstream businesses are also taking advantage of Hadoop. And as data supplier InfoChimps has discovered, Hadoop is fast maturing, with a growing selection of add-on and helper applications available to support deployments.

Keep in mind that not all big-data deployments are measured by total scale. Linkshare, for instance, only retains a few months worth or data, but each day it loads and must quickly analyze tens of gigabytes, so it's a big deployment measured on an interday scale. Perhaps the most important lesson detailed in this image gallery is to heed Richard Winter's advice to pay attention to all six dimensions of data warehouse scalability. Only then can you formulate an accurate request for proposal, test for the most demanding needs, and make appropriate technology investments that will meet long-term needs.

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Previous
1 of 11
Next
Comment  | 
Print  | 
More Insights
Slideshows
Strategies You Need to Make Digital Transformation Work
Joao-Pierre S. Ruth, Senior Writer,  11/25/2019
Commentary
Enterprise Guide to Data Privacy
Cathleen Gagne, Managing Editor, InformationWeek,  11/22/2019
News
Watch Out: 7 Digital Disruptions for IT Leaders
Jessica Davis, Senior Editor, Enterprise Apps,  11/18/2019
White Papers
Register for InformationWeek Newsletters
Video
Current Issue
Getting Started With Emerging Technologies
Looking to help your enterprise IT team ease the stress of putting new/emerging technologies such as AI, machine learning and IoT to work for their organizations? There are a few ways to get off on the right foot. In this report we share some expert advice on how to approach some of these seemingly daunting tech challenges.
Slideshows
Flash Poll