Hadoop Ecosystem Evolves: 10 Cool Big Data Projects - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Data Management // Software Platforms
News
2/26/2016
10:06 AM
Jessica Davis
Jessica Davis
Slideshows
Connect Directly
Twitter
RSS
E-Mail

Hadoop Ecosystem Evolves: 10 Cool Big Data Projects

In the 10 years since developers created Hadoop to wrangle the challenges that came with big data, the ecosystem for these technologies has evolved. The Apache Software Foundation is teeming with open source big data technology projects. Here's a look at some significant projects, and a peek at some up-and-comers.
2 of 11

Hadoop
Hadoop is really the flagship technology for open source big data. It grew out of a side project at Yahoo when developers needed a way to store and process the massive amount of data they collected with their new search engine. The technology was eventually contributed to the Apache Software Foundation. Today there are three major distributions from commercial companies -- Cloudera, Hortonworks, and MapR. One of Hadoop's creators, Doug Cutting, recently spoke with InformationWeek about the growth of his baby. We also recently put together a look at Hadoop's history. 

(Image: The Apache Software Foundation)

Hadoop

Hadoop is really the flagship technology for open source big data. It grew out of a side project at Yahoo when developers needed a way to store and process the massive amount of data they collected with their new search engine. The technology was eventually contributed to the Apache Software Foundation. Today there are three major distributions from commercial companies -- Cloudera, Hortonworks, and MapR. One of Hadoop's creators, Doug Cutting, recently spoke with InformationWeek about the growth of his baby. We also recently put together a look at Hadoop's history.

(Image: The Apache Software Foundation)

2 of 11
Comment  | 
Print  | 
Comments
Newest First  |  Oldest First  |  Threaded View
HM
50%
50%
HM,
User Rank: Moderator
2/29/2016 | 4:10:26 PM
Big data
Jessica, very insightful article. Another open source technology to mention is HPCC Systems from LexisNexis, a data-intensive super computing platform for processing and solving big data analytical problems. Its integration with Hadoop extends further capabilities providing a complete solution for data ingestion, processing and delivery. In fact, both libhdfs and webhdfs implementations are available. More at http://hpccsystems.com/h2h
pfretty
100%
0%
pfretty,
User Rank: Ninja
2/29/2016 | 1:55:37 AM
Evolution becomes even more valuable with augmentation
Hadoop is amazing technology with the ability process in a cost effective manner. However, its when organizations embrace augmentation tools that the real value surfaces.  This was echoed in a recent IDG Research Services study where IT leaders identified key benefits of embracing augmentation tools. 

 

Peter Fretty, IDG blogger for SAS
nasimson
50%
50%
nasimson,
User Rank: Ninja
2/28/2016 | 7:23:14 PM
Such mammoth projects
From Spark to so many other things, this list is awesome. I didn't know that Hadoop was the underlying framework behind such mammoth projects. Seems like an impressive portfolio for a ten year old tech.
Slideshows
Strategies You Need to Make Digital Transformation Work
Joao-Pierre S. Ruth, Senior Writer,  11/25/2019
Commentary
Enterprise Guide to Data Privacy
Cathleen Gagne, Managing Editor, InformationWeek,  11/22/2019
News
Watch Out: 7 Digital Disruptions for IT Leaders
Jessica Davis, Senior Editor, Enterprise Apps,  11/18/2019
White Papers
Register for InformationWeek Newsletters
Video
Current Issue
Getting Started With Emerging Technologies
Looking to help your enterprise IT team ease the stress of putting new/emerging technologies such as AI, machine learning and IoT to work for their organizations? There are a few ways to get off on the right foot. In this report we share some expert advice on how to approach some of these seemingly daunting tech challenges.
Slideshows
Flash Poll