Microsoft Releases Hadoop On Windows - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Software // Information Management
News
10/24/2012
07:25 AM
Connect Directly
Google+
LinkedIn
Twitter
RSS
E-Mail
50%
50%

Microsoft Releases Hadoop On Windows

Microsoft makes big data play with HDInsight Server, first beta release of Hadoop distribution for Windows operating system.

Big Data Talent War: 10 Analytics Job Trends
Big Data Talent War: 10 Analytics Job Trends
(click image for larger view and for slideshow)
The big data world knew it was coming, but now we know the official name of Microsoft's Hadoop software distribution: Microsoft HDInsight Server for Windows. The vendor announced the first preview release of the on-premises software on Wednesday at the Strata Conference in New York.

Microsoft also announced the third release of its Azure cloud-based Hadoop service, now known as Windows Azure HDInsight Service. Both were developed in partnership with Hadoop software developer and distributor Hortonworks.

HDInsight Server for Windows includes core Hadoop (meaning the Hadoop Distributed File System and MapReduce), plus Pig for MapReduce programming, Hive data query infrastructure, Hortonworks' recently introduced HCatalog table management service for access to Hadoop data, Scoop for data movement, and the Ambari monitoring and management console. All of the above have been reengineered to run on Windows and all are open-source components that are compatible with Apache Hadoop and are being contributed back to the community.

[ Want more on Hortonworks' software? Read Hortonworks' Hadoop Dilemma: Get Rich Giving Ideas Away. ]

The HDInsight Server is designed to work with (but does not include) Windows Server and Microsoft SQL Server. In the case of Windows, HDInsight is integrated with Microsoft System Center for administrative control and Active Directory for access control and security.

"We've spent tremendous engineering time integrating it with the Windows management platform, so we end up with enterprise-grade, enterprise-scale management, deployment, and integrated security with the Windows infrastructure environment," said Doug Leland, Microsoft's general manager of SQL Server Marketing, in an interview with InformationWeek.

The integration with the larger systems environment is a contrast with the Oracle Big Data Appliance, which hands all things Hadoop over to Cloudera's distribution of Apache Hadoop and proprietary Cloudera Manager administrative software (the appliance runs on Oracle Enterprise Linux, but deployment, admin, and user access is managed by Cloudera's software).

HDInsight Server for Windows also works with Microsoft's virtualization platform, including Hyper-V and the System Center virtualization fabric. "We're dramatically reducing what it takes to provision and manage either your head node or your data node," said Leyland. "We're creating templates that let you define what your head node and data nodes look like, and then we're using virtualization to dynamically spin up and provision nodes across however many instances that you want."

Microsoft is the first to deliver Hadoop on virtualized infrastructure, according to Leyland, but it won't be alone. Project Serengeti is an open-source effort designed to support rapid deployment, configuration, and management of a Hadoop cluster in a virtual environment. VMWare CTO Richard McDougall was slated to present on Serengeti at the Strata conference on Thursday.

The HDInsight (on-premises and cloud) connection to Microsoft SQL Server is all about business intelligence, and that starts with user-facing tools and components including Microsoft Excel, PowerPivot for Excel, and Power View. Few people don't have access to Excel, and it can handle data extracts from any Hadoop environment.

The PowerPivot add-in for high-scale in-memory data exploration and Power View component for data visualization are available as free downloads from microsoft, but they have to be licensed to run against Microsoft SQL Server through a client access license. In Office 2013, which is currently in preview release, PowerPivot and Power View are natively integrated with Excel.

How can you explore Hadoop data using Microsoft's Excel-based BI tools? There are currently two options: using Hortonworks' previously available Hive ODBC driver to access Hadoop data directly or using Scoop connectors to Hadoop (also previously available) to move data out of Hadoop and into the Microsoft SQL Server database or into SQL Server Analysis Services.

Microsoft also plans to make use of HCatalog metadata services, Leland said, but he was uncertain about what level of integration had already been achieved. Hortonworks partner Teradata is already using HCatalog, which is designed to enable users to define the structure and location of data within Hadoop to support fast and consistent access and analysis without actually moving data.

Microsoft's cloud-based HDInsight for Azure service, which is now in its third preview release, lets users spin up and deploy a Hadoop cluster within minutes, according to Leland. This service will compete with existing MapReduce services including Amazon Web Services' Elastic Map Reduce, which was introduced in April, 2009.

Microsoft did not disclose when the HDInsight Server for Windows Server and HDInsight For Azure service would be generally released, but it's Microsoft's usual practice to go through three community technology previews (each spanning several months) before making software generally available.

Since this is the first preview release of the on-premises software, which can be downloaded at no cost, it's a safe bet HDInsight Server for Windows won't be generally available before the second quarter of next year. The Azure cloud service is in the last preview stage, so it could be available by the end of this year or early next year.

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Slideshows
What Digital Transformation Is (And Isn't)
Cynthia Harvey, Freelance Journalist, InformationWeek,  12/4/2019
Commentary
Watch Out for New Barriers to Faster Software Development
Lisa Morgan, Freelance Writer,  12/3/2019
Commentary
If DevOps Is So Awesome, Why Is Your Initiative Failing?
Guest Commentary, Guest Commentary,  12/2/2019
White Papers
Register for InformationWeek Newsletters
Video
Current Issue
The Cloud Gets Ready for the 20's
This IT Trend Report explores how cloud computing is being shaped for the next phase in its maturation. It will help enterprise IT decision makers and business leaders understand some of the key trends reflected emerging cloud concepts and technologies, and in enterprise cloud usage patterns. Get it today!
Slideshows
Flash Poll