Microsoft Azure Data Lake Offers Enhanced Analytics Tools - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Cloud
News
9/28/2015
02:05 PM
Connect Directly
Twitter
LinkedIn
Google+
RSS
E-Mail
50%
50%

Microsoft Azure Data Lake Offers Enhanced Analytics Tools

An expansion of Microsoft's Azure Data Lake will include a new analytics service, and it will be ready for public preview by the end of the year.

10 Cloud Storage Options For Your Personal Files
10 Cloud Storage Options For Your Personal Files
(Click image for larger view and slideshow.)

Microsoft is building on its cloud offerings with an expanded Azure Data Lake, arriving with analytics tools designed to simplify big data, and with a new query language.

We first learned about the Azure Data Lake when Microsoft first announced it at the Build conference back in April. The data repository handles data of any size, type, and speed. It eliminates the complexities of processing and storing data while it makes it easier for businesses to get up and running with analytics.

The Azure Data Lake Store, as it has been renamed, will store structured, semi-structured, and unstructured data without forcing application changes as data scales. Data located in the Data Lake Store can be securely shared. It is also accessible from sensors connected to the Internet of Things.

[Office 2016, Windows 10 in China, and more from Microsoft's last week.]

According to a blog post published Sept. 28, the Azure Data Lake Store supports development of big data solutions through a variety of languages and frameworks. The new store works with the Hadoop Distributed File System (HDFS), so Hadoop tools like Hortonworks, Cloudera, and MapR can get the needed data for processing.

Microsoft also today announced Azure Data Lake Analytics, a cloud-based data processing and analytics service. The tool is built on Apache YARN. It scales instantly according to the power needed for each job. It's also cost-efficient; customers only pay for jobs when those jobs are running.

Azure Data Lake Analytics includes U-SQL, a new and scalable query language built on the same runtime that powers Microsoft's big data systems. With U-SQL, users can process queries to analyze data located in the Azure Data Lake Store, as well as information stored on SQL Servers in Azure, Azure SQL Database, and Azure SQL Data Warehouse.

T.K. "Ranga" Rengarajan, corporate vice president of data platforms at Microsoft, acknowledges how developers and data scientists struggle to successfully use existing technologies for big data.

"Code-based solutions offer great power, but require significant investments to master, while SQL-based tools make it easy to get started but are difficult to extend," he wrote on Microsoft's TechNet blog. "We've faced the same problems inside Microsoft and that's why we introduced U-SQL, a new query language that unifies the ease of use of SQL with the expressive power of C#."

(Image: Microsoft)

(Image: Microsoft)

Both the Azure Data Lake Store and Azure Data Lake Analytics will be available in preview later this year, Microsoft reports.

Microsoft adds the Azure Data Lake is supported by Azure Data Lake Tools for Visual Studio, which have been designed to foster an integrated development environment across the Azure Data Lake. It's also supported by Hadoop ISV applications spanning security, governance, data preparation, and analytics that can be deployed from the Azure Marketplace.

Ready today is HDInsight, the Apache Hadoop-based series included in Azure Data Lake that works with analytics services like Hive, Storm, HBase, and Spark. Managed clusters on Linux are now generally available, Microsoft reports.

Kelly Sheridan is the Staff Editor at Dark Reading, where she focuses on cybersecurity news and analysis. She is a business technology journalist who previously reported for InformationWeek, where she covered Microsoft, and Insurance & Technology, where she covered financial ... View Full Bio

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Phileaux
50%
50%
Phileaux,
User Rank: Apprentice
11/2/2015 | 8:24:42 AM
Re: Microsoft is too late. SQL and all other programming languages are over.
Yes, and Microsoft Access 1.0 will eliminate the need for DBAs and programmers in 1992. 

To date, every prediction that some technology will "end the need for specialists" has resulted in massive adoption and massive increase in the need for specialists. 
Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Author
9/28/2015 | 6:58:18 PM
The new 'open source' Microsoft
Microsoft's adoption of open source code as the basis for some of its analytics initiatives is impressive. And it's fair to say it would fall behind if it weren't adopting pieces like YARN and interfacing to Hadoop systems. Something worth watching: in addition to what it adopts, how much does it give back?
Slideshows
10 Ways to Prepare Your IT Organization for the Next Crisis
Cynthia Harvey, Freelance Journalist, InformationWeek,  5/20/2020
News
IT Spending Forecast: Unfortunately, It's Going to Hurt
Jessica Davis, Senior Editor, Enterprise Apps,  5/15/2020
Commentary
Helping Developers and Enterprises Answer the Skills Dilemma
Joao-Pierre S. Ruth, Senior Writer,  5/19/2020
White Papers
Register for InformationWeek Newsletters
The State of IT & Cybersecurity Operations 2020
The State of IT & Cybersecurity Operations 2020
Download this report from InformationWeek, in partnership with Dark Reading, to learn more about how today's IT operations teams work with cybersecurity operations, what technologies they are using, and how they communicate and share responsibility--or create risk by failing to do so. Get it now!
Video
Current Issue
Key to Cloud Success: The Right Management
This IT Trend highlights some of the steps IT teams can take to keep their cloud environments running in a safe, efficient manner.
Slideshows
Flash Poll