Take Advantage of In-Memory Analytics - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Software // Information Management
09:44 AM
Connect Directly

Take Advantage of In-Memory Analytics

Fast analysis, better insight and rapid deployment with minimal IT involvement: these are among the benefits of in-memory analytics, but different products are appropriate for different environments. Read our in-depth report on in-memory technologies.

How It Works

As the name suggests, the key difference between conventional BI tools and in-memory products is that the former query data on disk while the latter query data in random access memory (RAM). When a user runs a query against a typical data warehouse, for example, the query normally goes to a database that reads the information from multiple tables stored on a server's hard disk (see "Query Approaches Compared," below).

Click Image To View Larger Version

Query Approaches Compared
With in-memory tools, all information is first loaded into memory. If the in-memory tool is server-based, an administrator may initiate the load process; if it's a desktop analysis tool, the user may initiate the process on his or her workstation. Users then query and interact with data loaded into the machine's memory. Accessing data in memory is literally millions of times faster than accessing data from disk. This is the real, "speed-of-thought" advantage that lives up to all the hyperbole.

In-memory BI may sound like caching, a common approach to speeding query performance, but in-memory products don't suffer from the same limitations. Those caches are typically subsets of data, stored on and retrieved from disk (though some may load into RAM). The key difference is that the cached data is usually predefined and very specific, often to an individual query; but with in-memory tools, the data available for analysis is potentially as vast as an entire data mart. Another approach to solving query performance problems is for database administrators (DBAs) to analyze a query and then create indexes or aggregate tables in the relational database. When a query hits an aggregate table, which contains only a subset of the data, it may scan only a million records rather than the hundreds of millions of rows in a detailed fact table.

Yet one more route to faster performance is to create a MOLAP (Multidimensional Online Analytical Processing) database. But whether it's tuning or MOLAP, these paths to user-tolerable performance are laborious, time consuming and expensive. DBAs are often versed in performance tuning for transaction systems, but the ability to tune analytic queries is a rarer skill set often described as more art than science. What's more, indexes and aggregate tables consume costly disk space. In-memory tools, in contrast, use techniques to store the data in highly compressed formats. Many vendors and practitioners cite a 1-to-10 data-volume ratio when comparing in-memory systems to traditional, on-disk storage.

So while users benefit from lightning-fast queries, in-memory BI is also a big win for BI system administrators. Newell Rubbermaid, for example, says its BWA deployment has eliminated significant administrative time that was formerly required to tune queries. "The queries were so fast [after the in-memory deployment], our users thought some data must have been missing," says Rajeev Kapur, director of business analytics at Newell Rubbermaid. Yet the performance improvement didn't involve analysis of access paths or creation of indexes or aggregate tables as were previously required.

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
2 of 3
Comment  | 
Print  | 
More Insights
IT Careers: Top 10 US Cities for Tech Jobs
Cynthia Harvey, Freelance Journalist, InformationWeek,  1/14/2020
Predictions for Cloud Computing in 2020
James Kobielus, Research Director, Futurum,  1/9/2020
What's Next: AI and Data Trends for 2020 and Beyond
Jessica Davis, Senior Editor, Enterprise Apps,  12/30/2019
White Papers
Register for InformationWeek Newsletters
Current Issue
The Cloud Gets Ready for the 20's
This IT Trend Report explores how cloud computing is being shaped for the next phase in its maturation. It will help enterprise IT decision makers and business leaders understand some of the key trends reflected emerging cloud concepts and technologies, and in enterprise cloud usage patterns. Get it today!
Flash Poll