Real-Time Analytics is Spreading and Getting Easier - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Data Management
News
3/22/2018
08:00 AM
Connect Directly
Twitter
RSS
E-Mail
50%
50%

Real-Time Analytics is Spreading and Getting Easier

Is your organization in production with real-time analytics and event stream processing? Are you doing a pilot of the technology? Here's a look at where the technology is today, along with some of the more popular use cases.

Everything today is moving faster than it used to, from Amazon delivery times (2-day shipping to same-day shipping) to software development cycles (Agile instead of waterfall). We can look things up in an instant on Google, or via voice on Amazon Alexa, instead of looking in a book or going to the library. So it's hardly surprising that more organizations are looking to get faster with their analytic insights, and even point them at event stream data as well.

Gartner VP Distinguished Analyst W. Roy Schulte is a specialist in the area of event stream processing and real time analytics, and he provided an update on their growing adoption rates at the Gartner Data and Analytics Summit in Grapevine, Texas this month.

(Image: Pixabay)

(Image: Pixabay)

He said that while this type of work used to be typically processed in racks at the data center, today this processing has spread out to other areas, too, including at the edge, in branch offices, in factories, and sometimes even in trucks and on trains. InformationWeek reported on QVC's efforts to incorporate real-time analytics into its own operations last year.

(In a show of hands at Schulte's session at the Gartner event this month, about a quarter of attendees indicated that they have real-time event stream processing in production, and another few indicated they have it in pilot.)

Schulte said that this type of work has spread to regional cloud and corporate data centers and is proliferating in open source and hybrid open source/commercial deployments.

"The sophistication of the applications are richer and more powerful," he said. "We are seeing things like machine learning operating on top of event streams."

For organizations that are just getting started with analytics on event streams, Schulte said, "It's important to know that you already have a lot of data you can use." For instance, there is data coming from business systems, order records, streams of insurance claims, ATM information, transactions, and searches. Data can come from IoT devices, RFID, geolocation devices, and other instruments.

"Maybe I have days or weeks' worth of data. I correlate that with the fact that the machine finally broke. So I work to create a model that looks for factors that impact failure." -- Gartner's Roy Schulte

For those new to the concept, Schulte noted that an "event" is defined as a state change or something that happens, and "event streams" are a sequence of event data objects in order. An "event" is defined as anything that happens, or a state change. An "event stream" is a sequence of event data objects in order. Event data is not state data like a bank account balance or a truck location, and it is not reference data like a price list or a customer name.

When organizations think about real-time analytics, they are thinking about getting data in motion (as opposed to data at rest), Schulte said. There are a few common use cases for this kind of data, he said.

The classic use case for this is situational awareness. For instance, your real-time dashboard is updated. Maybe it monitors your trucking fleet or contact center or Twitter feed or stock portfolio.

Another use case is decision automation for sense-and-respond applications. For instance, the analysis of the data triggers an alert to possible fraud, or to a market opportunity, or to a weather change. That alert can be part of an automated system that then adjusts machine settings. This kind of decision automation can also be put into play in IT systems that generate alerts and automate actions.

A third use case applies analytics to historical event streams, and Schulte said that anything that is older than 15 minutes is considered a historical event stream.

"Maybe I have days or weeks' worth of data. I correlate that with the fact that the machine finally broke," he said. "So I work to create a model that looks for factors that impact failure."

Schulte said that open source software is a growing influence in the event stream processing space. There are 10 open source ESP platforms, and eight open source streaming data integration platforms. In addition, vendors embed open source technology in hybrid open/closed products. Among the big names are Apache Storm, Apache Spark and Spark Streaming, Kafka, R, Python, and Spark ML and MLib.

But organizations can expect the whole effort to get easier in years to come as more vendors incorporate ESP into existing tools. 

Virtually all IoT platforms incorporate some kind of event stream processor, and many other products from technology vendors now also include event stream processors, too. Schulte further noted that a growing number of BI and analytics platforms, including Microsoft Power BI,  now incorporate event stream processors.

"Expect to see more of these traditional data discovery and data reporting tools adding event stream processing," he said.

Jessica Davis has spent a career covering the intersection of business and technology at titles including IDG's Infoworld, Ziff Davis Enterprise's eWeek and Channel Insider, and Penton Technology's MSPmentor. She's passionate about the practical use of business intelligence, ... View Full Bio

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Slideshows
Top-Paying U.S. Cities for Data Scientists and Data Analysts
Cynthia Harvey, Freelance Journalist, InformationWeek,  11/5/2019
Slideshows
10 Strategic Technology Trends for 2020
Jessica Davis, Senior Editor, Enterprise Apps,  11/1/2019
Commentary
Study Proposes 5 Primary Traits of Innovation Leaders
Joao-Pierre S. Ruth, Senior Writer,  11/8/2019
White Papers
Register for InformationWeek Newsletters
State of the Cloud
State of the Cloud
Cloud has drastically changed how IT organizations consume and deploy services in the digital age. This research report will delve into public, private and hybrid cloud adoption trends, with a special focus on infrastructure as a service and its role in the enterprise. Find out the challenges organizations are experiencing, and the technologies and strategies they are using to manage and mitigate those challenges today.
Video
Current Issue
Getting Started With Emerging Technologies
Looking to help your enterprise IT team ease the stress of putting new/emerging technologies such as AI, machine learning and IoT to work for their organizations? There are a few ways to get off on the right foot. In this report we share some expert advice on how to approach some of these seemingly daunting tech challenges.
Slideshows
Flash Poll