Accelerate Action with Real-time Analytics

More Data & Analytics Live at Interop ITX

We are just beginning to witness an analytics evolution, a shift away from historical batch reporting to real-time analytics for data-driven decision making while events are happening. In a fiercely competitive, digital world where speed to insight and action have become more critical than ever before, decision makers do not have the luxury of relying on data that can only get periodically updated.

Analyzing real-time data feeds with high throughput used to be immensely difficult, time consuming to set up and expensive to scale. In the past when project sponsors asked for real-time reporting, analytics professionals would prudently push back on information timing needs to level set expectations with regards to what could be reasonably delivered. Compromises on right-time dashboards that directly connected to relational databases or used relational online analytical processing (ROLAP) solutions were common.

Historically only a few industries such as security and financial services truly needed real-time analytics. Few could achieve it. Today we see far more use cases for right time, just-in-time and real-time analytics than ever before. With the rise of social media, internet of things (IoT), mobile applications and online transactions that generate large scale continuously streaming data, real-time analytics applications are being adopted in a variety of industries.

Technologies that can capture, process and analyze streaming data in real-time without much data loss have significantly progressed. Serverless architectures are now capable of processing data streaming from a plethora of data sources to integrate necessary data in an instant and generate outputs with the lowest possible processing time. Queries can be executed in a continuous fashion enabled by a high degree of parallelism and automatic optimization. Horizontal scaling is completely automatic, elastic, and managed by cloud providers.

With visual point-and-click user interfaces, even novice technical professionals can configure a truly scalable system to ingest massive volumes and bursts of streaming or event data. The setup process is as simple as defining input and output data destinations. Within these user-friendly modern streaming analytics solutions, machine learning algorithms and alerts can also be applied. Mainstream dashboards and business applications now can render data as it flows in with moving visualizations and constantly updated metrics.

Numerous real-time analytics platforms and dashboards can easily, cost-effectively render live data feeds, display interactive streaming visualizations and compute alerts. Most solutions allow for processing and acting on both live and historical data with intuitive interfaces and SQL-like queries on incoming streams. In my Interop ITX session: Best Practices for Developing Real-time Dashboards, I will showcase Apache Storm, Kafka, Rickshaw D3.js, Signal R, Amazon Kinesis, TIBCO Stream Analytics, SAS Event Stream Processing, Zoomdata, Kinetica and IoT technical solution architecture design patterns for real-time analytics.

Real-time analytics powered by the cloud is within reach for everyone to enjoy. Cloud streaming analytics offerings have enabled shorter development cycles, dramatically lowered programming costs, simplified maintenance, reduced hardware and operational expenses, and provided faster time-to-value.

New data management challenges

Note real-time analytics platforms are only part of the solution. You also need to embrace new analytics patterns for ingesting, loading, storing, dividing and conquering huge volumes of event stream data. Extract, transform and load techniques of the past simply don’t scale.

Real-time analytics is an essential step in the digital evolution. The next step is triggering automated data-driven business workflows where real decisions are made.