Transforming An Antiquated Business Intelligence Process - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Data Management
08:06 AM
Rich Wagner
Rich Wagner

Transforming An Antiquated Business Intelligence Process

The personal story of how one executive transformed an unused business intelligence system.

12 Ways To Connect Data Analytics To Business Outcomes
12 Ways To Connect Data Analytics To Business Outcomes
(Click image for larger view and slideshow.)

Five years ago, I was leading IT strategy and innovation for a Fortune 500 global chemical company. One of my tasks was to improve our business intelligence, but I had a tough problem. I noticed key decision makers weren't viewing BI reports. Managers used the reports to monitor past performance and track their results against key performance indicators, but executives were not using them at all. I needed to rethink my reports and give executives something they could use.

When I asked the CFO how we could improve our reports, I realized we were doing things all wrong. The charts and graphs our business intelligence tool produced contained internal historical data – events our decision makers and finance team could not change. Instead, our CFO needed forward-looking demand drivers specific to our company – predictive insights that our finance team could actually act on. That was the moment I realized the most important information was where our business was going, not where it had been.

The true business drivers for the company required a combination of internal metrics and external micro- and macro-economic factors. Without combining these metrics to create a clearer picture of performance, the C-suite deemed our reports useless.

An antiquated process

I soon realized that this problem wasn't specific to our company. But, like most companies, a new idea is approached very cautiously and often is met by internal resistance. I knew I first had to build and test my theory to prove this kind of approach would indeed improve our ability to forecast.

I decided that this was important enough to do on my own time to avoid being trumped by other internal projects and day-to-day tasks. Once I could demonstrate the validity of this approach, I knew it was only a matter of finding the right person inside the organization who would champion this integration. So I began researching the issue, meeting with the chief economist of a chemical association who confirmed that we were indeed missing the leading external signals of chemical demand.

At the time, corporations that wanted to analyze the impact of external factors on their performance had to pay a heavy price for a skilled economist to use statistical modeling tools that were 20 to 30 years old and took months of data hunting and gathering followed by tedious analysis. The process was largely based on trial and error – guessing which external factors might impact your business, and then running manual tests to determine if that guess was correct.

Plus, because external data is constantly changing, these models were out of date as soon as they were developed. During a severe economic crisis, global companies were struggling to understand the fast-moving economy, and without the right set of tools, the problem was only getting worse.

(Image: Wavebreakmedia/iStockphoto)

(Image: Wavebreakmedia/iStockphoto)

Real-time data correlation

Over the last 10 years, extremely valuable global data on economic conditions, consumer behavior, and other useful data sets have become more readily available. The difficult process was building an infrastructure that could sift through millions of data sets to identify leading factors related to a company’s internal performance. This challenge is why more companies aren't harnessing the power of external data. The metrics are available in mass quantities, but how do businesses make sense of them all?

I constructed an in-memory correlation, pattern-matching, and forecasting engine that gathered third-party data from millions of data sets. It calculated if and how each external factor specifically impacted each internal performance metric, and to what exact degree. The end result was fact-based models of leading indicators that explained how much loss or gain a business would experience depending on its specific external performance drivers – such as the average amount of rain, or an increase in real estate prices in particular regions.

This solution improved our company's forecast accuracy for our largest division and provided an automated, systematic, and repeatable platform for a process that had always been manual, tedious, and often overlooked. Once we implemented this process, our executive team was able to perform market analysis in just a few hours; a process that used to take weeks and months to configure. Ultimately, we were able to confirm or deny several long-held assumptions about our business performance. For instance, we were able to quantify one specific external factor to determine exactly how long it would take to impact our business, and by how much. In other cases, we learned that some factors didn't affect our business as much as we believed, and were able to concentrate on those that truly mattered. These insights were integral in preparing the board and investor meetings.

[ And here is another transformation story. Read NFL CIO: Football's Digital Transformation. ]

Asking Why

As is the case with most large companies, not every department adopted this new solution over night. Yet, for our finance team and executives, the insight gleaned from this predictive analytics process quickly became a key part of the planning process. These changes wouldn't have happened if I hadn't asked myself, Why? Why weren't our executives using our reports? Why did we only operate using internal data? Why have we been using the process for years?

Questioning the validity of a widely used business intelligence process was not easy. It required that I spend a lot of my personal time looking for answers, but questioning processes that have been in place for years is where true innovation starts. While an entrepreneur may have to work long nights and go through rounds of failed attempts, in the end, the most rewarding part of innovation is not only seeing your hard work pay off, but also knowing that you've solved a major pain point for global businesses. Today, businesses can proactively address challenges and opportunities, ultimately improving bottom line profits. It all started with asking, Why? which transformed an antiquated process from the inside out.

Does your company offer the most rewarding place to work in IT? Do you know of an organization that stands out from the pack when it comes to how IT workers are treated? Make your voice heard. Submit your entry now for InformationWeek's People's Choice Award. Full details and a submission form can be found here.

With an extensive background in IT strategy and innovation, Rich Wagner has seen first-hand the power that external big data can bring to a company's financial performance. Today, as president and chief executive officer at Prevedere, Rich helps industry-leading companies ... View Full Bio
We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Apprentice
2/25/2016 | 4:58:53 PM
Re: Sources of External Data
Social media is one of many sources of data at Prevedere. Although not typical causal factors in changes of demand in global chemical and industrial manufacturing(my background) it is extremely valuable for our retail and consumer goods customers forecast.


User Rank: Ninja
2/15/2016 | 5:46:08 AM
Sources of External Data
Social media is a good source for external data. However, constant monitoring is required if value is to be extracted by the data sources of social media. Research firms such as, Gartner and IDC, etc., is also a good source to interact, collaborate with the industry and gain external data that is of value. Software firms have also made it extremely easy for business to connect to IoT devices to extract data. 

I wonder if a major source of external data has been overlook here.
2021 Outlook: Tackling Cloud Transformation Choices
Joao-Pierre S. Ruth, Senior Writer,  1/4/2021
Enterprise IT Leaders Face Two Paths to AI
Jessica Davis, Senior Editor, Enterprise Apps,  12/23/2020
10 IT Trends to Watch for in 2021
Cynthia Harvey, Freelance Journalist, InformationWeek,  12/22/2020
White Papers
Register for InformationWeek Newsletters
The State of Cloud Computing - Fall 2020
The State of Cloud Computing - Fall 2020
Download this report to compare how cloud usage and spending patterns have changed in 2020, and how respondents think they'll evolve over the next two years.
Current Issue
2021 Top Enterprise IT Trends
We've identified the key trends that are poised to impact the IT landscape in 2021. Find out why they're important and how they will affect you.
Flash Poll