Big Data Success Essentials: Tech, People, and Process

There's been plenty of talk in recent months about why we can't overlook the role of people in an analytics initiative. Conference speakers, software providers, and bloggers have looked beyond the software and the data to hammer home the importance of good analytics professionals and how systems have to get business people to appreciate data.

It's great how thought leaders are recognizing that all the world's complex circuitry, big data, and brilliant algorithms are as useful as a bucket of mud if people -- business leaders or customers -- can't benefit from the what the data shows.

However, it's not just people that have been forgotten far too often. Let's not lose sight of the importance of the amorphous blob that spreads through every organization: process.

Process can be our friend. When it gets in our way, it's our foe. Done right, it's what delivers data to where it's needed. In some cases the need for process is obvious, as it is in an Internet of Things implementation.

Take the now-common example of an IoT application that monitors the status and maintenance requirements of a remote piece of machinery. If the app identifies a potential maintenance need it issues a yellow alert. If the remote machine goes down, the alert goes out as red. The role of process: It identifies who should be made aware of the alert and what action has to be taken. Without process, all you have are pretty little lights, and a dead machine.

The need for process is less apparent in something like a sales analysis. The system design has to factor in who gets which reports, and also when issues have to get escalated. So, when sales in Region 5 are stable or growing, weekly reports may only need to go to the regional representatives and managers. But when the sales figures -- good or bad -- pass certain thresholds alerts go to those higher in the corporate food chain. Deciding who gets what and when, and perhaps what they should do, is all part of process.

All of us have heard of examples where data died on a spreadsheet because it didn't get to the right people, or because nobody ever told those people what to do with it. Think of poor machine that crashed because the maintenance alert never got to the maintenance worker. I remember one case that was cited some months ago where customer service app highlighted "at risk" customers, but nobody told the agents in the field.

We've also heard of the failures where a department head brought in an analytics professional to gather data without knowing what type of problem they hoped to solve. That's a process failure, and likely leads to a people failure. Equally bad is when that department head neglects to let others in the group or in other relevant groups know that they have a wealth of data. It may be intentional or it may be simple neglect, but the result is the evil data silo.

Process can't simply be an afterthought. It has to be built in from the start, from the "let's see what the data says about..." stage. Then it has to adapt as the analytics team and the business leaders learn. Think of that process as being iterative or organic, evolving as new data emerges, spawning new uses.

Having terabytes of data and amazing technology might seem nice, but it's meaningless if you don't put the data to work. We have good people. Process is about getting that data into their hands so they can take appropriate action.