Big Data Talent War: 10 Analytics Job Trends

Summer is prime time for widespread power outages, which often last for hours or even days. India last month endured what some news reports have called the worst blackout ever, one that left 600 million people without power for two days in a row. Smaller blackouts have lasted far longer, of course, but haven't affected nearly as many people.

Developed countries with seemingly more reliable electrical grids have experienced major outages as well. The Northeast blackout of 2003, for instance, left an estimated 55 million people in the U.S. and Canada without electricity for up to 16 hours.

The causes of power blackouts might vary, but the implementation of big data technologies can help prevent them from starting. According to Bert Taube, director of business development for Versant, a data management software company, energy utilities encounter many of the data management problems that other organizations find increasingly common: the complexity of juggling a wide variety of data sources, many of which need to be processed and analyzed very quickly.

The so-called "five V's" of big data--volume, velocity, variety, verification, and value--apply to the energy sector's data management issues well. The solution? A data measurement and analytics system that allows electrical utilities to process information in real time, and to take appropriate actions to avoid power shortages.

"A hurricane, you can't really do anything about it. You can evacuate people. But with a power grid, if you put a measurement system in place, you can impact the decision-making accordingly," said Taube in a phone interview with InformationWeek.

The 2003 North American blackout, for instance, was caused by several preventable factors, including miscommunication and a lack of coordination and automation, said Taube, Versant's resident smart-grid guru.

[ When big data is victim of power outages: Amazon Cloud Outage Causes Customer To Leave. ]

"A number of 345-kilowatt lines tripped hours before the blackout happened, but nobody could really check because there wasn't a centralized way of coordinating things," Taube said. "On top of that, a number of different systems failed. And when systems fail, and they fail together, a lot of bad things happen at the same time."

Today, however, real-time measurement systems can provide synchronized, real-time measurements of multiple points on the grid.

"If you have these measurements, and you do data processing in real time, you can determine metrics that will tell you if you're actually getting [into] such a scenario as a blackout," said Taube.

The phasor measurement unit, or synchrophasor, is a key component of an electrical grid monitoring system. It measures electrical waves on a grid and allows real-time measurements of multiple remote points. "You can calculate a whole set of metrics to make sure you realize what's really going on, and then make proper decisions that prevent these blackouts from happening," said Taube.

During the 2003 blackout, North American power grids had no phasor measurement units (PMUs) in place, according to Taube. Today, 150 PMUs are in use, with hundreds more planned.

Power companies over the years have developed a reputation as stodgy, inefficient bureaucracies--organizations resistant to innovation. But that's not necessarily true, Taube said. In fact, several U.S. utilities are particularly innovative. One is Southern California Edison, which Taube describes as a "very good utility. It's very innovative. When you go to their labs, you will see they invest a ton of money into real-time digital simulators."

Detroit Edison and Duke Energy earned Taube's praise as well for their work in smart-grid technologies.

"Utilities are not all the same. Some are very innovative, others are not," he said. "The innovative utilities realize there's a paradigm shift toward using a model-based (simulator) approach to deal with the grid issue. They realize this will give them a big increase in stability and dynamic deployment."

See the future of business technology at Interop New York, Oct. 1-5. It's the best place to learn about next-generation technologies, including cloud computing, BYOD, big data, and virtualization. Register today with priority code YLBQNY02 and save up to $300 on passes with early-bird pricing.