10 Powerful Facts About Big Data

Gartner estimates that 26 billion Internet of Things devices will be installed by the year 2020. Will our networks be up to the task of supporting them? With so many connected devices sending data back and forth simultaneously, we must start building applications in new ways so they'll scale efficiently while still providing continuous data performance.

The Internet of Things creates a flow of data, from the "thing" to a broker, which very often must be constant to be of real value. Imagine a smart traffic system, for example: Cameras would monitor roads to detect accidents, construction, poor weather, and other maladies of the morning commute. The information could be automatically sent to in-car dashboards to alert drivers about delays and suggest alternate routes. But it would defeat the purpose if this system didn't respond to new data in real time.

[Learn to counter trends, embrace sustainability, launch your startup and more at the Designers of Things conference's @dothingcon's #3DPrinting track]

It's not only the rising total volume of concurrent connections that increases the load on backend infrastructure and bandwidth. The critical nature of some of these systems also ramps up the pressure on networks to be connected constantly.

The city of South Bend, Ind., for example, installed 100 Internet-enabled sensors in its sewage system to help officials better understand water usage and flows. City managers now can monitor the entire system in real time and control individual pipe valves based on demand. From 2006 through 2012, the city invested $6 million and saved $120 million. But imagine if the system went down during the worst possible time, such as a severe rainstorm or flood. South Bend's Board of Public Works would be unable to adjust flows, and the system would become just as inefficient as the old system was.

Companies seem ill-prepared to close the gap between rising demand for bandwidth and the available capacity. Nearly 70% of respondents in a recent survey said they expected demand for WAN bandwidth to increase in the next year, but only 15% planned to expand network capacity.

Battling dead zones
The importance of peak performance and optimal application connectivity includes emerging markets and regions with low network connectivity and limited bandwidth. For example, to help reduce theft, Brazil has passed legislation mandating that all vehicles be equipped with real time tracking devices by the end of this year. But the idea will work only if the tracking technology installed in each car can deliver data over networks in real time -- otherwise, stolen vehicles won't be traceable. So for this ruling to be effective, it's imperative that even regions with low connectivity or bandwidth be able to receive theft notifications.

Bandwidth shortcomings aren't confined to emerging markets. The problem is encountered daily by train operators managing their rolling stock. For instance, whenever a train travels into a "dead zone" (i.e., when it goes out of signal range behind a hill or in a valley), the on-board application that monitors the track ahead for signals can take time either to completely reload or update all of the data that was missed when the connection is restored.

What can we do?
With increasing technological advancements, Internet of Things applications can incorporate tactics such as delta message delivery. In delta message delivery, after the data state has been delivered, only the deltas (changed data) are sent. For example, a flight arrival app used by airport operations and ground crews would show only status changes such as updated arrival times, gate changes, and baggage carousel changes, instead of sending the entire data state of the flights. This reduces the amount of data distributed across the network, which allows it to support more users at the same time.

Another tactic is more intelligent data recovery in the event of a network outage or a traveler passing through a dead zone. Instead of reloading the entire data state and waiting for every missed update, intelligent data applications can determine the latest and most relevant data to push through. This ensures that users not only have the most up-to-date information, but that they get it as quickly as possible since smaller amounts of data can be transmitted more quickly once connectivity is restored.

Unless we develop Internet of Things apps to cope with imperfect networks, the coming "connected age" may be more frustrating than helpful.

New standards, new security, new architectures. The Cloud First stars are finally aligning for government IT. Read the Cloud Hits Inflection Point issue of InformationWeek Government Tech Digest today.