Big Data Poses Challenges For Federal Agencies

Big Data Talent: 6 Ways To Snag The Best

While government officials say that advanced analytics have improved the speed of decision-making in many different federal agencies, these leaders also express plenty of concern about a range of issues surrounding big data. That's according to new survey commissioned by global IT services provider Unisys.

While it's not intended to be a complete picture, the Unisys survey, which culls responses from about 100 government officials, provides a glimpse inside the state of big data and analytics programs in the federal government.

[Looking for more on how government is using big data? Check out 10 Government Innovations Your Business Can Use.]

The US government has increased its big data play over the last few years. The Obama administration in 2012 committed $200 million to leveraging big data to drive actionable insights. In February the White House announced the appointment of US Chief Data Scientist DJ Patel, the first person with that title.

How Government Is Using Big Data

More than 60% of agencies are using big data to reduce costs including capital and operating expenses. For example, Unisys said, agencies can find incorrect invoices or payments that may have been made erroneously. Taking it a step further, they can use root-cause analysis to determine what caused those errors and correct flaws in the system.

In addition, 55% of agencies are using big data to improve their IT security, according to the survey. For instance, some are using analytics to detect advanced threats by automating the identification of inconsistencies in machine data. That process gives agencies better insight into how attacks happened. It can even enable them to stop ongoing attacks, Unisys said.

But US government executives are grappling with many of the same issues and concerns as their counterparts in the private sector are when it comes to big data, according to the Unisys survey.

For instance, one of the survey's findings is that while some agencies are far along the path to realizing the benefits of big data, other agencies "seem somewhat overwhelmed about the expertise and expense required to make the most of big data."

Organizations in the private sector frequently face the same problem.

Staff Shortages

In spite of the potential for cost savings and better security, only six in ten agencies have big data initiatives or plans in place. Those without big data plans cited the lack of staff resources -- specifically business and data analysts, data scientists, engineers, and data architects. One in three government officials surveyed said that it was difficult to locate experts with the necessary experience, but 68% said their agencies are hiring more data analysts. Half of the officials surveyed are in the market for a data analytics director.

That echoes a common theme of organizations everywhere looking to start big data projects. Data analytics skills are in demand, and it can be difficult to find the right people. This week IBM announced a new consulting practice to help close this gap among its clients.

IT Infrastructure Challenges

But there were a few other concerns that looked even bigger to agencies with existing big data projects. Of them, 73% said they were concerned about the strain the projects put on their existing IT storage, computer, and networking infrastructures. These executives were worried that getting ready for these projects would mean a long and expensive infrastructure refresh. Civilian agencies were more concerned than defense agencies about this issue.

Other big concerns included the challenge of creating analytic systems that could provide actionable insights (for 60%) and also the cost of implementing big data analytics (for 70%).

How will federal agencies address these challenges going forward? Almost all, 98%, said they would either maintain or increase their use of outside consultants to work on these projects in the next 12 months.