13 Big Data Vendors To Watch In 2013

The well-documented shortage of workers with advanced, data-science skills has sent nearly every organization on a mad dash to acquire the capability or be left behind.

Some have responded by trying to hire data scientists from the small, existing, expensive pool of talent. Others have sent current employees back to school, taking advantage of a blossoming number of university-level courses in data management and advanced analytics geared toward IT professionals already working in industry.

Unfortunately, neither approach is likely to fill the need. As Gartner recently estimated, only about one-third of global demand for big data-related jobs will be met.

But there is a third path: Outsourcing.

[ Read Outsourcing's New Reality: Choice Beats Cost. ]

"Look, [data scientists] are an extremely scarce resource," said Narendra Mulani, global managing director for Accenture Analytics in a phone interview. "Ideally, you should have it in-house, but is that feasible?"

On the other hand, Mulani and several other experts interviewed about the pros and cons of outsourcing the data-science function had strikingly similar advice. Although all said some aspects can be outsourced, all cautioned it was a mistake to think of these services as a commodity, given the degree of industry- and company-specific knowledge needed to derive actionable insights from complicated data models.

Moreover, every expert stressed the need for knowledge transfer. "You cannot completely outsource it forever," said Anjul Bhambhri, VP of big data at IBM in a phone interview.

Benefits Of Outsourcing

By far the top benefit of working with an outside organization is speed, said a number of people.

"The pros of outsourcing are clear -- you can get the results faster, and don't have to hire hard-to-find and expensive data scientists," Gregory Piatetsky-Shapiro, an analytics and data mining consultant and editor of KDnuggets, said in an email. Outsourcing "could be entirely appropriate for smaller companies or companies where data is not the main focus or gives them competitive advantage," he said.

A subtler advantage of using an outsider is the chance to examine the data with fresh eyes, without old assumptions or bad habits. "People internally look at the data same way, and it’s sometimes harder to step back, to walk away from history and process and comps," said Brooke Niemiec, divisional VP of customer relationship marketing and loyalty at JCPenney in a phone interview.

And unlike the inside groups, which might be bogged down with ad-hoc requests, an outsider has time for "exploratory analysis," said Niemiec, whose internal team of data analysts numbers around 20 people, including a Ph.D. statistician. "I feel I’m in a fortunate position," she said.

Even so, JCPenney outsources some parts of its data analysis while it works on "operationalizing" data science and driving it through the organization, said Neimiec. The goal is to spend less time reporting and much more time on "interpreting, exploring and communicating" insights from the data, she explained.

Niemiec acknowledged the political realities inside organizations. "You should want to own your major, strategic initiatives," she said about big data and the insights that can flow from it. "But you might also want third-party validation."

Some outsourcing is inevitable because assembling a team entirely in-house is unrealistic, said David Steier, director of information management for business consulting firm Deloitte in a phone interview. "Frankly, a lot of companies have no choice," said Steier, who told InformationWeek previously that data science is a team effort.

These teams will need functional, industry and horizontal skills, as well as technology and interface-design expertise, he said, adding that in conjunction with using outside help, every organization should "find those pockets of hidden talent" -- people with quantitative skills.

A final plus of outsourcing is the opportunity to scale the analysis workload up and down, depending on unique conditions. "Many organizations had huge [marketing] campaigns around the Olympics," said Accenture’s Mulani about a number of large, complicated, time-sensitive marketing analytical projects his company helped out on. Nevertheless, these were one-off projects. "We see that all the time," he said. Outsourcing Downsides

"Whenever you outsource to a partner something that involves deep intelligence about the business, you're putting part of your brain outside your body," said Chuck Densinger, partner and chief customer intelligence officer at customer intelligence company Aginity in a phone interview. But even with contract and service arrangements, he said, "You're not building a capability. You're not institutionalizing it. And you're beholden to that partner."

IBM's Bhambhri somewhat echoed this opinion. "Fundamentally we believe you cannot just outsource the whole job," she said. But her rationale is less about worries a partnership will go sour than the idea that data scientists are emerging as "organizational change agents." These people, she said, not only extract insights by asking hard, data-intensive questions, they must communicate them back to the business owners, who'll need to take action. "You can't take this whole discipline and outsource it, because you need that bridge between IT and the line of business," she said.

Bhambhri and others also shared the opinion that it is a mistake to treat data science as a one-off project, given that business-relevant data is always changing. "This is an integral part to how you're gaining new insights," she said. For the same reason, she doesn't believe relying on outsiders makes sense, long-term.

Every expert said the real downside of outsourcing is that an outsider might not have the necessary domain knowledge to pose the right questions. Sometimes this can be as basic as different taxonomy, such as the definition of "new customer." For this reason, all the experts emphasized the need to work closely with the outside data scientist or analytics partner, making sure that both sides understand and agree about the assumptions used in the model.

"These models aren’t static, aren’t a black box," said Eric Hills, chief evangelist and senior VP at price optimization solutions company Zillant. The model’s parameters need to be understood by the client, he said, adding, "We bring them up to speed, and train them on the model."

"Recognize your organization’s culture and understand that analytics and data management needs to be an integral part of an organization’s culture," Pankaj Kulshreshtha, senior VP and business leader of smart decision services at business process management company Genpact. "Without it, a lot of your initiatives and investments may not yield the right results," he said in an email interview.

Partnering Up

"Many of our clients are looking for a strategic partner," Accenture’s Mulani said. The work will typically result in new business models or intellectual property, "and you don’t want to leave that to be used by a competitor."

Like Accenture, IBM stresses its services include training and knowledge transfer. As Bhambhri put it: "When we say IBM helps companies with their big-data journey, it’s not just tools and technology. A big part is to train the organization so that when we leave, they have the right tools, technology and processes to do it themselves. A big part of our value-proposition is training."

And look for a partner with relevant domain expertise, not simply experience with big data or statistics, said Genpact’s Kulshreshtha. "Ensure that your analytics vendor has the experience providing custom solutions which can address your unique business situation," he said. "Most often, companies develop standard tools and technologies for solving analytics problems."

Storing and protecting data are critical components of any successful cloud solution. Join our webcast, Cloud Storage Drivers: Auto-provisioning, Virtualization, Encryption, to stay ahead of the curve on automated and self-service storage, enterprise class data protection and service level management. Watch now or bookmark for later.