As the data piles up, managing and analyzing these data resources in the most optimal way become critical success factors in creating competitive advantage and strategic leverage.

To address these challenges, companies are hiring data scientists. A data scientist is a relatively new job profile and combines a unique skillset consisting of a well-balanced mix of quantitative, programming, business, communication, and visualization skills. It speaks for itself that these profiles are hard to find in today’s job market. Universities are massively jumping on the big data and analytics bandwagon and are offering various masters programs in big data and analytics to close the gap.

Where to Get the SkillsetsThe shortage of skilled talent and data scientists in Western Europe and the US has triggered the question of whether to outsource analytical activities. This need is further amplified by competitive pressure on reduced time to market and lower costs. Companies need to choose between insourcing -- building the analytical skillset internally either at the corporate or business line level -- or outsourcing all analytical activities. Or they might go for an intermediate solution whereby only part of the analytical activities are outsourced. The dominant players in the outsourcing analytics market are India, China, and Eastern Europe with some other countries (e.g. Philippines, Russia, South Africa) gaining ground as well.

Various analytical activities can be considered for outsourcing, ranging from the heavy lifting grunt work (e.g. data collection, cleaning and pre-processing), analytical platforms (hardware and software), training, and education, to the more complex analytical model construction, visualization, evaluation, monitoring, and maintenance. Companies may choose to grow organically and start by outsourcing the analytical activities step by step, or immediately go for the full package of analytical services. It speaks for itself that the latter strategy has more risk associated with it and should thus be more carefully and critically evaluated.

Strategic ThinkingDespite the benefits of outsourcing analytics, it should be approached with a clear strategic vision and critical reflection with awareness of all risks involved. First of all, the difference between outsourcing analytics and outsourcing traditional ICT services is that analytics concerns a company’s front end strategy, whereas most ICT services are part of company’s backend operations.

Another important risk is the exchange of confidential information. Intellectual property (IP) rights and data security issues should be clearly investigated, addressed and agreed upon. Moreover, all companies have access to the same analytical techniques, so they are only differentiated by the data they provide. Hence, an outsourcer should provide clear guidelines and guarantees about how intellectual property and data will be managed and protected ( e.g., using encryption techniques and firewalls), especially if the outsourcer collaborates with various companies operating in the same industry sector. Consider an example of two banks, A and B, working with outsourcer XYZ to develop their analytical credit risk models. Bank A invested in state of the art data quality solutions whereas bank B did not. Outsourcer XYZ can now use the high quality data from bank A to build high performing analytical credit risk models, and sell those to bank B as well, thereby diluting the competitive advantage of bank A. This danger is further amplified by the many mergers and acquisitions witnessed in the outsourcing sector.

Furthermore, many of these outsourcers face high employee turnover due to intensive work schedules, the boredom of performing low level activities on a daily basis, and aggressive headhunters chasing those hard to find data science profiles. This attrition problem seriously inhibits the continuity of the partnership and a long-term thorough understanding of a customer’s analytical business processes and needs.

Plan for ChangeAnother often cited complexity concerns the cultural mismatch (e.g., time management, different languages, and local versus global issues) between the buyer and outsourcer. Exit strategies should be clearly agreed upon. Many analytical outsourcing contracts have a maturity of three to four years. When these contracts expire it should be clearly agreed upon how the analytical models and knowledge can be transferred to the buyer to ensure business continuity.

Finally, the shortage of data scientists in the US and Western Europe will also apply, and might even be worse, in the countries providing outsourcing services. These countries typically have universities with good statistical education and training programs, but their graduates lack the necessary business skills, insights, and experience to make a strategic contribution with analytics.

Given the above considerations, many firms currently adopt a partial outsourcing strategy, whereby baseline, operational analytical activities such as query and reporting, multidimensional data analysis, and OLAP are outsourced, while the advanced descriptive and predictive analytical skills are developed and managed in house.

This blog originally appeared on Bart Baesen's Ku Leuven website Data Mining Apps. Bart offers self-paced E-learning courses on Advanced Analytics in a Big Data World and Credit Risk Modeling in conjunction with SAS.