When it comes to outsourcing, everyone has a tale of woe. My favorite comes from a software vendor that outsourced a new development project: "They were amateurs and couldn't hit a deadline. It's also why we switched from PHP to Java. It was the wrong platform to start." Many will grumble in agreement, recalling their own horror stories.

However, bigger questions loom. Who picked the platform? (The unhappy customer did.) Did the customer check the work daily? (No.) Did it have automated status reporting? (It didn't.)

Outsourcing is a key part of every modern IT group. Problem is, we still don't seem to do it that well. Twenty-nine percent of the 530 business technology professionals responding to the InformationWeek Analytics 2010 Business of Outsourcing Survey have fired a vendor within the last 12 months. You can blame the partner--or grab a mirror.

Two big trends jump out from this year's survey of companies using IT outsourcing, and both speak to the importance of IT managing outsourcing better.

One is the growth of cloud computing and software-as-a-service initiatives--and the disturbing trend of IT trusting performance monitoring to the vendors. The other is the fact that IT outsourcing is moving up the stack, as vendors take over increasingly strategic functions. Nearly six of 10 IT shops outsource some critical function--management, engineering, or development; almost one-fourth keep executive and management functions in-house but look to outsource everything else. As companies rely more on outsiders, a lack of oversight, management, and even monitoring can have catastrophic consequences.

Our survey shows a continued rise in all types of outsourcing, everything from traditional hardware services and staffing to cloud applications and full-blown data center operations.

However, there are some serious levels of dissatisfaction. With end-user support and development of customer-facing applications, more than half of survey respondents say outsourcing has delivered lower quality. Cloud computing and SaaS get more favorable reviews, with the majority saying it has delivered better quality and 44% planning to expand use. However, there are problems there, too, with almost six out of 10 respondents relying on their cloud vendors to monitor their own performance.

Most IT shops have set the right goals for outsourcing: Freeing up staff for more strategic initiatives is the most-valued benefit, just above cost savings and better alignment of IT staff and costs with business trends. They're also worried about the right problems: unforeseen costs, communication problems, and the time required to manage subcontractors.

Cloud Growth Means Challenges

Cloud computing blurs the lines between what had been conventional outsourcing and internal operations, and it will test IT's management and control policies. But IT pros like what they're getting so far. Most companies (55%) using some kind of cloud computing or SaaS product think it's delivering better quality--with 37% citing that sweet spot of higher quality at lower cost. Interestingly, almost one-fifth say cloud/SaaS delivers better quality but at a higher cost. That higher cost story isn't one cloud computing providers usually tell, so it's a trend to watch closely.

Compare those findings to the dissatisfaction with the more mature category of outsourced end-user support: 59% of survey respondents say it's delivering lower quality than in-house support, including 13% who think it both costs more and provides lower quality. Just 28% say outsourcing improved support quality. So much for a great help desk. In customer-facing application development, over half think outsourcing has lowered quality. Cloud's relatively higher satisfaction helps explain why it's poised to grow.

Unfortunately, IT isn't preparing properly for cloud's growth, which could lead to some nasty surprises. Only 17% say they directly monitor the performance and uptime of all of their cloud and SaaS applications. A quarter monitor only mission-critical items, and a shocking 59% rely on their vendors to monitor themselves. It would seem logical that IT leaders could take all they've learned in the past decade of conventional outsourcing and use it to police the cloud. The problem?

"We don't see IT doing it all that well in the [conventional] outsourcing world," says David Rutchik, with the outsourcing consultancy Pace Harmon. Rutchik says outsourcing vendor management is talked about a lot, but it tends to be much more reactive than proactive.

Firing an outsourcing provider is the ultimate reaction--witness the 29% of IT shops that have done it in the past year. But try firing a SaaS provider after you've integrated the platform into your operations. Cloud can be a bit like a tick--a vendor can latch on easily to your organization and then is a pain to get rid of.

Cloud monitoring and management are critical but in some cases shouldn't be done by IT, Rutchik says. Sound like heresy? The reality is that SaaS providers often have a relationship with business units first--our SaaS research earlier this year found that IT organizations are driving the decision to use SaaS only a third of the time. Rutchik is seeing IT being brought it on the front end of SaaS deployments--to provide the proper governance framework, validate performance, and create interfaces. The IT organization will make sure SaaS fits the company's architecture, from bandwidth to desktops. But once the app's in a steady state, the business unit may be the right one take over its management, he says, since they're most affected by performance.

How To Manage The Cloud

To make sure IT manages the cloud properly, we recommend looking at the top concerns IT has about outsourcing in general, and then tailoring remedies for what's unique about cloud services. The rise of SaaS and cloud computing often is assumed to relieve work on overburdened IT groups, and it can, as our generally positive results show. But cloud computing extends vendor relationships further than ever for some companies, and customers must stretch their management practices to cover it.

What follows from our survey results are the top problems that plague outsourcing in general, followed by steps IT managers should take to protect cloud computing and SaaS projects from them.

Unforeseen costs: Organizations must factor in items beyond the core service, including bandwidth requirements, security upgrades, and monitoring costs.

Communication problems: Typically, cloud projects don't have a point project team or dedicated support reps from the vendor. You need to push for regular contacts before and after the project.

Lack of understanding of our industry: This can be a problem, especially for SaaS initiatives, since the economics of the cloud lean toward one-system-for-all, not industry-specific systems. It's critical to have a complete business requirements plan that can be matched to system functionality.

Time required to manage: Management of the cloud extends well beyond a functional team and should include security, network operations, and application groups, as well as business unit reps taking new responsibility, as Rutchik suggests.

Quality control and compliance: Extend your quality control beyond basic checks to actively monitor vendors. Problem is, only 17% actively monitor all their cloud/SaaS apps.

IT teams also should be involved in scrutinizing cloud deals before they're signed. They're different from conventional outsourcing agreements--a hybrid of outsourcing, software, and leasing--but they're still a major contractual commitment, says Richard Austin, a former general counsel for EDS Canada who now has his own Toronto practice, Austin Technology Law. "Don't let the decision be made at a midlevel within the organization," Austin advises. "Treat it as seriously as any other form of outsourcing."

More than half of our survey respondents say their companies have no system for managing outsourcing requests for proposals. Furthermore, 43% lack a contract management system once the contract's awarded, and 29% lack an online project management system. Can you say "Excel hell"?

Even companies with rigorous monitoring and measurement standards for outsourcers often don't use those measurements internally, says Mark Rosen, a specialist in Lean Six Sigma strategies who has worked on internal quality programs at Fidelity Investments, external quality programs for outsourcer Nadastra, and is now heading a quality initiative with a major U.S. retailer. "External vendors are often willing to provide the details and metrics to win the business. Management then falls short by not forcing the practice to the internal teams, often avoiding obvious performance issues with their own staff."

Companies need to be bluntly honest about the time and cost it takes to manage outsourcing, and then make that investment. Many aren't. It's critical not only for expanded use of the cloud, but also as outsourcing takes on ever-more strategic roles.

Well Beyond The Basics

An "outsource the basics" approach is still followed by 30% of our survey respondents' companies--typically, tasks such as testing, app support, help desk, and discrete development.

However, we're surprised to see how many companies have moved outsourcing up the stack. Seven percent of organizations look to outsource the entire IT function, including executive and senior management roles. Twenty-three percent keep executive and management functions but outsource everything else, including engineering and development roles; 21% keep engineering in-house but outsource development. Added up, 59% of companies outsource one or more of the strategic components of their IT operations.

With any outsourcing, but especially with higher-skill functions such as architecture and engineering, IT leaders have to ask this question: Can they keep their competitive advantage with that knowledge outside the business? It puts added pressure on managers to push providers for the innovation a company needs and its customers expect.

Consider data center operations. Many companies consider it non-core, and nearly half of our survey respondents outsource its operation or support to some extent. Yet some of the biggest strategic decisions in technology today center on the data center--virtualization, next-generation operating systems, and the mix of internal and external cloud computing. How aggressively is your outsourcing vendor pursuing these changes for you? Another area is the fast-moving nature of the Web. Only 10% of companies completely outsource their Web site, but nearly half use some outsourced resources. Companies can't expect outsourcers to have a deep enough understanding of your business in order to know what customers expect from your site in terms of search, content sharing, content ranking, live chat, and integration with Facebook and other social networking sites. Yet those things are vital to a vibrant Web site. Get that wrong, and you'll find your company tweeting in the dark.

The Outsourcing Threat

While most companies accept outsourcing as part of modern IT, it's important to acknowledge that 26% of our survey respondents say staffers see outsourcing as a threat to their jobs. The only surprise is it's not higher, given the fragile U.S. economy and high unemployment.

Companies of all sizes cut back across departments, including IT. In some cases, IT leaders have used this downturn to drive strategic change, like moving apps to a cloud model, that seemed too disruptive before. Even as the economy improves, some CIOs will continue to drive this change, resisting hiring as long as possible, pushing the limits of flexible staffing models and exploring cloud-based approaches.

"It's a gray moment for IT," says Terrence Gaughan, a partner with DevSelect, an outsourcing and IT staffing firm. "IT is getting charged with solving a problem and even given money to do it." However, CIOs will be reluctant to hire in a recovery that feels shaky. Says Gaughan: "It's going to be difficult to be the 'first' to rehire, especially for IT."

Large companies have discreetly formalized this approach, setting a fixed percentage of IT staffing as outsourced. One survey respondent's employer has targeted a 40% outsourcing model. Grumble at the model if you must, but there's a humane element to it: It's a lot easier to cut back a staffing contract than lay off one of your own.

To execute it, however, most teams must improve their management of all outsourcing. The early productivity and cost-saving gains we're seeing with cloud and SaaS initiatives may vaporize if companies don't invest in integrated monitoring and management systems. Break/fix outsourcing, app dev, and simple Web conferencing are still large parts of the outsourcing picture, but our survey shows the biggest growth is in outsourcing of more complex engineering tasks and broader cloud systems that must be tightly integrated into your ecosystem.

Outsourcing has always had the potential to produce competitive advantage--anything most companies struggle with, and some do well, separates winners and losers. Our data shows that even in the category where IT outsourcing is most successful, only 37% of companies hit that magical quadrant of "lower cost, higher quality," while in the worst category it's 17%. Expanding outsourcing to higher skill levels and into more sophisticated cloud computing initiatives will only raise the potential to gain advantage, and ramp up the stakes for effective outsourcing management.

One respondent summed it up well: "We regularly look to outsourcing at different points to move us forward, assuming certain responsibilities if it makes sense in terms of our overall business. We think of it as an evolutionary process." Remember, though, what drives evolution: Only the fittest survive.

Michael Healey is president of Yeoman Technology Group.
Write to us at [email protected].