Download the entire new issue of InformationWeek Tech Digest (free registration required).

Systems administrators and software engineers have spent the past five years facing down the problem of whether and how to use software to automate application life cycles.

As an industry, we've agreed to call this process of defining an application life cycle in software "orchestration."

Brian Adler, an architect at cloud management platform provider RightScale, defines orchestration as automating "short- or long-running, event-driven, or schedule-based tasks across a deployment of servers." Luke Kanies, CEO at IT configuration management developer Puppet Labs, defines it as "making ordered changes to multipart services across a wide range of hosts, in an even or progressive manner."

The common thread: We need to use software to handle systems admin tasks in a consistent and automated way.

Orchestration is the next evolutionary step after virtualization, which ended the "one application per server" model. Under that old model, any multisystem interaction -- such as an application hitting the main database server -- was handled in an ad hoc, manual way. Today, with virtualization, the entire life cycle of an application server is handled through software, but it's rarely automated and often performed inconsistently. Unfortunately, that's where many shops stopped.

Look at the data from our InformationWeek Data Center Convergence Survey. We define "convergence" as building a cloudlike data center with virtualized compute, storage, and network resources managed as services, possibly using a cloud software stack such as OpenStack or vCloud, and possibly sharing workloads with external public cloud services such as Amazon AWS or Microsoft Azure.

In our October 2011 survey, just 29% of respondents who said the fully converged data center concept guided their technology and personnel decisions had orchestration deployed in production or testing. When we did the survey again in December 2013, the results were about the same. The percentage with no automation plans also had dipped only slightly, from 39% to 35%.

Some holdouts might feel they don't need orchestration because they're using infrastructure-as-a-service offerings like AWS. For them, physical servers, networks, and physical security are the vendor's problem, and virtual machines are launched via APIs, not by support tickets to internal IT staff. For users of IaaS, handling the entire life cycle of a server automatically becomes easier. But the fact is, in our experience, most users of IaaS still handle server deployments, maintenance, and termination manually.

Among 2014 survey respondents not planning on convergence, the top reasons, tied at 32% each, were "no perceived business advantage" and "other projects having higher priority." Both responses translate to "no money, no will."

However, we'll bet many of these shops haven't truly run the numbers. As software engineers take advantage of virtualized servers to design more scalable, highly available systems that require different applications running on different servers -- multiple databases, load balancers, cache servers, and redundant application servers are all standard best practices for Web applications today -- management costs add up. Add in more frequent software updates as business units demand new features more often, and the result will be more multisystem interaction than ever before. In such a case, the savings from running virtual machine life cycles and interactions under automated software control generally exceeds the cost of putting automation in place.

4 use cases that beg for orchestration.
In the simplest sense, orchestration is about bringing more automation and repeatability to app deployments. Presented this way, orchestration seems more evolutionary than revolutionary, and perhaps not exciting enough to drive uptake at the scale that virtualization and IaaS have been adopted. But don't kid yourself into thinking "orchestration is something we basically already do."

Unless you're on the bleeding edge of IaaS architecture design, it's highly unlikely you have fully orchestrated deployments. And while the technology may seem evolutionary, the benefits from using it well -- in time savings, cost savings, higher availability, and more reliability -- deliver revolutionary changes in IT performance.

The best way to understand the revolutionary value of orchestration is through examples. The four common orchestration scenarios below are each onerous to implement without an orchestration framework, but fairly simple to implement with one.

1. Automated deployment
In a traditional software development environment, developers will use less-powerful and fewer servers than the production application deployment. In addition, developers waste a lot of time switching work from one branch of the code to another (say, from a new feature to a bug fix for the current version) because it requires setting up the development environment again. Therefore, one oft-touted benefit of IaaS is the ability to develop, test, and deploy software more easily.

Read the whole story in the new issue of
InformationWeek Tech Digest
(free registration required).