How many servers does Google have in how many data centers? Google's mum on that. But counting servers reveals as much about Google's infrastructure as tallying teeth tells you about a bear; better to worry about what the bear will eat next.

VP of engineering Douglas Merrill acknowledges Google uses a standard server setup--what he calls an "office in a box"--for provisioning IT services as it expands. It's possible, even likely, that Google has adopted a common design for data centers, too.

George Mason University professor Paul Strassmann suggested in a lecture last December that Google's Linux-based infrastructure is considerably cheaper to buy and maintain than a comparable setup of Sun Microsystems servers or Windows servers would cost. For IT shops that spend half their budgets just keeping machines up and patched, the implications are significant. Strassmann said IT pros know where they need to go--toward a Google-style architecture.

Some of the new servers going into Google's data centers are probably equipped with AMD Opteron multicore processors. Google won't confirm that, but one of the reasons AMD chips are selling so well to other companies is that they don't throw off as much heat as older alternatives. Google engineers, who pay close attention to microprocessor efficiency and heat dissipation, must find AMD chips hard to ignore. Intel is racing to improve the performance-per-watt of its own processor line.

Google has tackled some of the tough issues of data center management with internally developed technology. The company has patented homegrown designs for a better cooling baffle and fan mount for its rack servers. The U.S. Patent and Trademark Office lists 23 patents granted to Google, with 14 patent applications in the pipeline. That doesn't include patents obtained through acquisitions or filed outside the United States.

Though Google doesn't build its own fans or power supplies, it cares about such components. The company requests specific high-efficiency designs from manufacturing partners, says Urs Holzle, senior VP of operations.

Google's newest data center, under construction in The Dalles, Ore., has become a focus of media attention. An aerial photo of the data center appeared on the front page of The New York Times on June 14. Why Oregon? "We're always looking for candidate sites to host our infrastructure--selecting a site always means balancing a number of factors, including the availability of land and power, as well as a suitable workforce," Holzle explains in an e-mail interview. "The Dalles was one of the sites we found that met our needs, plus it's a beautiful area to live."

The availability of cheap power--widely cited as the main reason Google is building on a river in Oregon--isn't an issue for most companies. Edward Koplin, a principal in the Baltimore engineering firm of Jack Dale Associates, says that among his firm's clients, which include the Army Corps of Engineers, Citigroup, and Wells Fargo, "not one of them brings up as an issue, 'How cheap can my power be?'" But then, their data centers aren't growing the way Google's are.

Google declines to discuss how much it spends on electricity, but its financial documents note that data center costs have been rising. In a June filing, Google attributes an increase in the cost of revenues to, among other things, increasing data center costs, which include depreciation, labor, energy, and bandwidth costs.

Yet, even more than the availability of power and local subsidies, beauty may explain Google's decision to locate in Oregon. Rob Enderle, principal analyst at the Enderle Group, suggests that locating in an affordable, attractive, relatively undeveloped community represents an established strategy in high tech to attract and retain talent. "That was the old Microsoft model," he explains, noting that Redmond, Wash., was a nice place to live without much industry at the time Microsoft got started. "So once you got there it wasn't like you were going to go anyplace else. And they were able to keep turnover to an absolute minimum as a result."

Electric power may be expensive, but it's cheap compared to brainpower. "For as long as I can remember, organizational growth has been the No. 1 issue," says Holzle. "If you need to grow quickly but don't want to fall apart as a company, you need to focus a lot more on hiring, training, and nurturing the right culture."

Return to the story:
Google Revealed: The IT Strategy That Makes It Work Continue to the sidebars:
Google's Brew Of Open Source And Custom Code
and Profile: Google Technologist Knows Problem Solving Firsthand