NYSE Euronext is in the process of equipping two new data centers in the greater New York and London areas and will use them to reposition itself as a technology leader of trading services.
But ultra-modern no longer means ultra-big servers or ultra-complex infrastructure. On the contrary, Google, Yahoo, Amazon, and Microsoft, companies with massive data center capacity, are building data centers with common components -- PC-style servers, disk drives, and I/O devices -- on a massive scale.
At the NYSE facilities, a small portion of each data center is devoted to systems running Sun UltraSparc or IBM Power-based servers, perhaps 80 servers in all, says Steve Rubinow, CIO of NYSE Euronext. But most are Intel and AMD-based servers.
"We have a collection of blades on racks and individual servers -- a mix -- with a lot of storage and networking gear. They are almost exclusively x86 (instruction set) servers," he said in numbers that approach but fall short of 10,000. The servers are four CPU units running dual cores, or eight cores per server, a low-cost sweet spot in the server market.
Amazon and Google are reported to have built 40,000- to 50,000-server data centers; Microsoft boasts the Chicago data center it opened last fall will eventually have 300,000 servers and is believed to be the largest in the world.
The NYSE x86 servers are virtualized "but not heavily," meaning the number of virtual servers per physical machine has been kept light, rather than trying to utilize every CPU cycle with a high number of virtual machines. In some settings, 16-30 VMs per server is considered a heavy load.
Any system that is highly latency sensitive is placed on a lightly virtualized server or non-virtualized server, while the more latency tolerant systems can be doubled up in multiple virtual machines per server.
NYSE customers don't get to choose the level of latency they want. On the contrary, "everyone goes through the same pathway," said Rubinow, so trading services are delivered on a level playing field for all customers. But NYSE wants its trading services to be the lowest latency systems in the world, he said.
NYSE Euronext CEO Duncan Niederauer said the data centers were designed with similar, uniform architectures on both sides of the Atlantic, one in Mahway, N.J., which has allowed the firm to gradually reduce IT staff. As legacy systems were consolidated or retired, the staff members who served them tend to retire also, he noted.
Rubinow refused to say how many people are needed to manage a 400,000-square-foot data center but said the number was in the dozens, "less than 100," and represented a considerable expense reduction in the budget over the 12 data centers previously operated.
Only 40% of the data center is intended for NYSE systems. The remainder is for partners' co-location equipment to be close to the core trading systems.
In addition to ease of management, the similar data centers provide a simpler development target, a simpler product testing environment. With a short learning curve, Rubinow says, the staff of one would feel at home in the other.
The data centers had to be situated where two utility districts shared a border, so electricity was being supplied by two separate systems. "We have two networks with no crossover paths. There's two of everything. We have no single points of failure," Rubinow said.
Unlike Amazon, Microsoft, and Google, the NYSE had to stay close to trading centers to suppress latencies rather than be located on the Columbia River or other sources of cheap power.
"Until recently, NYSE didn't run data center facilities for customers," noted Rubinow. Now it's in a position to provide data close to the real time of market operations.
NYSE, like other exchanges, knows both what its own customers and customers on other exchanges are bidding for equities, bonds, options, derivatives, and other trading instruments. It can match up that information with financial services from Bloomberg, Reuters, Dow Jones, Thompson, and other suppliers.
Rubinow and Niederauer see new services springing out of the information flow and trading services that will have a global reach. Rubinow says the data centers will use less power than their predecessors, although "green" isn't exactly the goal. "To be a successful competitor, our systems have to be efficient," he says.