Massive Expansion in Progress at Microsoft Data Centers

Published: May 14, 2008

by Alex Woodie

In its quest to compete with Google for Internet utility computing supremacy, Microsoft is ramping up its construction of data centers to new, cloud-like heights. According to Debra Chrapaty, the corporate VP in charge of Microsoft's datacenters, the company is installing 10,000 new servers per month into its data centers. The monumental task is made somewhat easier thanks to recent advances in using steel shipping containers as turnkey data centers that can be dropped off practically anywhere.

During a speech at the Microsoft Management Summit 2008 conference in Las Vegas two weeks ago, Chrapaty, whose official title is corporate vice president of global foundation services, provided a glimpse into Microsoft's data center operations, and in particular how it's coping with the tremendous growth spurred by Microsoft's various business initiatives.

Chrapaty's team is in charge of building and running data centers for Microsoft. "We essentially run the infrastructure for Microsoft," she told the audience. Considering that Microsoft is a $60 billion company with 79,000 employees around the world, that's a pretty big job.

Here are some statistics about the requirements of Microsoft's data processing system for internal uses, as well as for external use through its MSN, Windows Live, and Hotmail services, according to Chrapaty. Her team must store and manage 400 million images, 20 billion documents, and processes 2 billion queries per month. It must keep track of 550 million MSN users, 350 million HotMail accounts, and 1 billion Windows Live ID authentication requests per month. Microsoft Messenger must be available to handle 350 billion messages per day. Altogether, Microsoft Web sites garner 10 billion page views per month.

"Sometimes even I find it overwhelming in terms of its size and scope," says Chrapaty, whose IT resume includes stints at Bertelsmann, EMI Records Group, the Federal Reserve Bank of New York, and E-Trade.

Obviously, these are busy days for Chrapaty. Since she joined Microsoft five years ago, the company has grown its server count by a factor of 15, she says. And the company must continue to install servers and build data centers to keep up with Microsoft's enormous demands, particularly as it ramps up its "Cloud" initiatives with its various Windows Live properties.

Currently, Microsoft is meeting that need by adding 10,000 new servers per month, Chrapaty told the audience. Many of these servers are being installed in data centers that are composed of a series of 40-foot steel shipping containers--or CBlox in Microsoft terminology--filled with racks of servers and other computing equipment.

Drop the CBlox off with a semi-truck, plug it into the electrical grid, hook it up to a network connection, and you're good to go. "Containers are really, really cool," Chrapaty says. "You can do some very innovative things with containers as part of total cost of ownership."

CBlox will form the entire first floor of Microsoft's new data center in Chicago, which with a capacity of 100 megawatts will be the largest data center in the world, according to Chrapaty. "We're looking at about 300,000 servers in the Chicago facility," she says, adding that facility will be able to ramp up the computing density to 1,200 watts per square foot, compared to average densities of 400 to 450 watts per square foot.

Microsoft also dropped CBlox units off recently at its data center in Boulder, Colorado, where the computers will be used to calculate topographic features as part of the new version of Microsoft's Virtual Earth Web service. But the Boulder CBlox has a green twist--instead of plugging into the standard electrical grid, Microsoft is tapping into wind power to fuel its CBlox. Because CBlox are often installed as standalone entities, cooling costs can also be reduced by tapping into cooler outside air, instead of running the air conditioner, she says.

Managing such a large environment is obviously a challenging task. Chrapaty's group doesn't have the luxury of managing a Windows-only infrastructure; as a result of acquisitions, the company also has Sun Microsystems (presumably Solaris) and Linux servers to manage, she told the audience. (Back in the day, Microsoft had a sizable investment in IBM's proprietary AS/400 server line to run its MRP system, but it's believed those systems have been migrated to SAP's ERP software.)

Microsoft relies heavily on Systems Center Operations Manager to keep its servers running on a day-to-day basis, according to Chrapaty. She rattled off some horrendous statistics concerning trillions of rows of data generated per day to keep the hundred-thousand-odd server infrastructure well monitored and well oiled. Considering the company recently committed to opening up Ops Manager to monitoring other platforms, Microsoft's own IT shop should benefit considerably.

Chrapaty owned up to being a configuration junky. "Configuration management is the heart and soul of everything," she said. "When you have hundreds of thousands of servers in your environment, if you don't have your config management right it's really impossible to run your environment."

Microsoft, of course, has hit a rough patch with its next-gen configuration management database (CMDB) offering, System Center Service Manager, which was to have shipped by late last year but now isn't slated for delivery until 2010. Chrapaty's group used to roll its own, but now uses Ops Manager and Microsoft's Configuration Management to keep track of assets.

One Microsoft technology Chrapaty's group is currently using is HyperV, the new hypervisor software that is still in testing and slated to become available by late summer. Chraptay hopes to use HyperV to push the server utilization higher among Microsoft's huge distributed computing infrastructure. She told the MMS audience that if she can boost Microsoft's current CPU utilization rate from an average of about 17 percent to just 20 percent over the next three years, she will be able to save Microsoft $2 billion.

RELATED STORIES

GDCM Seeks to 'Defrag' the Data Center for Higher Efficiency

Operations Manager 2007 Goes Cross-Platform, Hooks Into Tivoli, OpenView

System Center Service Manager Delayed Two Years by Microsoft

Rackable Builds Data Centers in Shipping Containers, Too

                     Post this story to del.icio.us
               Post this story to Digg
    Post this story to Slashdot