Windows Compute Cluster Server is Ready, Microsoft Says

Published: June 14, 2006

by Alex Woodie

Microsoft on Friday reached a significant milestone in its short high-performance computing (HPC) career when it announced it has released to manufacturing (RTM'ed) Windows Compute Cluster Server 2003, the new 64-bit operating system that it hopes will shake up the low end of the HPC market. The company also promises to turn heads with a better-than-expected showing on the Top 500, a list of the world's largest supercomputers, which will be updated at the end of the month.

It's been two brief years since Microsoft revealed its HPC intentions to the world and threw down the gauntlet at the feet of the Linux and Unix ecosystems, which have dominated supercomputing for so long. Now, after rounds of beta tests and a single release candidate earlier this spring, the Redmond software giant says Windows Compute Cluster Server 2003, or Windows CCS, is good to go. The operating system is shipping only a year later than predicted, which is lightning fast compared to other Microsoft projects these days.

There aren't too many areas in the software world where Microsoft is a newbie, but HPC is one of them. Some have hypothesized that the software giant's past transgressions in the desktop and business server market would dampen enthusiasm in the academic and scientific realm, where the free flow of ideas and source code are paramount. So far, Microsoft is abiding by this ethos, and it has even pledged to turn over the changes it made to the open-source Message Passing Interface (MPI) standard for connecting the nodes of a supercomputer. Windows CCS' implementation, MSMPI, is based on and compatible with the Argonne National Labs MPICH2 implementation of MPI2.

While Microsoft does its best to fit into the HPC world in some ways, there's no mistaking that the company really wants to transform this form of computing in drastic ways. Chairman and chief software architect Bill Gates talked last fall about his vision of ubiquitous supercomputing and the far-reaching effects this could have on people's lives.

In lieu of the killer supercomputing app, Microsoft will concentrate on traditional HPC areas such as fluid dynamic computations, oil and gas reservoir simulation, and drug discovery. The company has the backing of what it says are the premier developers of commercial HPC software, which have ported their applications from Unix and Linux to Windows. Last week's announcement listed more than a dozen ISVs that have completed the ports: ABAQUS, Absoft, ANSYS, The BioTeam, CD-adapco, ESI Group, Fluent (which was just acquired by ANSYS), Livermore Software Technology, The MathWorks, The Mecalog Group, MSC Software, Parallel Geoscience, Schlumberger Information Systems, SilverStorm Technologies, The Portland Group, Tyan Computer, Voltaire, and Wolfram Research. In addition to these vendors, Microsoft also counts a number of system builders, network device makers, and infrastructure software vendors as HPC partners.

Users will be able to submit jobs to Windows CCS through a Microsoft-supplied job scheduler, or a third-party job scheduler from Platform Computing. This is going to be a boon to users, says John Borazan, lead program manager for Windows CCS. "The way we've implemented support, the two job schedulers can interoperate," he says. "That's going to be tremendously valuable for customers that want to add Windows-based clusters to environments that already have Unix or Linux" clusters.

Windows Compute Cluster Server 2003 will be generally available in the coming months, with a starting price of $469 per node. The product will only support X64-based hardware from Intel and AMD. The MS-MPI protocol will support standard Gigabit Ethernet, InfiniBand, and any other network that provides a WinSock Direct-enabled driver.

The product itself will be shipped in a two-CD package. The first CD contains Windows Server 2003 Compute Cluster Edition, and the second CD contains the Microsoft Compute Cluster Pack, a combination of interfaces, utilities, and management infrastructure that makes up Windows Compute Cluster Server 2003.

Setting up a Windows CCS cluster involves installing the operating system on the head node, joining it to an existing Active Directory domain, and then installing the Compute Cluster Pack. At that point, the administrator can either push Windows CCS out to the supporting nodes automatically, or install the software manually. Although Windows CCS nodes can be used to run general purpose Windows workloads, users must purchase a separate copy of Windows to do so; Windows CCS can't run standard Windows workloads.

Microsoft has streamlined the setup procedure to make it as easy as possible to get a HPC up and running, a nod toward customer demand for easier-to-use HPC software. "Although clusters are much more affordable today, they're still too complex," Borazan admits.

Windows CCS has already been put to use at several early-adoption sites, including aQuantive, BAE Systems, CASPUR, Cornell University's Computational Biology Service Unit, National Center for Atmospheric Research, Northrop Grumman, Petrobras, Queens University, Tokyo Institute of Technology's Global Scientific Information and Computing Center, the University of Cincinnati's Genome Research Institute, and Virginia Tech's Computational Bioinformatics and Bioimaging Laboratory.

Several early adopters have already deployed fairly large clusters that demonstrate the ample processing headroom and scale-out capability of Windows CCS, Borazan says. "Some of the customers that have put the product into production early and made purchases are doing so with a fairly high number of nodes," he says. These include a bank in the U.K. that initially deployed a 700-node cluster, then purchased a license for an additional 2,400 nodes, and a U.S. oil and gas company that has deployed a 2,800-node cluster.

At least one of these referenced customers will earn Microsoft a coveted spot near the top of the Top 500 list of the world's largest supercomputers. "We will have a score to talk about at the end of June to demonstrate that headroom," Borazan says. "We can't say where on the Top 500, but it will be very competitive . . . I think people will be pleasantly surprised with how much scalability there will be."

While the company chases large accounts, what it's really after are the greenfield sites where HPC has yet to make any inroads. "The thing we're excited about, what makes our contribution unique, is we're at a place in the evolution of computing that we can bring HPC to a mainstream audience," Borazan says. "Traditionally, its been confined to market niches in certain verticals. We will pursue those. . . . But within five years, we want to allow it to be on everybody's desk."

RELATED STORIES

"Microsoft Ships First Release Candidate for Windows Compute Cluster Server 2003"

"HPC Version of Windows Server Goes to Public Beta"

"Gates Lays Out Vision of Future of Supercomputing"

"Microsoft Confirms Windows Server HPC Edition Due in 2005"

"Windows HPC Edition Is in the Works"