HPC Version of Windows Server Goes to Public Beta

by Alex Woodie

While Microsoft is a relative newbie on the competitive high performance computing (HPC) circuit, the software giant is determined to gain some respect and to show that it's serious about HPC. It took a big step in this direction at a supercomputer conference being held in Seattle this week, where it issued a public beta of Windows Compute Cluster Server 2003, and outlined its ambitions for the HPC market.

The list of the world's 500 biggest supercomputers used to be dominated by proprietary RISC and Unix-based systems from the likes of Cray, SGI, Hewlett-Packard, and IBM. These supercomputers were tremendously expensive, which meant they could only be used by the biggest governmental organizations tackling the most important problems, such as simulating nuclear explosions.

In recent years, computer technology has evolved considerably, and the list of the 500 biggest HPC systems, which is updated every June and November at www.top500.org, has become increasingly composed of X86 and X64 clusters (Intel- and AMD-based systems now account for more than half), and the vast majority are now running the open-source Linux operating system. This shift toward inexpensive computers and free software has enabled less wealthy organizations to tackle a range of problems such as drug discovery and crash-test simulation.

Microsoft sees this shift, and wants its share of the glory, if not the business, which amounts to about $7 billion annually for HPC, just a fraction of the overall market. Microsoft's HPC ambitions were first revealed in May 2004, when it revealed it was working on a version of Windows Server for supercomputer-type workloads. A select group of beta testers got their first glimpse of Microsoft's HPC product, called Windows Compute Cluster Server 2003, at the Professional Developer's Conference in Los Angeles in September.

Yesterday marked the beginning of the public beta test for this product, which is expected to ship during the first half of 2006. The company also announced it would spend millions of dollars to set up 10 HPC centers at universities around the world, and highlighted some of the ISVs it is working with to provide the HPC applications that will run on its Windows clusters.

'Flops are Free'

Microsoft officials say the cost of supercomputers will continue to decline to the point where HPC systems will become much more prevalent. "HPC as a computing tool is growing and is no longer restricted to a niche high-end for academic research," says John Borozan, the group product manager in Microsoft's Windows server group in charge of HPC marketing. "The idea of personal supercomputing, or having a desktop cluster, is becoming much more of a reality."

We're on the cusp of a new era of HPC, according to Microsoft. In 1991, a 10 Gigaflop supercomputer from Cray would have cost $40 million, which put the system out of reach of everybody except the federal government. By 1998, the cost of a 10 Gigaflop system, such as Sun's HPC10000 computer, had dropped by a factor of 40, to about $1 million. Today, that 10 Gigaflop Shuttle system is available for about $4,000 online from Newegg, according to Microsoft.

The massive amounts of computing capacity available today is reflected in HPC system sales. According to IDC, the number of HPC installations grew by 70 percent in 2004, to about 150,000, and the growth was driven primarily by clusters that cost less than $50,000. This sub-$50,000 portion of the market will continue to grow at a healthy clip, and will eventually come to dominate the entire HPC market in terms of the number of installations, according to IDC. Larger HPC systems, including those costing between $50,000 and $250,000, and those costing between $250,000 and $1 million, will also experience growth, according to IDC charts.

Microsoft's plan is to target workgroup (sub-$50,000) and departmental ($50,000-to-$250,000) segments of the HPC market, with focuses on manufacturing, geosciences, life sciences, oil and gas exploration, financial services, and public market sectors.

Microsoft Targeting Linux for HPC

While the majority of HPC systems today run the Linux operating system on relatively affordable X86- or X64-based clusters, that's not to say the systems are easy to use, Borozan says. "Despite how economical these systems have become, they're still pretty tough to deploy. You have to be pretty savvy" in stitching everything together, he says.

Instead of requiring chemists or biologists to become computer scientists as well as experts in their own fields, Microsoft's goal with HPC is to make the deployment as easy as possible. "That's where we think we can remove a lot of the complexity for end users and developers," Borozan says. "That's the core part of our value proposition."

Microsoft says that by working with its ISV partners and OEM partners like Dell, HP, and IBM, it can do much of the work of building HPC systems beforehand, so users can get as close to a finished "turnkey" product as possible. To this end, Microsoft has pledged to closely follow the existing standards in the marketplace, including Infiniband and Gigabit Ethernet inter-connects, use of Winsock Direct and OpenIB protocols, and the open-source MPICH2 messaging interface from Argonne National Lab.

Microsoft has done some work to make sure that MPICH2, which is used for dividing HPC workloads among the various nodes, works well with its Windows HPC variant. These security and performance enhancements have been contributed to the open-source community, Borozan says. The company is also planning to include a job scheduler (often one of the most expensive components of a cluster) with Windows Compute Cluster Server 2003. It has worked with some of the premier developers of job schedulers for clusters to get those to support Windows.

In terms of ISVs, Microsoft has identified 19 ISVs that account for 80 percent of the HPC application sales in its targeted markets, and partnered with them to bring their applications to Windows. These ISVs each have applications running on Windows now, and some will participate in benchmarks prior to the release of Windows Compute Cluster Server 2003 next year.

The ISVs Microsoft is working with include: Accelrys; Ansys; BioTeam; CD-Adapco; Dassault; ESI Group; Fluent; Landmark; Livermore Software Technology Group; MathWorks; MSC Software; Parallel Geoscience; Platform Computing; Schlumberger; the University of Illinois Urbana's NAMD group; and Wolfram Research.

Borozan says that while the initial upfront acquisition cost of a Windows HPC system might not be much different than competing Linux-based systems, over time, users will save time and money because it is easier to manage. This will be helped by the fact that Windows Compute Cluster Server Edition 2003 will work with Microsoft Operations Manager and Systems Management Server. Pricing for Windows Compute Cluster Server Edition 2003 has not been set, but don't expect it to be too high, Borozan says. Think more along the lines of the Standard Edition of Windows Server 2003 than the high-end Datacenter Edition.

Up to this point, Microsoft has worked very closely with Cornell University in Ithaca, New York, which has deployed Windows clusters and been on the leading edge of using Windows for HPC. Microsoft is now working with the HPC programs at nine other universities. These universities include: the University of Texas; the University of Utah; the University of Tennessee; the University of Virginia; Southampton University in the U.K.; the University of Stuttgart in Germany; the Nizhni University in Russia; the Tokyo Institute of Technology in Japan; and Shanghai Jiao Tong University in China.