Windows HPC Edition Is in the Works

by Timothy Prickett Morgan

Although Microsoft has refused to confirm the many reports that say so, it appears the company is working on a version of its Windows Server platform specifically tailored for the high performance computing market. That Microsoft would branch off from Windows 2003 Server to create an HPC Edition makes perfect sense for a number of reasons, but the main benefit would be to cut off one of the major growth areas for the Linux market, while filling its own coffers.

All that the Microsoft spokesperson I talked to would say when asked about Windows Server HPC Edition was this: "Microsoft has been doing work in the HPC space for quite a while, but hasn't made any official announcements regarding a supercomputing/high-performance-computing product. But they are evaluating the best way to enhance and package HPC capabilities for customers. Microsoft's view is that building high performance computers from clusters of standardized server hardware is emerging as an important usage scenario in general, and for Windows Server, to solve technical and business problems that only a few years ago required dedicated supercomputers. This approach is complementary to 'scale up' computing in which Windows runs on single large servers with up to 64 processors."

Glad we cleared that up.

Let's try again. Thirty years ago, the HPC market was dominated by massively expensive vector supercomputers that ran a collection of Fortran and C programs on Unix operating systems. A vector processor is a fancy name for a specialized computer that does floating point math very fast, and being a specialized machine for a relatively tiny market, vector processors had to be expensive by definition. In the mid-1990s, as HPC computing requirements went up faster than budgets (particularly for weather modeling and weapons research), research organizations in government and academia swiftly adopted many of the supercomputing programs to run on so-called massively parallel supercomputers that were created from clusters of Unix workstations or servers. These servers were linked together by special fast switches and software based on the Message Passing Interface standard. This MPI approach gradually went corporate, and with the advent of Linux clusters on cheap X86 iron in the past few years, it has gone mainstream. Microsoft wants a piece of the action.

Since 1992, Microsoft has been working with Dell, Intel, and the Cornell Theory Center of Cornell University, in Ithaca, New York, to create MPI-based clusters running a variety of Windows operating systems. There is a plethora of software available to make Windows clusters, but most companies are moving from Unix to Linux clusters because of the close relationship that exists, from a code execution point of view as well as from a system administration perspective, between the Unix and Linux systems.

The people jabbering about this rumored HPC Edition for the past few weeks didn't stop to realize two things. First, MPI is an open standard and Microsoft can easily weave it right into the guts of Windows, either at the communications layer or within the Common Language Runtime (CLR) execution environment of the .NET Framework. The latter would be more useful, since it would better insulate programmers from the complexities of having to program for parallel environments. (That's the theory, anyway.) Microsoft could partner (say with MPI Software Technology, one of the experts in this area that has Windows-compatible MPI code already done.

Here's the other thing they forgot. With the Services for Unix (SFU) layer of Windows Server 2003, which was significantly expanded in February and is now free, Microsoft has a Unix development and runtime environment inside Windows. This could also be extended with MPI, allowing Fortran and C applications written for parallel Unix clusters to be more easily ported to parallel Windows clusters. In theory, the parallel Unix applications would not have to be tweaked much (but would have to be recompiled) to run within the SFU environment on Windows. This is not such a big deal. Companies moving from Unix to Linux clusters are already doing it, in fact.

There are other possibilities, including creating a grid environment, as Sun Microsystems has, that aggregates the processing capacity of servers in an MPI cluster with desktops residing on the corporate network to create an even more massive parallel supercomputer. What Grid Engine does for Solaris—creating a virtual processing pool for parallel applications—Windows Server HPC Edition could do for Windows.

The main thing Microsoft has to realize is, if it wants to get into the HPC market, it not only has to more tightly integrate MPI and other protocols with the Windows platform, it will have to compete on price as well. The dirty little secret in the parallel Linux cluster market is that a lot of these machines are not running expensive server editions of the commercial Linux software (if they are using commercial versions at all), but rather are using stripped-down versions of desktop Linux editions, which basically have the kernel, some compilers and libraries, and the clustering software installed.

Windows Server HPC Edition, whether it comes out later this year or next year, is going to have to be very inexpensive to beat Linux in the HPC market. Linux didn't just take off in the HPC market because it was like Unix. It took off because it was cheap or free. If Microsoft can make programming for parallel supercomputers easier, through the magic of CLR and the future Visual Studio 2005, it may be able to charge a slight premium for a future Windows Server HPC Edition.