Microsoft Makes Gains in HPC Market

Published: November 14, 2007

by Alex Woodie

Things were looking up for Microsoft's fortunes in the high performance computing (HPC) business this week as the software giant made several announcements at the Supercomputing 2007 conference in Reno, Nevada. First, the initial public beta of the forthcoming new HPC variant of Windows Server 2008, called Windows HPC Server 2008, was posted. Microsoft also made advances on the Top 500 list of the world's biggest supercomputers, launched a parallel computing initiative, and hired an expert in parallel computing.

Microsoft got into the HPC business about 15 months ago, when Windows Compute Cluster Server 2003 started shipping. Since then, the software giant has been working to build credibility in a market that's dominated by exotic processor architectures running Unix operating systems at the high end, and by inexpensive clusters of X64 servers running Linux everywhere else. The perceptions of Microsoft held by those in the HPC community, which is dominated by open source advocates, has made this an uphill battle for Microsoft.

Despite the challenges, the company appears to be making some headway, particularly in the financial services industry, which accounted for the bulk of Windows Compute Cluster Server 2003 sales, according to Jeff Wierer, a senior product manager with Microsoft. Another successful niche is the oil and gas industry, where Microsoft announced support from three more companies developing HPC software.

Wierer was in Reno for the SC07 event, where Microsoft announced that the second version of its supercomputer operating system will be called Windows HPC Server 2008. The company also announced the first public beta release of HPC Server 2008. The second public beta will come when Windows Server 2008 is released to manufacturing, which is expected to occur during the second quarter. The RTM of HPC Server 2008 is expected 90 to 180 days after that, Wierer says.

Users can expect a big performance boost with HPC Server 2008, according to Microsoft. On the company's 2,048-core cluster located at a data center in Rainier, Washington, which it uses to develop and test HPC Server 2008 (and which ranks as one of the 100 biggest supercomputers in the world), HPC Server 2008 scored 30 percent higher on the LINPACK benchmark than CCS 2003, without changing any of the hardware. System load and prep time was also cut down to two hours, thanks to the use of Windows Deployment Services in HPC Server 2008, Wierer says.

HPC Server 2008 will bring new features like support for high-speed networking, new cluster management tools, new failover capabilities, a service oriented architecture (SOA) job scheduler, and support for partners’ clustered file systems, including those from Panasas, Sanbolic, Quantum with StorNext, and Hewlett-Packard with PolyServe. IBM has also announced plans to support HPC Server 2008 with its General Parallel File System (GPFS), but it's not ready yet.

As more HPC organizations gain experience with Microsoft's HPC offerings, the ecosystem around the core HPC offering gets deeper, Wierer says. "As you move along and you're building out your clusters, first you figure out your operating system, your development environment, how am I going to do the resource scheduling, and inevitably you ask 'Do I have the high speed networking support to move stuff around, especially if I'm doing MPI-based jobs,'" he says. "And inevitably, you'll hit a data-bound error, so how do I get that information to them. Maybe I need a parallel file system. Or I need to get it to them in a certain latency, so I need a high speed file system. So as you move further and further up the stack, we're going to have solutions from Microsoft or third parties that address all the needs across HPC."

Microsoft also improved its standing on the Top 500, the list of the biggest supercomputers in the world, which is updated every six months. There are currently six Windows servers on the list, including Microsoft's 2,048-core cluster in Rainier, which at 116 on the list was the most powerful Windows computer on the planet. Windows also was running on the clusters ranked 239, 309, 406, 407, and 422.

Microsoft also launched its new Parallel Computing Initiative, which is a project to develop and popularize Microsoft technologies that can help in parallel and multicore environments. Chief among the new technologies Microsoft wants to push is Parallel Extensions to the .NET Framework, which can help developers create HPC applications.

The company also hired Dan Reed, currently director of the Renaissance Computing Institute (RCI) in North Carolina, to be the director of scalable and multicore computing at Microsoft Research. At RCI, a joint venture of the University of North Carolina at Chapel Hill, Duke University, North Carolina State University and the state of North Carolina, Reed became known as an expert in parallel computing, which is the field he will be pursuing at Microsoft.

"I am enormously excited, as these are among the most interesting technical problems in computing, and they are my long-time professional interests," Reed wrote on his blog. "I will be working with Microsoft researchers and product developers, as well as industry partners and academics. It doesn’t get any cooler than this."

RELATED STORIES

SGI and Microsoft Partner on Windows Supercomputer Clusters

Details Emerge in Microsoft's Top 500 HPC Setback

Windows CCS Falls Off Top 500 Supercomputer List

Windows Compute Cluster Server 2003 Goes GA

Windows Compute Cluster Server is Ready, Microsoft Says

                     Post this story to del.icio.us
               Post this story to Digg
    Post this story to Slashdot