Cray Subcontracts SuSE for "Red Storm" Linux Super Cluster

by Timothy Prickett Morgan

SuSE, which is arguably the number two supplier of commercial Linux operating systems, this week announced that Cray had selected it over rival Red Hat to act as a subcontractor to deliver the Linux licenses and support services for the 100 teraflops "Red Storm" parallel supercomputer cluster that Cray is building for Sandia National Laboratories in Albuquerque, New Mexico. Back in October 2002, Cray named chip maker Advanced Micro Devices as the primary hardware subcontractor, opting for its 64-bit "Hammer" Opteron processors over Intel's own 64-bit Itanium processors.

Sandia is home to some pretty big supercomputing iron, including the ASCI Red parallel supercomputer built by Intel. ASCI Red has 9,632 333 MHz Pentium II processors that collectively are rated at 2.4 teraflops on the Linpack Fortran benchmark commonly used to rank the processing capacity of supercomputers. Cray could have tried to build Red Storm using its Cray X1 multistreaming vector machine or even used off-the-shelf Itanium-based two-way or four-way servers, but instead chose to create what it calls a three-dimensional mesh of servers employing AMD's Opteron processors and its HyperTransport interconnect.

The initial $90 million for the first phase of the Red Storm project has been pegged for a machine that can deliver either 20 teraflops or 40 teraflops of peak performance, and they expect to have this machine operational in late 2004. However this gets done--whether it is using future dual-core Opterons, using Opterons with beefed up floating point units that can do twice as many instructions per clock, or using a cluster with twice as many processors--the 40 teraflops level seems to be what Cray, AMD, and now SuSE are striving for. But just in case something goes wrong, Cray is only guaranteeing 20 teraflops of power by late 2004.

SuSE has been trying to get into niche areas where Red Hat isn't focused so it can gain market share and differentiate itself. Email serving with its Open ExchangeServer 4, which is a ruggedized version of the open source Postfix email server with SuSE's own groupware added, is one such area. High performance computing is another area that SuSE has been chasing--ever since it was a partner of Compaq's on the AlphaServer line many years ago, in fact. Another area where SuSE trying to differentiate itself is with the support of AMD's processors. SuSE was the first of the commercial Linux distributors to demonstrate an OS running on the Opteron processors, and says further that its implementation of Linux for the AMD line of chips includes asynchronous I/O and multipathing, which will be important for the SAN storage used in the Red Storm cluster. (This support is already available in 32-bit SuSE and Red Hat implementations for Intel Pentium processors.) SuSE was probably more eager for the Cray business than Red Hat, too. Given all this, it was a no-brainer to choose SuSE over Red Hat for the Red Storm super.

Under the agreement, SuSE will also be providing unspecified developer support to Cray. The financial terms of the deal between Cray and SuSE were not disclosed. At 2 GHz, the AMD Opteron that will be ready this year should be able to do around 4 gigaflops apiece, which is what a 1 GHz Itanium 2 can do. (Intel has really worked on the floating point performance of the Itanium chips.) Assuming AMD can get a much more powerful Opteron chip in the field by next year, it should be able to hit around 6 gigaflops to 8 gigaflops of peak performance per processor. Call it 8 gigaflops for argument's sake. That means that the 20 teraflops version of Red Storm will have about 2,500 AMD Opteron processors in it, and the 40 teraflops version will have around 5,000 processors. If AMD goes for a cluster based on two-way server nodes--and this seems the most likely--then somewhere between 1,250 to 2,500 licenses of SuSE Linux Enterprise Server will be required for Red Storm. If AMD opts for a four-way server node, the numbers are cut in half to 625 to 1,250 server nodes. So, depending on what Cray and AMD do, SuSE will sell from 650 to 2,500 licenses to its commercial Linux, which at $800 a pop at list price works out to between $520,000 to $2 million. Premium support and other services will cost a lot more than this. Suffice it to say, the deal SuSE has with Cray is worth many millions of dollars.