Big Blue Commercializes Blue Gene/L Linux Supercomputer

by Timothy Prickett Morgan

The SC2004 supercomputing show is being held this week in Pittsburgh, and IBM is taking the opportunity to not only show off that a partially completed version of its Power-Linux Blue Gene/L supercomputer is the number one machine on the Top 500 supercomputer list, but to also launch a commercialized version of the Blue Gene/L system that anyone, not just the U.S. government that sponsored the development of the cluster, can buy.

At the end of September, IBM had already pre-announced that the Blue Gene/L supercomputer that it is building for the U.S. Department of Energy for Lawrence Livermore National Laboratory would top the Top 500 list. So this is not exactly news. But IBM has delivered the machine to LLNL now, and has increased its peak performance from 45 teraflops to nearly 71 teraflops.

IBM announced the Blue Gene concept nearly five years ago and said that it would create a 1 petaflops supercomputer using 1 million processors and $100 million of its own money. After some initial design work, IBM cleverly rolled a Blue Gene/L prototype into its ASCI Purple contract with the DOE, which is using supercomputers to simulate the decay in the U.S. stockpile of nuclear weapons, among other things (such as new weapons design). In effect, IBM has got Uncle Sam to pay for the development of Blue Gene, just like Cray has been able to get a $90 million contract with the DOE to create the "Red Storm" parallel Linux-Opteron cluster for Sandia National Labs; Red Storm has just been productized, too, as the Cray XT3.

Dave Turek, IBM's vice president of deep computing, said that IBM would call its commercial Blue Gene/L machines the eServer Blue Genes, and that it would offer them in racks with 1,024 processors per rack (each with two cores), with the cores running at 700 MHz. (Blue Gene/L is based on a stripped down 32-bit PowerPC 440 embedded processor.) Such a rack delivers a peak 5.7 teraflops of processing power, and these are exactly the same racks that LLNL is buying. In fact, IBM will eventually sell Blue Gene/L systems that span from one to 64 racks, with that top end being the projected peak performance of the final 360 teraflops machine that LLNL will get sometime in the spring when IBM is done building it. The commercialized Blue Gene/L machines will run a stripped down Linux kernel on their processing nodes, and will run Novell's SuSE Linux Enterprise Server 9 on I/O and management nodes in the cluster. Turek says that IBM intends to charge $2 million for a rack, including the cost of Linux licenses but not for external disk storage. In March, IBM was showing off a prototype deskside Blue Gene/L machine that houses 128 PowerPC cores in a deskside box, and Turek says that IBM will likely commercialize this for developers sometime in 2005 as commercial Blue Gene/L sales take off.

The massively parallel design of Blue Gene/L, its relatively low price, and the very low heat profile of the resulting cluster is going to be very attractive to HPC customers, but probably also to companies like Google, Yahoo, eBay, and so forth, who have custom computing clusters with tens or hundreds of thousands of nodes running Linux or Unix.

Turek says that IBM decided to commercialize Blue Gene/L only this summer, and has committed to delivering two more generations of machines at this point. The next generation of Blue Gene/L will scale to over 1 petaflops, as IBM promised back in 1999, and the third generation will span up to multiple petaflops.

IBM is taking orders for the eServer Blue Gene/L machines now, and will ramp up production through the spring. Turek says that while IBM did not build up an inventory of parts for Blue Gene/L because it had not planned to commercialize it beyond the LLNL project, he says that IBM Microelectronics will have no problem meeting demand to the custom chips the supercomputer uses as orders take off.

In addition to the Blue Gene/L announcement, IBM also announced that it has sold two giant BladeCenter clusters running Linux for running supercomputer applications. The first one went into Seoul National University, and it is comprised of a 420-node cluster using the JS20 PowerPC 970-based blades, which have two processors running at 2.2 GHz. This machine will have an aggregate processing capacity of 5 teraflops, which used to sound like a big number. An even bigger JS20 blade cluster, dubbed MareNostrum, was just acquired by the Spanish government and will be comprised of 2,282 of the two-way JS20 blades and will deliver an aggregate of 40 teraflops of peak performance when it is completed. The MareNostrum machine will be run by the Spanish Ministry of Education and Science in Madrid, and it has already got three-fourths of it completed and has demonstrated a peak performance of 31.4 teraflops on the Linpack test.