IBM to Break Petaflops Barrier with Blue Gene/P

Published: July 10, 2007

by Timothy Prickett Morgan

The race is on to build the first supercomputer that can break 1 petaflops of aggregate number-crunching power, and IBM is hoping to be the first company to do it with its future Blue Gene/P massively parallel supercomputer. Blue Gene was conceived in 1999 by IBM Research as a project to test out a minimalist, vastly expansive parallel supercomputer design that would gang up a very large number of fairly simple dual-core PowerPC processors to create an energy efficient Linux box capable of breaking through the 1 petaflops barrier using as many as 1 million processors.

By 2002, IBM had talked the U.S. Department of Energy into spending $290 million to get a slice of the Blue Gene box, now known as Blue Gene/L and installed at Lawrence Livermore National Lab, as well as to build the ASCI Purple parallel Power-AIX supercomputer. Both machines are involved in managing America's nuclear weapons stockpile and in designing a new generation of warheads, which must be designed and tested virtually rather than physically because of the Nuclear Test Ban Treaty. The current iteration of Blue Gene/L, which was commercialized and is now available for purchase by companies and governments around the world, running at LLNL is based on 131,072 dual-core PowerPC 440 chips running at 700 MHz and has a Linpack Fortran benchmark rating of 280.6 teraflops.

With the second-generation Blue Gene/P, IBM is moving to a faster 850 MHz clock speed on the PowerPC processor cores and also upgrading from dual-core PowerPC 440s to quad-core PowerPC 450s. IBM is expecting that the peak aggregate computing power of Blue Gene/P will be around 3 petaflops, with sustained performance in the range of 1 petaflops. The Blue Gene/P design puts 4,096 processor cores in a rack, and the full configuration will have 216 racks and 884,736 cores to have that peak 3 petaflops rating. IBM is also adding symmetric multiprocessing (SMP) capability onto the Blue Gene/P boards, but it is unclear how far that SMP will scale.

Here's how Blue Gene/P is assembled. IBM puts four single-core PowerPC 450 chips in a single chip package; this package has 8 MB of L3 cache memory (double that of the prior machine). Then, each Blue Gene/P system board has 32 sockets, and one of these quad-core modules (QCMs) is plunked into the socket, giving 128 cores per board. Each board can support 2 GB of main memory (four times that of prior boards), and a rack of Blue Gene/P has 32 of these boards in it. That gives you the 4,096 cores per rack. Each Blue Gene/P QCM can do 13.6 gigaflops, which gives you 13.92 teraflops per rack. Add up 216 of these racks with a high-speed optical network and you hit 3 petaflops.

Importantly, Linux applications written for Blue Gene/L will run on Blue Gene/P, and equally importantly given the amount of electricity supercomputers consume and the heat they generate, Big Blue is claiming that Blue Gene/P will be a factor of seven more power efficient than other supercomputers. This comparison presumably excludes Blue Gene/L, which is very energy efficient compared to X64 clusters or parallel vector supers.

The Department of Energy is footing the bill for the first Blue Gene/P machine, and that government agency will install that machine at its Argonne National Laboratory later in 2007. The Max Planck Society and Forschungszentrum Julich in Germany are also planning to each install Blue Gene/P machines later this year, and Stony Brook University and Brookhaven National Laboratory in New York and the Daresbury Laboratory in Cheshire, England, have also placed orders for Blue Gene/P machines.

And no, Blue Gene/P will not run i5/OS. But customers who have complex simulations to run in every industry are looking at machines like Blue Gene/P to install to do their simulations because of the relative compactness and energy efficiency of the design.

IBM is not, of course, providing pricing information on Blue Gene/P. The initial Blue Gene/L machine came from a $200 million investment by IBM into its research arm back in 1999 and then Uncle Sam paid $100 million for the first box in 2002, which eventually reached a peak performance of 360 teraflops. That worked out to $278 per gigaflops for Blue Gene/L. If you assume some price/performance improvements--call it twice the bang for the buck compared to Blue Gene/L--then a 1 petaflops Blue Gene/P--that's 100,000 gigaflops--should set you back about $150 million or so.

RELATED STORIES

Top 500 Supers: Moore's Law Is Alive and Well

Dual-Core Processors Begin Takeover of Top 500 Super Ranking

Top 500 Supers: Brace Yourself for Petaflops Systems

IBM's Blue Gene/L Shows Off Minimalist Server Design

                     Post this story to del.icio.us
               Post this story to Digg
    Post this story to Slashdot