News While It's Still Hot

Blade Servers Make It to the Top HPC Sites

Published: November 13, 2007

by Timothy Prickett Morgan

The 30th edition of the semi-annual Top 500 listing of supercomputers in the world was announced today at the Supercomputing Conference 2007 in Reno, Nevada, and if Moore's Law is a constant in the IT racket, so is the desire by the largest supercomputer facilities in the world to consume ever more large aggregations of computing power. The availability of multicore X64 processors that stay within a fixed thermal envelope coupled with a more or less standard Linux application environment and cheap InfiniBand interconnect and even cheaper Gigabit Ethernet links between nodes is making high performance computing cheaper than ever before.

The November 2007 Top 500 list is noteworthy in that India has broken into the Top 10 for the first time with a new blade server cluster it acquired from Hewlett-Packard. IBM's Blue Gene Linux-Power supers continues to give Big Blue an edge among big systems, and exotic machines from Cray, Sun Microsystems, and Silicon Graphics have either appeared on the list or will in the next iteration.

The turnover rate in the Top 500 list is staggering--the machine listed at number 500 on this 30th list would have been halfway up the list only six months ago--and so is the diversity of interconnections and approaches to building very powerful HPC iron. The ante to be on the current Top 500 list is 5.9 teraflops, up nearly 50 percent from the 4 teraflops it took to get on the list only six months ago. The aggregate performance of the Top 500 machines in the list is now 6.97 petaflops, up 41.7 percent from the 4.92 petaflops on the 29th list six months ago, and up 96.9 percent from the 3.54 aggregate petaflops on the 28th list a year ago. This rate of increase in aggregate power on the list is twice that of Moore's Law, which in recent years doubles the performance of processors every 24 months or so. (It used to be 18 months, but the 120-watt thermal ceiling and 3 GHz or so clock ceiling for X86 and X64 architectures has stretched Moore's Law out a bit since 2003 or so.) The good news is, for most supercomputing workloads, moving to lots of cores all connected on the chip has no negative effect on performance. The shift to multicore processors means that supercomputer centers can cram one quarter as many server nodes into a cluster as they could do only three years ago and still deliver the same performance.

As has been the case for many years, X86 and now X64 processors dominate the Top 500 list. X86 and X64 processors accounted for 413 of the 500 machines on the 31st ranking, or 82.6 percent of machines. Interestingly--and not surprisingly given the year-long lead that Intel has had in delivering four cores in a single processor socket over Advanced Micro Devices--Intel had gained ground, and has done so at AMD's expense. Intel X64-based machines now account for 335 machines on the list, and including the 19 Itanium-based machines, the Intel Inside logo is on 354 machines on the Top 500 list for November, up significantly from the 289 machines on the list six months ago. To say that the HPC market really, really wants quad-core processors--and is ready to easily and eagerly absorb octo-core chips as soon as anyone delivers them--is an understatement. The count for machines using AMD's Opteron X64 processors has dropped to 78 systems, down from 105 machines six months ago, and machines using IBM's Power processors have similarly dropped to 61 machines, down from 85 six months ago. It is not so much that machinery is being unplugged on the list (although this does happen eventually) as it is a situation that labs buys lots of the fastest, coolest thing. Two years ago, that was the Opteron or Power processors. This time around, it is quad-core X64 chips.

But the top of the Top 500 is still held by IBM's Blue Gene/L massively parallel super, which is installed at the Lawrence Livermore National Laboratory. Blue Gene/L has been in the top spot since November 2004, and was recently upgraded to a sustained 478.2 teraflops on the Linpack Fortran matrix math benchmark, This sustained performance on Blue Gene/L gets IBM halfway to its stated goal from back in 2000 to reach 1 petaflops with the Blue Gene project. Blue Gene/L runs a variant of the SLES 9 Linux operating system from Novell and has 212,992 of IBM's PowerPC 440 dual-core processors. The second system ranked on the November list is the BlueGene/P machine installed at Germany's Forschungszentrum Juelich (FZJ), which is rated at a sustained 167.3 teraflops. Number eight and 10 on the list are also Blue Gene machines, rated at 91.3 teraflops at IBM and at 82.2 teraflops at the Brookhaven National Lab.

Number three on the Top 500 list is a new SGI Altix ICE cluster built using quad-core Xeon processors--not the Itaniums that SGI has been trying to push as aggressively as anyone could for the past six years--that has a sustained performance of 126.9 teraflops. This new cluster is installed at the New Mexico Computing Applications Center. The Altix ICE machines employ a modular blade server architecture that separates computer, memory, and I/O onto blades.

Numbers four and five on the current Top 500 list are both blade server clusters built by HP, one for an Indian industrial giant and the other for an unspecified Swedish defense agency. Tata Sons has installed a 117.9 teraflops cluster in its computational research laboratories that is comprised of 1,800 of HP's BL460c Xeon blade servers, which are plugged into its Blade System 7000 c-class chassis. (Can you remember when the COCOM export rules would have never in a million years allowed such a powerful computer to be exported from the United States?) This blade cluster runs Linux and uses an InfiniBand interconnect. The 2,200-node blade cluster installed in the defense of Sweden that ranks number five on the list also runs Linux and uses InfiniBand to lash server nodes together; it is rated at 102.8 teraflops.

The three remaining machines in the top 10 are from Cray, and are based on the Linux-Opteron cluster design that made its debut in the "Red Storm" machine that Cray built for Sandia National Laboratories ad which has been commercialized as the XT3, XT4, and XT5. In fact, number six on the list is the Red Storm machine itself, which runs Cray's Unicos Unix variant, has 26,569 Opteron cores, and is rated at 102.2 teraflops of sustained performance. Number seven on the list is the "Jaguar" XT3/XT4 hybrid installed at Oak Ridge National Laboratory, which runs a mix of Unicos and SUSE Linux and which is rated at 101.7 teraflops. Rounding out the top 10 is a machine dubbed "Franklin," which is an XT4 cluster installed at Lawrence Berkeley National Laboratory. This machine has 19.320 Opteron processor cores and is rated at 85.3 teraflops. It also runs a mix of Unicos and Linux. Like SGI, Cray is moving to a blade-style form factor with its XT5 machines.

Linux has ruled the Top 500 list for quite some time, and the advent of a Windows alternative from Microsoft has not, as yet, changed that. Linux is the sole operating system on 426 of the 500 machines in the November ranking, or 85.2 percent of the machines, and is also usually the other operating system in the 34 hybrid platforms that make up another 6.8 percent of machines on the list. Commercial Unixes, FreeBSD, and Mac OS together make up 6.8 percent of the list, and Unix is usually the other side of any hybrid machine on the list, so Unix can be said to have a 13.6 of the list--a huge fall from the early days of the Top 500 list when very few people had heard of Linux. Windows now has six systems on the list, giving it a 6 percent share of machines, up from two machines six months ago. But the Windows slice of the computing pie is relatively small--about 0.7 percent of the 6.97 aggregate petaflops on the list, compared to a 70.3 percent share for Linux and a 92.4 percent of shared capacity if you blend in the mixed Linux-Unix platforms. If you consider Unix-only systems, then Unix has a 6.9 percent of the total capacity on the current Top 500 list, but if you consider hybrids, then Unix has a 29 percent share of the petaflops represented on the November ranking.

Windows Compute Cluster Server has a long way to go to make a dent in the Top 500 list, but this is not Microsoft's goal anyway--at least not yet. Just like Linux attacked proprietary and Unix supercomputers from the bottom up, Microsoft wants to use its considerable influence in the scientific workstation space and among smaller, newer kinds of HPC customers who have no Unix or Linux experience--and hence no prejudices against Windows--who want to graduate up to server clusters with hundreds of gigaflops to maybe a few teraflops of performance on the Linpack test. These machines won't even show up on this list--and would not have shown up even years ago. But just the same, there is a lot of money to be made here.

In terms of vendor share of the November 2007 Top 500 supercomputer list, IBM has 232 machines, giving it a 46.4 percent share of boxes and a 45 percent share of installed petaflops on the list. HP had 166 machines on the list, and garnered a 33.2 percent share of machinery and 23.9 percent of total petaflops. Cray had only 14 machines on the list, of 2.8 percent of machines, but because of the size of its Red Storm-derived machines, had a 7.4 percent of capacity. SGI has 22 machines on the list, and 7.3 percent of capacity, and Dell has 24 machines, and a flat 7 percent of capacity. These five vendors have 458 of the 500 machines on the list, and everyone else has one, two, or a handful of boxes--Hitachi, Fujitsu, Sun Microsystems, NEC, Bull, Appro International, Linux Networx, and California Digital.


Top 500 Supers: Moore's Law Is Alive and Well

Microsoft Back on the Top 500 List of Biggest HPC Systems

Dual-Core Processors Begin Takeover of Top 500 Super Ranking

Top 500 Supers: Brace Yourself for Petaflops Systems

Linux Clusters Continue to Expand in Top 500 Supers Ranking

Top 500 Supers List Dominated By Exotic Clusters

Top 500 Supers List Dominated by Teraflops-Class Machines

Linux, X86 Clusters Take Over Top 500 Supercomputer Ranking

Top 500 Supercomputer Ranking Gets Top Heavy

                     Post this story to
               Post this story to Digg
    Post this story to Slashdot

Sponsored By

Who's ahead in the Unix vendor race?
Who has the best technology?
Which vendors do you trust?

Gabriel Consulting Group is conducting a survey to find out how real world technology professionals view Unix vendors. The first 200 respondents to tell us what they think about Unix vendors and their technology by taking our survey will receive a $10 gift certificate. All answers and email addresses will be held confidential and you will never receive spam or sales pitches from us.

Follow this link to take part in our survey.

Editors: Dan Burger, Timothy Prickett Morgan, Alex Woodie
Publisher and Advertising Director: Jenny Delroy
Advertising Sales Representative: Kim Reed
Contact the Editors: To contact anyone on the IT Jungle Team
Go to our contacts page and send us a message.

Sponsored Links

COMMON:  Join us at the annual 2008 conference, March 30 - April 3, in Nashville, Tennessee
Vision Solutions:  Fast, Easy Recovery from AIX Data Loss is the resource for job transitions after age 40



IT Jungle Store Top Book Picks

The System i Pocket RPG & RPG IV Guide: List Price, $69.95
The iSeries Pocket Database Guide: List Price, $59.00
The iSeries Pocket Developers' Guide: List Price, $59.00
The iSeries Pocket SQL Guide: List Price, $59.00
The iSeries Pocket Query Guide: List Price, $49.00
The iSeries Pocket WebFacing Primer: List Price, $39.00
Migrating to WebSphere Express for iSeries: List Price, $49.00
iSeries Express Web Implementer's Guide: List Price, $59.00
Getting Started with WebSphere Development Studio for iSeries: List Price, $79.95
Getting Started With WebSphere Development Studio Client for iSeries: List Price, $89.00
Getting Started with WebSphere Express for iSeries: List Price, $49.00
WebFacing Application Design and Development Guide: List Price, $55.00
Can the AS/400 Survive IBM?: List Price, $49.00
The All-Everything Machine: List Price, $29.95
Chip Wars: List Price, $29.95

The Four Hundred
Power6 Blades Finally Come to Market from IBM

Power Systems Division: A New Unit, i5/OS and iCluster Included

System i VIP Initiative Boosts Sales, Says IBM

As I See It: The Paradox

The Linux Beacon
Red Hat to Use Automation, Virtualization to Eat the Server Space

Red Hat Puts Out Fedora 8 Rev of Development Linux

Intel Announces First "Penryn" Xeon Processors

Mad Dog 21/21: Symphony for the Devil

Four Hundred Stuff
PowerTech Ships i5/OS Syslog Connector for SIEM

Change Management Software Gets Boost from Mighty Ant

Attachmate Ships Emulator, Touts Tolly Report

BCD Delivers Major Update of WebSmart ILE

Big Iron
IBM Acquires BI Software Specialist Cognos for $5 Billion

Top Mainframe Stories From Around the Web

Chats, Webinars, Seminars, Shows, and Other Happenings

Four Hundred Guru
V6R1 CL Enhancements

Copy Message Descriptions

Admin Alert: Five Benefits of a High-Availability System

System i PTF Guide
November 10, 2007: Volume 9, Number 45

November 3, 2007: Volume 9, Number 44

October 27, 2007: Volume 9, Number 43

October 20, 2007: Volume 9, Number 42

October 13, 2007: Volume 9, Number 41

October 6, 2007: Volume 9, Number 40

The Windows Observer
Visual Studio 2008 to Ship By End of November

Intel Quietly Releases 'Montvale' Itanium Kickers

Microsoft Unveils Free Enterprise Search Product

Windows Home Server Now Available

The Unix Guardian
Sun Wrings Profits from a Flat Fiscal First Quarter

Power6 Blades Finally Come to Market from IBM

Intel Quietly Releases 'Montvale' Itanium Kickers

IBM Brags About Its Power6 Server Shipments

Four Hundred Monitor
Four Hundred Monitor's
Full iSeries Events Calendar


Breaking News

Printer Friendly Version

Subscription Information:
You can unsubscribe, change your email address, or sign up for any of IT Jungle's free e-newsletters through our Web site at

Copyright © 1996-2008 Guild Companies, Inc. All Rights Reserved.
Guild Companies, Inc., 50 Park Terrace East, Suite 8F, New York, NY 10034

Privacy Statement