Guild Companies--Intel Talks Up HPC Prowess With Big Itanium Win

Intel announced that it has sold a 20 teraflops Linux cluster based on its Itanium 2 processors, Red Hat Linux, and Quadrics interconnection switches. For the first time in many years, Intel has a system that is among the most powerful machines in the world, and for the first time, that system is based on its 64-bit Itanium 2 processors. Intel made the announcement at the recent Supercomputing 2003 trade show in Phoenix.

The cluster, which is code-named "Thunder," will be installed at Lawrence Livermore National laboratory, which will also be home to IBM's ASCI Purple and Blue Gene/L supercomputers. Thunder is comprised of 1,002 of Intel's "Tiger4" white box servers, most of them equipped with four 1.4 GHz/4 MB L2 cache Itanium 2 processors.

Lawrence Livermore is shooting for 20 teraflops, so only 3,840 processors will be in the compute nodes. Each of those 1.4 GHz "Madison" processors is rated at about 5.25 gigaflops, so when you add them up, that's 20 teraflops of aggregate raw computing power.

The Quadrics federated switch interconnect offers very low latency, according to Rick Hermann, who runs Intel's HPC division. He says, in fact, that the Thunder cluster has demonstrated 75 percent to 80 percent efficiency running the Linpack benchmark.

Perhaps the most amazing things about Thunder have nothing to do with Itanium 2, which has excellent floating point performance and, after some serious price cuts by Intel on the 1.4 GHz part, are now fairly priced against 32-bit alternatives in the Intel Xeon DP and MP lines or even the 64-bit Opteron processors from Advanced Micro Devices.

What is amazing about Thunder is that the cluster will go from idea to installed machine in under five months. It will be operational in January 2005. And equally stunning is that it will only cost $20 million. The RISC/Unix parallel supers from only a few years ago cost ten times as much for roughly the same performance. California Digital is the system integrator for the contract.

If Lawrence Livermore had gone with the 1.5 GHz parts and filled up the servers fully, it could have hit 22.4 teraflops - only a modest performance improvements - but would have cost a lot more. A 1.5 GHz Madison costs $4,227 in 1,000-unit quantities, but a 1.4 GHz chip only costs $2,247 for the same tray of chips.

What seems a real mystery is the fact that Lawrence Livermore didn't opt for the low-cache, HPC variant of Madison that Intel just announced, which runs at 1.4 GHz, has 1.5 MB of L2 cache, and only costs $1,172 per 1,000 chips. LLNL might have workloads that are cache sensitive as well as cash sensitive.

Still, the cost of the interconnection fabric on supercomputer clusters is now exceeding the cost of the base servers they link together, and memory and storage dwarf processor costs, too. You can only push these things so far.

In addition to the Thunder super deal with Lawrence Livermore, Intel will announce a new advanced computing program within its HPC division that will have a $36 million war chest to help key hardware and software ISVs, government agencies, academic institutions, and national labs to push future technologies like Infiniband and 10 Gigabit Ethernet to accelerate the use of commodity technologies in the high performance computing space.

"We're in a unique position to align the industry to solve these problems," says Herrman. Intel has a lot to gain by having its 32-bit and 64-bit architectures adopted for HPC workloads, since HPC users generally preview and debug the technologies that eventually go into mainstream computing.

Intel is already clearly gaining momentum. If you look at the semi-annual top 500 list of supercomputers that were announced at Supercomputing 2003, Intel machines have made leaps and bounds. In the most recent list, there are 189 Intel-based supercomputers, with 32 of them based on Itanium and 157 of them based on Pentium 4 Xeon processors. In the June 2002 Top 500 ranking, Intel had only three systems in the list, and then jumped to 59 systems in December 2002. In June 2003, Intel-based machines accounted for 119 systems, with 19 of them based on Itanium and 100 of them based on Pentium 4 Xeon.