SPEC Introduces MPI Supercomputer Benchmark

Published: August 2, 2007

by Timothy Prickett Morgan

The Standard Performance Evaluation Corporation last week announced a new benchmark that gauges the performance of parallel computer clusters that make use of the popular Message Passing Interface (MPI) protocols that lash the machines together so they can cooperate to do computational jobs.

The SPEC MPI2007 test was created with the assistance of Argonne National Lab in the United States and the University of Dresden in Germany as well as with support from Advanced Micro Devices, Fujitsu, Hewlett-Packard, IBM, Intel, QLogic, and Sun Microsystems. The test comes in 32-bit and 64-bit modes, and runs on Linux, Unix, and Windows servers. The current mid-sized dataset for the MPI2007 benchmark works on clusters with up to 512 processors, and SPEC is working to make a larger dataset to test more monstrous clusters.

The code in the MPI2007 test is based on a mix of actual end-user supercomputer workloads, all of which have silly names that do not mean much outside of the major supercomputing centers. The test is designed to show how SMP, NUMA, and cluster architectures using MPI scale when running different workloads, and the result is a composite score. The MPI2007 test has two metrics, a base metric that runs straight off the CD from SPEC without any tuning and a peak metric, which allows the changing of compilers and compiler flags as well as other tunings to optimize performance.

So far, AMD, HP, QLogic, and SGI have run tests, and none have offered peak metrics as yet. QLogic worked with the University of Cambridge in the United Kingdom, which tested a cluster based on Dell PowerEdge 1950 servers using Intel's dual-core "Woodcrest" 5160 processors running Linux. The servers were connected with InfiniBand fabrics. With 32 cores, this cluster was rated at 3.31 on the SPEC MPI2007 test and scaled nearly linearly until 256 cores were loaded up with the test, where it had a rating of only 22.3. At 512 cores, performance dropped off dramatically, to 33.3. Clearly, somewhere around 400 cores, adding the extra cores did not get this machine anywhere--which is why the world needs benchmarks.

HP tested its own cluster, based on the BL460c blades, using the same Woodcrest chips in dual-socket blades and using InfiniBand interconnect as well; this cluster used Red Hat Enterprise Linux 4 Update 4. With 16 cores, this machine was rated at 1.75 SPEC MPI2007, and performance started to drag by the time 128 cores were tested, with a rating of 11.9. With 256 cores, the base rating fell to 19.8, and HP did not bother testing 512 cores.

Two other clusters tested by QLogic and SGI showed similar performance, even though they used Opteron and Itanium processors instead of Intel's Woodcrest chips.

You can see the full list of SPEC MPI2007 results here. Hopefully Unix suppliers will get in the game and put out some performance numbers soon; you can bet Microsoft, the underdog in high performance computing, is itching to show it can take on Linux on X64 iron, flop for flop.

                     Post this story to del.icio.us
               Post this story to Digg
    Post this story to Slashdot