The i5 Tests Well on SAP Data Warehousing Benchmarks

by Timothy Prickett Morgan

While IBM has shied away from testing the new eServer i5 Power5-based servers on the more common TPC-C online transaction processing or the SAP Sales and Distribution (SD) ERP benchmark test, the company knows that i5 buyers want to see something that allows them to compare the performance of an i5 to alternative platforms. To that end, IBM has just released another SAP Business Warehouse (BW) benchmark result, and the i5 is holding its own against the competition.

Back in December 2004, IBM tested a four-way i5 550 with 1.65 GHz Power5 processors, equipped with 8 GB of main memory and i5/OS V5R3 on the SAP BW test, which examines the performance of a server in loading two sets of data into the warehouse (measured in rows per hour) and then running queries (which are measured in navigation steps per hour) against that data. That i5 550 server was set up using the SAP R/3 4.70 code and the 3.0 B edition of Business Warehouse, and it was able to sustain an average throughput of 34.3 million rows per hour in the first load, 11.2 million rows per hour in the second load, and 78,948 query navigation steps per hour in the query phase of the test. During the latter part of the test, the machine was able to sustain 96 percent CPU utilization.

In the most recent test, which IBM did in March and has just made public now, an i5 550 configured with i5/OS V5R3 and the new SQL Query Engine and materialized query tables (MQT) enhancement, was able to load 34.4 million rows per hour in the first load and 11.4 million rows per hour in the second load. That is a miniscule change. But with the SQL enhancements that are now available for DB2/400 starting this week through a group PTF patch to the database, IBM was able to get the CPU utilization during the query part of the data warehousing test up to 99 percent and was also able to do 90,324 navigation steps. That's an 14 percent increase of performance for adding some software patches to the system--which is basically a free upgrade as far as SQL query performance goes. By the way, the latest i5 550 being tested used the R/3 Netweaver '04 release of the ERP suite and the 3.5 release of the Business Warehouse data warehousing software from SAP. It is unclear how much of the performance gains can be attributed to changes to DB2/400, SAP R/3, and Business Warehouse, but it is probably safe to say that a lot of the performance gains were due to the SQL enhancements in DB2/400.

With these enhancements, the i5 550 offers almost the exact same performance on the SAP BW test as an eight-way xSeries 455 server using 1.5 GHz Itanium 2 processors, which Big Blue tested last summer. While the xSeries 455 had about the same load times running DB2 UDB 8.1 on top of SUSE Linux Enterprise Server 8--loading rates of 33.5 and 10.95 million rows per hour on steps one and two of the BW test--the IBM Itanium box running at 96 percent CPU could crank through 98,424 navigation steps. Of course, that Itanium system is not using the fastest 1.6 GHz/9 MB cache Itanium processors, and it is similarly not using current levels of software. IBM has announced DB2 UDB 8.2 (code-named "Stinger" and Novell has announced SUSE Linux Enterprise Server 9 since last summer.

A four-way rx5670 Itanium-based server from Hewlett-Packard running the same test on top of Windows 2003 and SQL Server 2000 was not able to load data as fast on the first test (only 24 million rows per hour), did okay on the second load (10.6 million rows per hour), but only processed 66,852 query navigation tests on the third part of the SAP BW benchmark with the CPUs running at 99 percent of capacity. A 32-bit ProLiant DL740 server with eight 2.8 GHz "Gallatin" Xeon MP processors and 8 GB of main memory running Windows Server 2003 Enterprise Edition, SQL Server 2000, and SAP R/3 4.70 could handle 37 million rows per hour on the first load, 6.7 million rows per hour on the second load, and cranked through 81,792 query navigation steps running at 95 percent of CPU capacity.

It would be very interesting to see what a four-socket DL585 using dual-core Opterons would do on this SAP BW test, and similarly, it will be interesting to see what an eight-socket DL740 will do using the new "Potomac" 64-bit Xeon MPs, which have a higher clock speed and bigger caches as well as support for 64-bit versions of Linux and Windows.

I think it would also be interesting to see some other things. First, IBM should have the i5 tested across the line on the BW test, so people can reckon the scalability of the i5 line on this very specific kind of workload. Second, IBM should have the entire i5 line tested--520 Express, 520, 550, 570, and 595--on the more meaningful SAP SD test, which examines the performance of the machine running the core SAP R/3 ERP software suite. IBM did test an i5 520 running the SD test in May 2004, but that is not the entire i5 line; scalability on SAP probably does not correlate with the relative CPW benchmarks that IBM provides for the i5 line. (By the way, IBM has run a broad set of tests on the Power5-based p5 servers, so it is not like IBM is philosophically opposed to the idea.) While there are lots of OS/400 servers supporting data warehouses, nearly all OS/400 servers run ERP software--and there is no off-the-shelf ERP software that is denser than R/3. Because of this, it is an excellent product to stress-test the systems.

I would also like to see IBM run the entire i5 line through the paces on the TPC-C OLTP test. My sources at IBM say that this is not going to happen, and they do not explain why. Considering that the iSeries is first and foremost an OLTP platform, shying away from the TPC-C and SAP SD tests is not just inexplicable, it is downright silly. Unless IBM has something to hide, of course. I don't think it does, which makes it even more perplexing.