The System i Still Owns the SAP BI Data Mart Benchmark

Published: November 5, 2007

by Timothy Prickett Morgan

It's lonely at the top, apparently. In recent posting in her blog on IBM's developerWorks site, Elisabeth Stahl, who is the program manager of performance marketing for Systems and Technology Group, was lamenting that no one else in the IT industry had yet done a test on SAP's BI Data Mart benchmark test as the System i platform was getting another machine tested on it.

"When I hike, I love to get to the top of the mountain," Stahl writes. "Then it's time to break out the peanut butter and jelly sandwiches and enjoy the view. But I do admit that I feel a bit lonely when there's nobody else up there. No one to say hi to. No one to talk about where you are from. No one to ask to take your picture."

Stahl, who has been dealing with system performance at IBM for two decades, knows exactly what the problem is, but there is no hint of this in her blog. The problem is that IBM has climbed the wrong mountain. Everyone else is playing king of the hill over on the Sales Distribution benchmark test, which has been the core SAP ERP benchmark for many, many years and which happens to be the test IBM ran on its OS/400 and AIX servers alike until Big Blue apparently got nervous about direct comparisons being made. Particularly by smart-aleks who edit newsletters and add pricing information for the SAP configurations tested on the SD test. IBM stopped running the SD test on the eServer i5 machines back in 2004.

The funny bit, to me at least, is that the i5/OS and OS/400 platform is generally used as an ERP system and only secondarily as a BI box. As is the case with many midrange customers, i5/OS and OS/400 shops tend to put data warehouses and data marts onto Windows-based servers running Microsoft's SQL Server, which has basic OLAP functionality built in. There are also pre-packaged and native data marts available from New Generation Software, jut to give one example. So the BI Data Mart benchmark seems to be exactly the wrong test to run on i5/OS if you want to show its ERP muscle. And by doing so, IBM looks like it doesn't understand its own customers and, even worse, looks like it has something to hide.

Having said all that, any benchmark test result on any AS/400, iSeries, or System i machine is to be welcomed. I don't want IBM to take away tests, but run more.

As I previously reported, IBM talked about the performance of the new 9406-MMA System i 570 using the Power6 processor back at the July launch of the machine. Earlier this year, when IBM dumped the SAP Business Warehouse (BW) benchmark it has been using instead of the SD test since late 2004 for the BI-DM test, the company tested two Power5+ machines. In January, IBM tested a four-core i5 570 with 2.2 GHz Power5+ chips and 64 GB of main memory; this machine could process 51,875 query navigation steps per hour. In April, IBM doubled up the cores and memory in this machine, and more than doubled performance to 114,687 query navigation steps per hour. Both of these machines ran i5/OS V5R4 and SAP's NetWeaver middleware. (Not WebSphere.) With the launch of the Power6-based System i 570 in July, IBM ran the BI-D test on a four-core machine using the 4.7 GHz processors, configured with 64 GB of memory. This machine was able to handle 92,716 query navigation steps per hour, a 79 percent increase over the four-core Power5+ machine tested earlier this year. (Why IBM didn't put the pedal to the metal and run it on a 16-core box is beyond me, but maybe there are scalability issues that crop up in SMP inside V5R4M5. This is not the first time that IBM or other server vendors have done this.)

In September, IBM ran the BI-DM test on a System i 520 with two 1.9 GHz Power5+ cores activated. The machine was rigged with 16 GB of main memory, and ran i5/OS V5R4, DB2/400, and NetWeaver 7.0, and with 97 percent of the CPU capacity utilized, this 520 box was able to handle 26,224 query navigation steps per hour. This is right about where you'd expect such a machine to fall compared to the other 570 machines tested.

RELATED STORIES

Power6-Based System i Performance: Your Mileage Will Vary

The i5 Shows Linear Scalability on SAP Benchmark

The i5 Tests Well on SAP Data Warehousing Benchmarks

The i5 Tops SAP Warehousing Benchmark

IBM Tests eServer i5s on SAP Benchmark

                     Post this story to del.icio.us
               Post this story to Digg
    Post this story to Slashdot