Unisys, IBM Further Prove Linux Performance on OLTP

by Timothy Prickett Morgan

The proponents of Linux in the enterprise have been saying that the operating system can scale, and scale well, on enterprise-class platforms that are either identical to or similar to those that run more established Unix or Windows platforms. The TPC-C online transaction processing test has a number of flaws, to be sure, but it is still one of the touchstones that server makers and server buyers use to gauge the performance of various products. And if recent benchmarks by Unisys and IBM are any indication, the new Linux 2.6 is scaling about as well as expected.

On a recent set of tests done by Unisys, the company pitted Novell's SUSE Linux Enterprise Server 9 against Red Hat's Enterprise Linux 4 on its ES7000 "Aries" 420 Itanium-based servers. The company's intent was not to just show how the two operating systems performed on the TPC-C OLTP tests, but to also show the relatively minor differences between the two Linux platforms running the same test on the same iron using the same middleware, as well as to show how Linux stacked up against Windows and Unix platforms.

Unisys ran RHEL 4 on an Aries 420 configured with 16 of Intel's 1.6 GHz/6 MB cache Itanium 2 processors. This is the biggest single system image that Unisys delivers on Itanium iron, but why Unisys didn't use the 1.6 GHz/9 MB cache version of the chip is a bit of a mystery. This machine was configured with 256 GB of main memory, 12.8 TB of disk capacity, and ran Oracle's 10g Enterprise Edition database. It was capable of cranking through 327,829 transactions per minute (TPM) on the TPC-C test, and after a relatively small 8.8 percent discount, this machine delivered a price/performance of $4.40 per TPM. The identical machine running Novell's SLES 9 cranked through 322,805 TPM (which is 1.5 percent less throughput compared to RHEL 4). Unisys is a reseller of both Red Hat and Novell Linuxes, and sells them for the same price, so the fact that the SLES 9 configuration delivered a price/performance of $4.48 per TPM is due entirely to the tiny performance difference between RHEL 4 and SLES 9 on the TPC-C test on this particular Linux server. All of the hardware and software that Unisys tested is available as of the end of July.

IBM has told its customers that Linux running on its pSeries and iSeries Power5-based servers can scale to 16 processors as well, but it has not yet delivered a version of the OpenPower Linux-only Power-based servers that scales beyond four processor cores. And that is why IBM has not yet shown the performance of Linux on a big box like the one Unisys used to demonstrate Linux performance on its ES7000s. Still, even with only four processor cores, an OpenPower or p5 570 box delivers a lot of performance and at a pretty attractive price. For some reason, IBM demonstrated its Linux performance on a p5 570, even though the OpenPower machine offers significant discounts on hardware--particularly for a heavily configured system. This is not easy to explain, but that is what IBM did. Moreover, IBM's configuration, which ran RHEL 4, is not available until February 7, 2006, and because of the TPC-C test's rules, we can't be sure why. It probably has to do with software patches on RHEL 4 and maybe IBM's DB2 8.2 database that help boost performance. But that is just a guess.

In any event, IBM took a p5 570 server with four 1.9 GHz Power5 cores, put 32 GB of main memory and 12.1 TB of disk in the machine, loaded up RHEL 4 and DB2 8.2 and was able to do 197,669 TPM at a cost of $3.93 per TPM. It is interesting to note that IBM's benchmark results came out more than a week after Unisys', and that IBM offered a stunning 44.7 percent discount to get to that price/performance level, which beat Unisys by a bit. IBM doesn't have a lot more room to discount, even if moving to the OpenPower machine (which can only run Linux, not for a technical reason, but because IBM says so). If the OpenPower box shaved off as much as 10 to 20 points on the price, IBM's sales reps would probably be unwilling or unable to shave off another 20 percent.

What IBM was probably interesting in showing more than price/performance is that it could do more TPC-C work with four cores than what a four-core Itanium box could do running Linux on the TPC-C test. Back in November 2004, HP tested a four-core Integrity rx4640 server using the 1.6 GHz/9 MB Itanium 2s running RHEL 3 and Oracle 10g. This machine had 128 GB of main memory and 19.2 TB of disk capacity on it, which is a lot of iron for a four-way server, to be sure. Even running the earlier RHEL 3 version of Linux, this machine could do 161,217 TPM at a cost of $3.94 per TPM after a 35.2 percent discount. With RHEL 4, this HP box probably could do about as much work as the Power5 box with four cores, given the substantial performance improvements in the Linux 2.6 kernel--but it is hard to say for sure.

The real question this raises for me is why Linux is not scaling well to 16 processors, since adding four times the processors (and I realize I am cutting across architectures here) only added about 70 percent more throughput. Back in October 2004, NEC tested its "Asama" Express 5800/1320Xd, which is a 32-way Itanium server that employs NUMA techniques for scalability. Running an early release of SLES 9, NEC was able to process 683,575 TPM on a server that had 32 of Intel's 1.5 GHz/6 MB cache Itanium 2 processors, plus 512 GB of main memory and 41.3 TB of disk capacity. The Express5800 machine was configured with Oracle 10g Enterprise Edition, and the whole setup cost $4.1 million, or $5.99 per TPM, after a 37 percent discount.

There is no direct Windows comparison that we can make with any of this iron, except for an earlier Unisys ES7000 Aries 420 machine that was equipped with Microsoft's Windows 2003 Server Datacenter Edition and the Oracle 10g database. This machine used slightly slower 1.5 GHz/6 MB cache Itanium 2 chips on the Aries 420, plus 128 GB of main memory and 14 TB of disk; it was able to do 291,413 TPM at a cost of $4.98 per TPM after a 14.8 percent discount. Unisys pricing has come down since this TPC-C report was published in May 2004, of course, so the disparity in price/performance may not be there. But the fact that a Windows setup did about 12 to 13 percent less work than a Linux box is probably significant.