Competition Heats Up for Entry and Midrange Servers

by Timothy Prickett Morgan

Several months ago in our sister publication, The Four Hundred, I presented a comparison of entry and midrange boxes running Windows, Linux, Unix, and OS/400. I should have published our original analysis earlier in 2004 in this newsletter but never got around to it. But as 2005's IT budget cycle is just ramping up, it's important for you to see how Windows is stacking up to the rest of the server market, so I have updated our analysis.

Quite a few things have happened since I originally did our price/performance analysis of entry and midrange servers last summer. IBM launched the eServer p5 Unix variants of the "Squadron" Power5 machines, and Unix rival Sun Microsystems delivered similarly powered Opteron servers running Solaris, Linux, and Windows. IBM also announced the midrange eServer i5 550, which runs OS/400, and has changed the pricing on all i5s. For the Linux market in particular, IBM also launched the OpenPower 720 variant of this box, which only supports Linux and has much lower pricing than the i5 and p5 variants of the exact same machine. Intel has, to its credit, rolled out its 64-bit Xeons for two-way servers, but their impact is not yet being felt in the market, since 64-bit Xeon machines are priced essentially the same as 32-bit variants and so little software has been tweaked to run them in 64-bit mode yet. It is still a 32-bit Xeon world, from a software perspective. But that will change--though only gradually.

Most vendors now offer a number of competing alternatives in the same price class in the entry and midrange space, which makes comparisons tedious. That's why I have created a giant table that outlines the configuration, performance, and price/performance of Linux, Unix, Windows, and OS/400 servers in several different power classes. I picked the most representative vendors for each architecture, and reasonable configurations, to show how different bang is delivered for different bucks within an architecture as you move up a product line and across different platforms. The resulting price/performance comparison is not meant to be exhaustive, but illustrative.

The Metrics of Comparing Price/Performance

Because of the way that processors, systems software, and application software is priced, most of us count processors when thinking about relative performance. Counting CPUs doesn't make much sense. In fact, it makes much more sense to bracket performance in transaction processing performance bands. This is not an exact science, either, of course, but the comparisons in this report attempt to create configurations of Unix, Winows, Linux, and OS/400 servers that have roughly 25,000 TPM, 50,000 TPM, and 100,000 TPM of performance. (Those TPM ratings are, of course, transactions per minute on the TPC-C online transaction processing benchmark test; in many cases, I had to estimate the TPM rating for servers. So they are TPM's TPMs.)

I also added a category called Entry, which compares the lowest-performing server in each type that any vendor brings to market. In many cases, this is the same machine that is shown in a higher category. Think of the Entry category as the ante into a given architecture, in terms of the minimum price and performance you can start out with. The ante is low in the Wintel and Lintel space, and it is higher in the Unix and OS/400 space. (However, IBM has always slugged OS/400 server performance to artificially create lower performance points, for which it charges more money compared with other entry products. This is what happens when you have a mini-monopoly market.)

The TPM ratings for all of the machines in the table are based on our estimates of the throughput a heavily configured machine would have on the TPC-C online transaction processing benchmark test. I realize that this is mixing performance of a heavily configured machine and pricing on a base processor, but it is the best metric I have that can span all processor families. I want to gauge the value of a base processor configuration and isolate the CEC, operating system, and database cost.

That logical, physical, or dynamic partitioning and a good workload manager (which allows many different kinds of applications to share a single server) can be easily and correctly used to justify the higher acquisition costs of one server over another with less impressive features. The comparisons in this report assume that all machines are running OLTP workloads against relational databases with all of their computing capacity dedicated to that task.

But in the real world, Windows and Linux servers rarely get much above 25 percent of peak CPU capacity, while some proprietary and Unix servers can get into the 60 to 70 percent range. If you assume that you can load one machine with twice as much real work, then a more expensive server can be in spitting distance with even cheap Lintel iron, which has very inexpensive open source operating systems and databases that nonetheless need to have support added on.

The performance and price/performance table I have created for entry and midrange servers is based on actual performance figures on the TPC-C online benchmark tests (where available) or reasonable estimates for the maximum performance of a given processor if its memory and I/O were not bottlenecks. In the case of the i5 and p5 machines, performance estimates are based on IBM's well-documented CPW and rPerf benchmarks, which are variants of the TPC-C test. Sun doesn't do TPC-C tests anymore, but with the performance and value of its Sun Fire Z class servers, which use Advanced Micro Devices' Opteron processors and run Sun's own Solaris Unix variant, it might reconsider getting back into the TPC-C benchmarketing game. Hewlett-Packard has run TPC-C tests on both its Xeon-based ProLiant servers running Windows and its Itanium-based Integrity servers running its HP-UX Unix variant on entry and midrange boxes in this power class (also commendable). Linux performance on the ProLiant machines running either Oracle or MySQL databases is largely based on intelligent guesswork on my part. (While I would rather not have to make estimates at all, the relative scarcity of performance information forces one to make good guesses. I just figure it is my job to save all of you the trouble. Write once, use many times.)

I tried to put the various machines in roughly the same performance class. This is not always an easy task, since different kinds of processors and server architectures yield different performance levels. I tried to pick configurations with approximately 25,000, 50,000, or 100,000 TPM of power.

Having figured out what configuration could support a given TPM rate, I went online and priced the configurations shown in the table, including the base CPU complex, a certain amount of what I deemed was base memory and disk, a respectable tape drive, an operating system, a relational database, and client access licenses (which are needed for Windows).

Unix Is Setting the Pace of Pricing, Not Linux or Windows

While Wintel machines get a lot of credit for setting the price/performance pace in the server market, Wintel machines are, as you would expect from a quasi-proprietary platform, quite a bit more expensive than alternative Lintel iron, at least in terms of bang for the buck. In mid-2004, when I reviewed IBM's pricing for its pSeries Unix product line and Hewlett-Packard's Integrity rx Series of Itanium boxes, I said that pricing in the Unix market was not just aggressive but actually insane. Now IBM has launched the eServer p5s, and Sun has jumped in with its Opteron-based Sun Fire Opteron machines, the two-way V20z, and the four-way V40z, and pricing has gone beyond aggressive, through insane, and right out to stark raving mad. The aggressive competition among IBM, Sun, and HP in Unix, and the desire for these vendors to sell Unix against Wintel and Lintel iron in entry and midrange shops, is what is driving the server market. That is the converse of what a lot of people will tell you, but Intel iron is cheap, and configured Wintel and Lintel boxes are not necessarily cheap. That entry sticker price on an X86 server is not necessarily the best indicator of its value in terms of bang for the buck. This is important, and Linux shops have to realize this.

With the Power5-based Squadron boxes, IBM promised to keep pricing for the basic hardware components of the i5 and p5 variants of the Squadrons in lock step. I have checked this out, and it appears to be basically true. However, the price of operating system, database, and other systems software (like OS/400's 5250 Enterprise Enablement features) skew wildly apart. If you want to know how IBM can afford to charge anywhere from $.43 to $1.64 per TPM on the p5 Squadron boxes--that's with AIX and Oracle10g configured--all you need to know is that IBM is trying to charge $2 per TPM for i5 entry and midrange machines running i5/OS Standard Edition and roughly $6 to $7 per TPM for the same boxes running i5/OS Enterprise Edition (that's the one with 5250 turned on). Once again, the OS/400 server base is subsidizing IBM's aggressive market share gains in the Unix market, only this time it is doing it through much higher software fees and 5250 enablement fees, rather than also charging more for hardware. This is progress of a sort. But it is not the progress that many of us had hoped for when IBM began revamping the OS/400 server line in 2003 and again in 2004.

To be fair, the revamping of the iSeries product line has resulted in big improvements in price/performance and some lower hardware and software costs (through downshifting OS/400 software tiers), compared with the AS/400 products of the late 1990s and early 2000s. This is, of course, welcome. But the eServer i5 does not live in a vacuum. It lives in more than 245,000 data centers, where people who are just as zealous about Unix, Windows, or Linux are trying to get rid of the AS/400. Having such a price/performance gap with the outside would does not help the OS/400's cause. It's a tough call, though. While the "Green Streak" promotions IBM had for the iSeries in 2002, and the test trials of the new Express packaging, prove that there is a great deal of elasticity in the market for OS/400 wares, there may not be enough in the poor worldwide economy to actually make the i5 literally competitive with Windows, Linux, and Unix alternatives. Some pretty important people at IBM have obviously come to that conclusion; otherwise it would cut i5 software prices.

And just to be precise, the performance of the p5 machines shown in the table is based on rPerf numbers for machines running AIX 5L 5.2, last year's Unix operating system from IBM. IBM's initial rPerf estimates for the eServer p5 machines indicate that it expects to be able to get 30 percent more performance out of exactly the same p5 iron, regardless of processor make, model, or configuration, just by switching to AIX 5L 5.3. At least as far as TPC-C is concerned, AIX running Oracle has far outstripped OS/400 running DB2/400 on essentially the same iron.

IBM's p5 pricing, when coupled with Oracle's equally aggressive 10g database pricing, puts the heat on Sun, but Sun's Opteron server line can take it if AMD can crank up the processing speed and double the cores on its chips in fairly short order. HP's Integrity line of Itanium servers should be able to keep pace when the 1.9 GHz/9 MB cache "Madison" Itanium 2 chip is put into the Integrity servers, possibly as early as next week. (HP has a January 18 server announcement, but may not get the fastest Madison chips into its entry machines, only the midrange and high-end boxes. We'll keep you posted.)

Like IBM (which plays the p5 off the i5 to maximize overall server shipments and profits; those who can jump to Unix do and those who can't stay and pay), Sun can play its Opteron machines against its UltraSparcs, and HP can play its Integrities against its vast installed bases of HP 9000 (Unix), HP 3000s (proprietary), and AlphaServers (Unix and OpenVMS, which is proprietary). Prices are not coming down on UltraSparcs and HP 9000s to meet the IBM p5, Sun Opteron, and HP Integrity server pricing. These server makers are carving up their installed bases into those who want to stay with more expensive technologies that have large portfolios of software and less expensive technologies that wish they had a lot of software working on them. The cheap prices are, in effect, a twisted kind of reimbursement for the hassle of porting code to these new architectures.

One of the big factors in the server market, and one that drives architectural choices, is how Oracle packages and prices its databases. With Oracle 10g, the company added a new lower-cost database license, called Oracle 10g Standard Edition One, which costs only $4,995 per processor for two-way machines. This is a lot less expensive than 10g Standard Edition--which costs $15,000 per processor for four-way machines--and 10g Enterprise Edition--which costs $40,000 per processor for larger machines.

Nothing demonstrates this better than comparing the cost and bang for the buck on X86 servers and OpenPower servers running Red Hat and Oracle 10g. The OpenPower machine scales down to one processor, but it scales up to four, which means Standard Edition One can't run on it. So for every CPU in the box, IBM offers screaming performance and good value on the hardware, but the OpenPowers pay a $10,000 penalty per CPU on the database pricing. You can understand now why IBM is rushing to get a 1U, 2-way OpenPower 710 out the door sometime in the first quarter to actually take two-way X86 servers head-on for market share in the Linux space. By the way, the fully loaded four-way OpenPower 720E boxes beat the tar out of four-way X86 servers, in both performance and price/performance, and actually have the same or lower initial acquisition costs for equivalent configurations. When IBM brings out the OpenPower 710, it will be able to run Oracle 10g Standard Edition One, and my estimates say this two-way machine will offer more performance for the same money as uniprocessor and two-way Lintel servers.

Straightforward and simple pricing for software is something we are seeing more and more of these days, and Linux pricing has a lot to do with this. Even AIX has fairly straightforward pricing: $385 per processor on either an i5/p5 Model 520 or Model 550, and $1,225 on an i5/p5 Model 570. And it only costs a few hundred bucks to add the Virtualization Engine features to AIX 5L 5.3, which support micropartitioning and come "free" in OS/400 and i5/OS. So either it is worth more--in which case IBM should cut i5/OS prices and jack up AIX prices--or it is worth a few hundred bucks--in which case IBM should cut i5/OS prices, and by more than a few hundred bucks.

Details on the Comparisons

The devil is in the details when it comes to making any performance and price/performance claims. Here is how the servers in the comparisons in this report were put together.

Linux and Unix Servers

There is not a lot of data about Linux performance, but it is getting better. Still, we are in uncharted territory when estimating OLTP performance on Linux and open-source databases. In the estimates put together for this report, we had to guess what the performance of Oracle10g would be running the TPC-C benchmark test. Right now, Oracle has the lion's share of commercial database sales on the Linux platform, so choosing to put Oracle 10g on Red Hat (also the volume Linux operating system leader) on HP ProLiant servers (the volume Linux server maker) seems obvious enough. And guessing the performance is not all that tough: it should be about the same as a Wintel machine, plus or minus 10 percent. We decided to go conservative and lop 10 percent off the performance of a Wintel machine, just because there is simply not as much experience in the market tuning Linux as there is in tuning Windows or OS/400, for that matter.

Because open source is a real alternative, we decided to also make a second comparison, with MySQL's MaxDB database, a product of joint development between SAP and MySQL. MySQL runs on 20 different platforms, not just Linux machines, and is often hailed as an alternative to Oracle's eponymous databases, Microsoft's SQL Server, and IBM's DB2. But these databases are more familiar to corporate customers, and they have many features that MySQL is lacking. MaxDB is another open-source database that is coded and sold by MySQL, but it is based on the Adabas mainframe database, from German software company Software AG. I think this is a more appropriate database to compare with OS/400's DB2/400, particularly since it can actually run SAP's R/3 and mySAP software suites, which MySQL can't do. But there is no performance information for MaxDB. We are making a wild guess about its performance for the sake of comparison.

Lintel iron running Oracle10g Standard Edition One (the new Oracle version for two-way servers that costs $4,995 per CPU) and Red Hat Enterprise Server 3 with Standard Edition support ($799 for a license) offer excellent bang for the buck. A Linux machine with MySQL MaxDB 7.5 with entry support costs under $10,000, but if our guess about performance is right (and it may not be), this machine would offer slightly worse value for dollar.

Moving up through the performance bands (25,000 TPM, 50,000 TPM, and 100,000 TPM), the spread between Unix and proprietary servers and a Lintel box gets very wide, much wider than with Windows machines. For four-way machines, the cost of Oracle10g Standard Edition (which is for four-way machines and is not to be confused with Oracle10g Standard Edition One) on this box is $15,000 per processor; support for MaxDB 7.5 is $9,000 per processor, plus the $1,490 per processor for the license.

As we said earlier, IBM will move aggressively in the two-way Linux market with its OpenPower 710, slated for the first quarter of 2005. This machine will probably offer a slight price/performance advantage over Lintel servers and even a little bit better value than IBM's own AIX-based p5 machines.

Pricing in the Unix server market, which has IBM, Sun, and HP pitted against one another, is extremely aggressive. The Unix machines in the comparisons are configured with a Unix operating system and the appropriate level of Oracle 10g for that box. As the table clearly shows, Unix vendors are driving the price/performance curve in the entry and midrange server market, but they are doing it by getting a lot more performance out of machines and charging a bit more money for them.

Windows Servers

There are many different Windows 2003 editions, which are differentiated by the number of processors they can span, the size of the main memory they can address, and certain other features. The same holds for the SQL Server 2000 database, which has an Enterprise Edition with all the bells and whistles and costs $19,999 per processor, as well as a simpler version, called Standard Edition, which can span up to only 2 GB of main memory. Another differentiation in software pricing between Microsoft and other platforms is that Oracle 10g, Linux and Unix platforms, and OS/400 (with its integrated DB2/400 database) are all licensed for an unlimited number of users; whereas Windows Server 2003 has per-user fees (called Client Access Licenses, or CALs) that customers have to buy in addition to the basic operating system license.

Perhaps most significant, Unix and OS/400 servers have workload managers that allow many applications to run concurrently on a single machine, driving up use to 60 percent or higher for many customers; Windows has lacked a decent workload manager, and companies have had to break up workloads across multiple machines to isolate them, and they often see average server utilization in the range of 10 to 25 percent.

The Windows servers shown in this entry and midrange server comparison are based on the actual performance ratings recently published by Hewlett-Packard for selected ProLiant servers running Windows 2003. While a baby ProLiant server running Windows Server 2003 Standard Edition and SQL Server 2000 Standard Edition has lots of power, there is something that imposes a limit: SQL Server Standard Edition is limited to machines with 2 GB or less of memory. Moving to SQL Server 2000 Enterprise Edition, to address more memory, would drive the cost of SQL Server 2000 from $4,999 per processor to $19,999 per processor; and such a move would require a move from Windows Server 2003 Standard Edition to Enterprise Edition, which would raise the Windows price from $1,199 to $3,999. That's an extra $17,000.

Clearly, the software is the dominant factor in overall base server pricing, and mostly because of per-processor fees and associated Client Access Licenses. Unix, Linux, and OS/400 do not have CAL fees--they are generally an all-or-none proposition. Microsoft's per-user pricing cuts against the grain of the marketing practices in the entry and midrange server market, and it is only because the TPC-C test does not accurately reflect these CAL costs that a Windows box can be competitive at all. Microsoft doesn't like to talk about this much, as you might imagine.

OS/400 and i5/OS Servers

The entry eServer i5 servers from IBM are a bit strange, in that they have their performance slugged, and to make comparisons with these entry i5 servers easier, I tried to find Windows, Linux, and Unix machines in the same power or price class as the i5 Model 520 Express machines. This is not always easy, since IBM gears down the Model 520 Express configuration so far that you would only need a half, a quarter, or even less of a Wintel, Lintel, or RISC/Unix box to match the performance of the i5 Model 520 Express boxes. Moving up into comparisons with true i5 machines--the real Model 520s, plus the new Model 550s and the base Model 570s.

IBM wants to co-opt Linux, Windows, and Unix and bring those environments underneath the skins of the iSeries platform, and now the new Power5-based i5 servers, which can support Linux and AIX natively and Windows on X86-based co-processor cards. The i5 can't compete directly with cheap Wintel and Lintel iron, but that cheap iron is used very inefficiently; whereas and i5 server can manage lots of workloads and run at a much higher utilization. An efficient OS/400 server with Linux partitions and Windows-based Integrated xSeries Server (IxS) co-processor cards, all running at near peak performance, should offer bang for the buck that is roughly equivalent to a rack of Wintel and Lintel machines with different workloads isolated on different machines, running the same kinds of applications. It comes down to case-by-case comparisons and monitoring how workloads change over the day, the weeks, and the months of the year.

Ultimately, it is the comparison of the i5s to the iSeries that will determine how quickly customers buy a new machine (particularly those on older AS/400 machines without a formal upgrade path) or upgrade to the i5 line. While being able to demonstrate big improvements in price/performance when comparing the i5 to the iSeries is a necessary condition in order for the i5 to thrive, it is certainly not sufficient. The i5s offer roughly twice the performance per processor as the iSeries machines they replace, and anywhere from 30 to 50 percent better bang for the buck.