Windows Server Takes on Big Unix Boxes

by Timothy Prickett Morgan

As hard as it might be to believe this, in the entry and midrange server market, it is Unix servers, not Lintel or Wintel servers, that are setting the price/performance pace. And up in the enterprise server space, Unix dominates, but vendors of Linux, Windows, and proprietary machines are all trying to get a piece of the action, too. With Windows Server 2003 firmly established in the departments and data centers of a majority of companies in the world, it is a foregone conclusion that anyone looking at adding applications today is looking at Windows.

As with the entry and midrange markets, some vendors now offer a number of competing alternatives in the same price class within the enterprise server space. However, the ante to get into the enterprise server market is much higher, and there are fewer alternatives and fewer suppliers, even though the profits in this sector can be very high, at least compared with entry and midrange platforms.

Our Metrics for Comparing Price/Performance

Because of the way that processors, systems software, and application software are priced, most of us count processors when we think about relative performance. Counting CPUs doesn't make much sense. In fact, it makes much more sense to bracket performance in transaction processing performance bands. This, of course, is not an exact science, either, but the comparisons in this report attempt to create configurations of Windows, Unix, Linux, and OS/400 servers that have roughly from 100,000 to 1.5 million transactions per minute (TPM) of processing power.

As I usually do for the articles that appear in the IT Jungle publications, I have created a giant table of performance and price/performance characteristics for those of you who can't wait to see the bottom line results. You will find the table accompanying this story painfully detailed and, I hope, useful as you think about the big iron you are contemplating buying.

The table shows the maximum estimated performance on the TPC-C online transaction processing benchmark test for the number and speed of the processors shown for each configuration. That performance rating is based on various performance metrics from vendors and actual TPC-C test results; it is the maximum throughput of the configuration if I/O and main memory are not constrained, not the performance of the priced configuration shown. Because main memory is such a big component of the price of a server these days, we have configured a reasonable amount of main memory to each configuration, so memory prices (or, rather, the lack of memory prices) do not skew the price/performance of a configuration. The comparisons we have made are to show, as best as possible, the bang for the buck for the core central electronics complex of the server and the basic software needed to make it usable for OLTP workloads. Obviously, customers will need to add disk storage, memory, and other features as their own workloads require. It is important to realize that the TPM ratings for all of the machines in the table are based on our estimates of the throughput a heavily configured machine would have on the TPC-C online transaction processing benchmark test. We realize that this is mixing performance of a heavily configured machine and pricing on a base processor, but it is the best metric we have that can span all processor families. We want to gauge the value of a base processor configuration and isolate the CEC, operating system, and database cost.

All machines in the table have an operating system and a relational database configured for the number of processors shown. The Unix machines in the comparison have a Unix operating system--AIX 5L V5.2 on IBM's eServer p5s, Solaris 9 on Sun Microsystems' Sun Fire Enterprise machines, and HP-UX on Hewlett-Packard's Integrity servers--as well as an appropriate edition of Oracle's 10g database. The Windows machines--from HP and Unisys, which set the pack in the entry, midrange, and high-end Wintel markets--have the appropriate edition of Windows Server 2003 (Enterprise Edition on the four-way HP ProLiant boxes, and Datacenter Edition on the 16-way and 32-way Unisys ES7000 boxes) and SQL Server 2000 Enterprise Edition. The eServer i5 boxes have i5/OS V5R3 and its integrated DB2/400 database installed. For Linux systems, I showed the two Linux test results run to date on the TPC-C test, a four-way server running Red Hat Enterprise Linux 3 and a 32-way server from NEC running Novell's SUSE Linux Enterprise Server 9.

For the server comparisons, we are bracketing performance (very generally) into the following power bands: 100,000 TPM, 250,000 TPM, 350,000 TPM, 500,000 TPM, 1 million TPM, and 1.5 million TPM. Having figured out what configuration could support a given TPM rate, we went online and priced the configurations shown in the table, including the base CPU complex, a certain amount of what we deemed was base memory and disk, a respectable tape drive, an operating system, a relational database, and client access licenses (which are needed for Windows).

While these performance and price/performance comparisons are useful for comparing machines that support only a single database and an OLTP applications that is stressing it, we are aware that these comparisons do not differentiate machines based on the logical, physical, or dynamic partitioning and workload manager (which allows many different kinds of applications to share a single server) features that often differentiate these boxes and that can be easily and correctly used to justify the higher acquisition costs of one server over another with less impressive features. The comparisons in this report assume that all machines are running OLTP workloads against relational databases with all of their computing capacity dedicated to that task.

But in the real world, Windows and Linux servers rarely get much above 25 percent of peak CPU capacity, while some proprietary and Unix servers can get into the 60 to 70 percent range. As was the case with entry and midrange servers, if you assume that you can load up one machine with twice as much real work, because of its partitioning features, then a more expensive server can be in spitting distance with even cheap Lintel iron, which has very inexpensive open-source operating systems and databases that, nonetheless, need to have partitioning support added on.

The performance and price/performance table we have created for enterprise servers is based on actual performance figures on the TPC-C online benchmark tests (where available) or reasonable estimates for the maximum performance of a given processor if its memory and I/O were not bottlenecks. In the case of the i5 and p5 machines, performance estimates are based on IBM's well-documented CPW and rPerf benchmarks, which are variants of the TPC-C test. Sun doesn't do TPC-C tests anymore, but as we have said in past reports, with the performance and value of its Sun Fire Z class servers, which use Advanced Micro Devices' Opteron processors and run Sun's own Solaris Unix variant, the company might reconsider getting back into the TPC-C benchmarking game, particularly if its forthcoming boxes scale into the 250,000 TPM or higher performance bracket. On entry and midrange boxes, HP has run TPC-C tests on both its Xeon-based ProLiant servers running Windows and its Itanium-based Integrity servers, running its HP-UX Unix variant, but on the big boxes HP is really focusing on Unix, not Windows, and arguably Unisys is setting the price/performance pace in the enterprise Windows market with its ES7000 servers. Linux performance on the ProLiant machines, running either Oracle or MySQL databases, is largely based on intelligent guesswork. NEC has tested its big 32-way "Asama" Express5800 Itanium 2 server running Linux on the TPC-C test, and this is pretty much the only enterprise-class Linux box that anyone has stress-tested on an OLTP test (that will change, we are sure.)

Details on the Comparisons

Unix and Linux Servers

There is no question that IBM's "Squadron" Power5 servers offer the highest performance per processor in the server market, Unix or otherwise, and as best as we can figure, these machines are also the undisputed price/performance leader in the Unix market and in the server market at large for enterprise-class configurations. This is a position that IBM has been working toward this for 15 years, and the company hopes that its lead in performance and value will allow it to gain enough market share to become the undisputed leader in the Unix server market within the next couple of years. While this is an admirable goal, both Sun Microsystems and HP have considerably larger Unix installed bases, and stealing the next 10 points of market share away will be tougher for IBM than the past 10 points were.

Having said that, however, IBM is delivering the technology to lend credence to the idea. From the test data we have seen, IBM can demonstrate about a 16 percent performance improvement moving a p5 box from the Oracle 10g database to its own DB2 8.1 Unix database. Moreover, for OLTP workloads, moving to AIX V5.3 from AIX V5.2 (which is what we have shown in our comparisons), performance should increase another 30 percent. The reason why IBM will be able to deliver such a big performance boost is that the Power5 chips have simultaneous multithreading, or SMT, which means each processor core has two virtual instruction pipelines that look and behave like two processor cores, as far as software is concerned. IBM's implementation of SMT is just head and shoulders above anything that anyone else has ever brought to market. This means that by moving from Oracle 10g to DB2 8.1, and from AIX V5.2 to V5.3, a 64-way p5 implementation of the Squadron server can improve its performance from about 2 million to 3 million TPM, and no new hardware is required. That means that for each of the configurations we show in our price/performance table, performance can be boosted by around 50 percent without raising the price of the server. And that is why the p5s will win any comparison, be it for raw oomph or bang for the buck, for the foreseeable future--even without any discounting, which will also be happening. And that big performance increase may be why IBM could give customers pushback on deep discounts on the p5 iron.

While HP's Integrity server line running HP-UX keeps pace with the p5 Unix server line from IBM in the entry, midrange, and low end of the enterprise Unix server spaces, at list price HP is not keeping pace with the biggest Unix server configurations. One of the reasons why this is true is that the Itanium core does not do as much work as a Power5 core; it takes 96 1.5-GHz Itanium cores to do what 48 1.9-GHz Power5 cores can do on the TPC-C test. This radically increases the cost of CPU-based pricing for operating systems and databases. But aside from that, HP is also charging a slight premium on its hardware, too. To compete with IBM, HP has to rely on deep discounts to counterbalance the extra software licensing costs and to try to differentiate the Integrity line in as much as it supports HP-UX, Windows, Linux, and OpenVMS; whereas the IBM Squadron boxes can only support AIX and Linux across all processors plus a few processors running OS/400 (IBM is limiting OS/400 use on the p5 boxes to drive i5 sales). The Integrity server is a more broadly appealing platform, and for some customers that is worth a premium, because it can be used as a server consolidation platform or be repurposed wholesale at some future date, if need be.

While Sun continues to be in a neck-and-neck battle with HP to be the Unix revenue leader, its Sparc-based Sun Fire machines running Solaris 9, even those using the new dual-core UltraSparc-IV processors, do not offer the scalability on OLTP workloads (based on our estimates, not on any data Sun has published) that even the Itanium-based Integrity machines offer. (There is a reason why Sun stopped doing TPC-C tests, after all, and focuses on selective industry benchmarks that show off the multithreading capabilities of its Solaris operating system and Sparc iron.) While the Sun Fire V20z and V40z servers running Solaris offer the best bang for the buck in the entry Unix space--meeting or even beating IBM's p5s--the Sparc machines compare rather badly. They are overpriced and underpowered. The former is necessary for Sun to run at breakeven, presumably, as it shifts gears to the X86 market, and the latter just drives up software costs on its Sparc platforms so long as vendors like Oracle treat a core as if it were a CPU. And even then, by the time the major software vendors agree to price software by the CPU slot, not by the CPU core (probably by mid-2005), all of the major Unix players will have dual-core chips, and Sun will still be at the same performance disadvantage. The delay in the delivery of the UltraSparc-III and UltraSparc-IV processors by two years has damage that continually accrues. While Sun's massively multithreaded chip designs will probably prove to be the right approach over the long haul, that vision is three years away, and there is a lot of revenue to be chased in the meantime with Sparc servers built by both Sun and Fujitsu-Siemens. In contested big Unix iron accounts, Sun has nothing it can do but cut prices or walk away from deals and let Fujitsu-Siemens, IBM, or HP win. Letting Fujitsu-Siemens win is the lesser of those three evils, clearly, since there is still a Solaris account to chase in the future. Hence, the Sun-Fujitsu partnership announced in 2004.

On the Linux enterprise server front, HP and NEC currently ship big Linux machines, and Fujitsu is working on a 32-way Windows-Linux machine based on Itanium that will come out in the middle of 2005. The performance data is very thin, and there just is not enough data to make a lot of assumptions about how these machines perform on OLTP-style workloads. Pricing on big Itanium-Linux servers seems to be a bit of a premium compared with IBM's p5 servers, roughly the same as all but the biggest HP Integrity-Unix servers, and considerably lower than Sparc-Solaris boxes in the same power class.

Windows Servers

While Microsoft and its Windows server partners have tried to convince customers that Windows is the rightful alternative to Unix in the data center, this is a particularly hard sell to customers who are well acquainted with Unix and who are not exactly happy with the security of the Windows platform. The Windows desktop--however different it is from the Windows server--does not engender trust, at least not in the way that stodgy old Unix does.

Still, HP, IBM, and Unisys, offer big Windows boxes to customers that want to shift from Unix, or to those that have outgrown four-way Windows boxes with their workloads and have no need or desire to learn Unix. These machines (in the case of IBM and Unisys, but not HP) may not span the full breadth of the Unix alternatives, but they are plenty big enough for fast-growing Windows shops that do not want to move to clustered databases and do want to stick with more traditional SMP servers.

Unisys is the volume leader in the big Wintel box space, and its ES7000 machines running Windows 2003 Datacenter Edition, SQL Server 2000 Enterprise Edition, and equipped with an appropriate number of Client Access Licenses, are absolutely competitive with all but the most aggressive Unix server pricing in the 100,000 TPM to 300,000 TPM pricing bands where its machines play.

If one assumes that HP can push the performance of its Integrity line running the 64-bit version of Windows Server 2003 Datacenter Edition about as hard as it can push HP-UX, which is a reasonable assumption for TPC-C but not necessarily for real-world workloads, then the biggest Windows boxes are priced about the same as big Unix iron, and a little bit less of the CAL count can be kept down.

OS/400 Servers

Somewhat surprisingly, big eServer i5 iron is delivering good bang for the buck compared with alternatives--with some qualifications, of course. The first qualification is that customers buying such machines had better be getting around 40 percent off list price or this is not true. That is roughly the discount level in contested Unix accounts. To get that discount level, you may have to strike the fear of unplug into the heart of IBM. This is made somewhat problematic by the fact that HP is pushing Itanium machines (Itanium is not exactly looking like a good long-term bet these days, even if it does end up being one), and Sun talks a lot about Opterons, but has no big iron boxes based on it (and charges up the yin-yang and out the wazoo for UltraSparc-III and UltraSparc-IV servers). If you lean on IBM, it will probably suggest the eServer p5 machines, which among high-end servers offer the undisputed price/performance lead. IBM may say no to i5 discounts and try to force your hand to a p5 migration, which for RPG and COBOL applications is a big headache. To call IBM's bluff means porting applications to big Wintel iron from Unisys, IBM, or HP running on the 32-bit versions of Windows (not a good idea, because of the 64 GB main memory barrier in 32-bit operating systems) or on the 64-bit version of Windows (which is only yet available on Itanium machines). Even if you are running third-party applications that are supported on all of these platforms, the amount of money you save in negotiating a deep discount on a Unix or Windows box might be eaten alive by the migration off the i5. It just ends up not being worth the trouble to migrate, and IBM surely knows this.

The other qualification is that the performance data for the eServer p5 systems is showing these machines at their worst. IBM has only run a few TPC-C tests on the p5s so far, but the indications are that the company is going to be able to substantially improve performance from tuning and from moving up to AIX 5L V5.3. The gap between i5 and p5 performance on the same exact iron is going to get wider with time.

The original eServer i5s were announced in May 2004, and throughout the summer IBM fleshed out the line, which now spans from uniprocessor Model 520 Express boxes all the way up to the 64-way Model 595s. As the line was being fleshed out, we have been presenting a series of articles that compare the performance and price/performance of the new i5s against the prior generation of iSeries machines, as well as against Windows, Linux, and Unix alternatives. The Squadron eServer i5s, like their predecessors, the iSeries, sometimes hold their own against other platforms, as is the case with big iron configurations, and sometimes they don't do quite so well, such as the smaller i5 machines.

IBM is charging a pretty hefty premium on the very small machines that are the feeder systems into the OS/400 platform, and while this may seem inexplicable or counter-intuitive, there is a reason for it. We think IBM is deliberately trying to boost sales and profits per machine in the entry i5 market, for both itself and its reseller partners, rather than going for a higher-volume, lower-price-tag marketing approach, like the Windows and Linux markets are doing. If the job of marketing means doing the math on pricing and then the evangelizing a platform's selling points against the competition, in the entry i5 market, IBM is not really marketing so much as pretending the i5 is in a market unto itself. And if it is not careful in the coming years, the entry i5s really will be isolated, just like the Multiprise 3000 and zSeries 880s are today. If you have never heard of these two machines, you are not alone. They are niche entry mainframes that very few people have ever heard of, and that is true because of IBM's protectionist pricing practices, which prop up profits in the short term but have lost the company a customer base over the long haul.

For big iron boxes, by which we mean servers that are capable of supporting thousands of real end users hammering away on complex online transactions and can handle several hundred thousand transactions per minute or more, the i5s are not looking so bad. The list prices IBM has set for i5 Model 570 and 595 servers running OS/400 Standard Edition (that's the one without 5250 green-screen transaction processing activated) are roughly in the same ballpark as big iron systems running Windows, Unix, and Linux when you put the same main memory on the boxes and add in the cost of the operating system and a relational database management system. And because IBM has seriously curtailed the charges for green-screen processing capacity, which is delivered through 5250 Enterprise Enablement features, on the eServer i5s, even big Model 570s and 595s running OS/400 Enterprise Edition are not all that much more expensive than machines without that 5250 capacity turned on. (IBM is basically charging big bucks for the first six processors using 5250 capacity, and is then giving it away for free on any additional processors.)

There's no surprise why this is the case: IBM cannot afford to alienate its largest OS/400 customers, which have complained about the high cost of 5250 interactive processing capacity in the past seven years. The intended effect of this partial change of heart related to 5250 capacity pricing (or the side effect of this change, if it was not IBM's intention) is that for the first time in many years, the high end of the OS/400 platform is without a question competitive with alternative platforms.

We are also happy to report that IBM seems to be discounting nicely on its big Model 570 and Model 595 servers. Big OS/400 shops generally got their AS/400 and iSeries servers with discounts that ranged from 30 to 40 percent over the past several years. Recently, one OS/400 shop that bought a bunch of the 16-way versions of the Model 570s said that it was able to get those machines with a 45 percent discount, adding that IBM was hot to trot on the deal. That is about the discount level that Unix players were giving at the height of the dot-com boom and in the aftermath of the bubble bursting, when they were grinding against each other trying to win any deal they could. We would be happier if IBM and the other server makers would just set list prices for their servers that were their street prices, and thereby removing all of the wheeling, dealing, and shenanigans.

You can laugh now.