Midrange Unix Servers: HP and Sun Need to Catch IBM

by Timothy Prickett Morgan

Entry servers get a lot of attention because of the huge volumes of machines that get sold quarter after quarter and the large number of unique customers who buy them. High-end servers get perhaps more than their fair share of attention in the server market because they are the flagship machines of vendors and they highlight the research and development companies have poured into their designs to either get some differentiations or to pull up even with the competition. While the entry machines get the volumes and the enterprise machines get all the glory, the midrange is still in many ways the belly of the market.

This was the case in the commercial Unix server market from Day One, and it is still true today, although everyone must concede that the definition of a midrange box is a slippery one. When PC-based servers and their operating systems were not fit for much more than print and file serving and supporting a few users running accounting applications a decade ago, X86 servers have grown into X64 machines with a great deal of power. Windows and Linux servers with performance that rivals that of midrange RISC/Unix gear have been around for several years, and they are now getting the sort of memory and processing scalability, reliability, virtualization, and system management features that have been lacking on these X86 platforms for many years.

Nonetheless, for many companies, the main server they use is a RISC/Unix box, and the kind of box they use is a midrange box that is more powerful, less costly, and more flexible than an entry Unix machine of today and of a midrange machine of years gone by. These companies have strong feelings about their Unix platform, they have created their applications and tuned them for these platforms, and they are loathe to change them. This is why the Unix market, which used to account for about half of the total worldwide server sales during the dot-com-Y2K-ERP bubbles of the late 1990s, has only contracted to 30 percent of the total market. During 2005, after several years of cut-throat competition in the Unix midrange and a dramatic shift of midrange shops to high-end entry boxes and low-end enterprise shops to midrange boxes, the Unix market has once again stabilize. Aggregate revenues go up or down a percent or two each quarter, but are not declining at the fast rates they did for the Unix space from 2001 through 2004.

Because we all know about Moore's Law, everyone was expecting this shift to happen. It just came perhaps a little earlier and moved a lot faster than many might have predicted. The server market has seen similar transitions before. The initial shift to minicomputers from mainframes in the late 1970s and early 1980s, and then another shift again as minicomputers were revamped and the economy went south in the late 1980s. Then, again when the first wave of Unix boxes came out in the early 1990s and the economy was doing badly again. Another downshift from big iron mainframes and other proprietary machines happened in the mid-to-late 1990s again, but this time it was driven not by a bad economy, but the rush to get on the Internet, fix Y2K problems, and jump on the ERP software bandwagon. This latter downshift of servers from proprietary to Unix boxes is what made Unix the supreme platform of that era when ranked by the revenues and profits they brought to their makers. The Windows market, which has knocked King Unix off its perch during 2005, is being driven by a similar downshifting. While the world's economies are stumbling along, IT departments are still in recession mode, and they are trying to cut back on the number of platforms they support and on the money they spend for servers, operating systems, and middleware. Moving off Unix and to Windows or Linux running on very respectable X64 or Itanium iron is one way to do that.

But not everyone, of course, wants to make such a move, and the price/performance improvements and discounting levels are good enough among Unix server suppliers that many companies can either make the case to jump from one Unix vendor to another or stay right where they are and worry about other issues. Some companies have Unix-based applications that are tightly tied to one or another hardware architecture--and those of you who programmed your applications using specific hardware features or undocumented APIs to boost performance know who you are--or they have deep Unix skills and they don't want to move even if an economic reason can be presented for why such a move is sensible. The fact that most IT departments are overburdened and cannot really contemplate a platform shift--regardless of the economic benefits--is what keeps all legacy installed bases--mainframes, AS/400s, and Unix servers--chugging along.

Some day, someone will say the same thing about Windows and Linux, and whatever follows them, too.

Market Overview

As I said when I went over the high-end and the entry Unix server markets earlier this year, without any doubt, IBM has been an increasingly dominant factor in the Unix server business, across all form factors and SMP scalability, since the Power4 processors debuted four years ago. IBM is, in fact, arguably the main reason that the Unix server business has seen any growth at all.

While IBM had been in the Unix racket since 1990, the company was not really a credible contender against Hewlett-Packard and Sun Microsystems in the Unix server market until the "Star" family of 64-bit PowerPC processors came to market in 1997. As the dot-com boom got under way, IBM got aggressive about smaller Unix servers and became a contender in the market for pizza-box infrastructure servers that were pioneered and sold like hotcakes (and code-named "Flapjack" to reflect their popularity) by Sun. By 2001, when IBM delivered the first dual-core processor in the world, the Power4 "Spinnaker" chip for its "Regatta" line of pSeries servers, the dot-com boom was over, the binge spending was all but done, there was a glut of server capacity in the data center, and IBM had surpassed both Sun and HP as the RISC/Unix supplier with the highest single-core performance. From this Power4 foundation, IBM has done three more revs on the processors--Power4+, Power5, and just last month with the initial Power5+ chips--and has revamped its server portfolio once (in the summer of 2004 with the p5 "Squadron" servers, with some new tweaks in two designs in the more recent entry Unix boxes from early October 2005).

IBM has been relentless in driving up the performance of its pSeries Unix machines, and in recent years, both HP and Sun have had problems with their own processor technologies, which have made it a little easier for IBM to gain some market share and get a lot of mindshare.

Hewlett-Packard, for instance, has taken a tremendous amount of grief because of its decision to stop development of its homegrown PA-RISC processors and move its entire Unix server line over to Itanium processors. The 18- to 24-month delay in the delivery of the initial Itaniums, the "Merceds," plus their relatively weak performance meant that HP had to keep the PA-8700, PA-8800, and PA-8900 chips alive and continue to develop them for its HP 9000 and Superdome servers. If the second-generation of Itaniums, the 1 GHz "McKinley" chips, had come out prior to the Power4 announcements, as was clearly the plan, HP would have been sitting pretty. And the next generation "Madison" Itaniums, which clocked at 1.5 GHz when they were announced in 2003 and were expected to be pushed up to 1.8 GHz or even a little higher in the summer of 2004, would have been more than enough to keep pace with Power4+. The dual-core "Montecito" chips might have hit the market in late 2004 or early 2005, and been right there when IBM debuted its new Power5 processors and the Squadron p5 servers that came out in the summer of 2005. But the Merced delay pushed the whole roadmap out, the Madison chip topped out at 1.6 GHz with the 9 MB on-chip cache (instead of 1.8 GHz or 1.9 GHz), and now, as of October 2005, the dual-core Montecito processors that were expected to be finished by the end of 2005 (after several delays) and rolled into new servers in early 2006 are now expected in mid-2006--and that is if Intel doesn't slip again. Adding to HP's pain is the fact that Intel has scaled back the clock speed on the Montecitos, which will come out at 1.6 GHz with 12 MB of cache per core when the design was intended to scale up to 2 GHz or even 2.2 GHz. Moreover, the "Foxton" performance acceleration feature that was expected to debut in the Montecito chip is going to be deactivated, as will be the 667 MHz front side bus that would have been necessary for the higher clock speeds. While Intel is saying it can deliver on the 2X performance increase promises it was making to Itanium customers with Montecito processors, what seems clear from all of this is that Intel was shooting for a nearly 3X increase with Montecito compared to Madisons--an increase that would have finally, without question, put Itanium at parity or better compared to IBM's Power processor family. The cumulative three to four years of delays in the Itanium processor roadmap, and no one has been hurt by this more than HP.

While Sun has done a great job in the entry Unix server market in recently months by finally delivering its "Galaxy" line of entry machines based on Solaris 10 and Opteron processors, even with the delivery of the "Jaguar" dual-core UltraSparc-IVs last year and the substantially improved performance through the "Panther" UltraSparc-IV+ chips, IBM still holds a substantial performance advantage on many workloads with the Power5 and Power5+ chips. Even with kickers to the Panthers coming and dual-core Sparc64 processors from Fujitsu-Siemens due at the end of 2006, IBM is going to hold the performance lead for the midrange until Sun gets in gear and gets more scalable Galaxy servers out the door. If Sun can deliver Galaxy boxes with up to 16 cores, it is possible that Sun can close the price/performance gap in the Unix midrange as it did in the Unix entry server space. Ironically, doing so may hurt its prospects for sales of the more expensive Sun Fire Sparc-based server line. But in the computer business, it is a wise thing for a vendor to eats its own lunch before someone else does.

As is the case with the entry RISC/Unix or Itanium/Unix machines from Sun and HP credit, their respective Galaxy and Integrity midrange server lines will be able to run Windows and Linux in addition to their respective Solaris 10 and HP-UX 11i operating systems. IBM can support its AIX 5L Unix and Linux on the its p5 servers, but it cannot offer Windows support. And only Sun can offer 32-bit X86 Linux and Windows support that has any decent performance. Running 32-bit Linux or Windows on the Itanium-based Integrity line from HP means putting up with pretty substantial degradation (about 50 percent); the Integrity machines do offer native Itanium Windows and Linux support, but customer applications have to be ported and recompiled from X86 platforms. The HP Integrity machines also run OpenVMS and the NonStop kernel, too.

The Metrics of Comparison

Midrange Unix servers are used in a wide variety of applications, ranging from the core data processing box for small and medium businesses (including applications and databases), as well as Web and other infrastructure servers for larger businesses. Increasingly, thanks to the integrated clustering capabilities in DB2 8 and Oracle10g databases, these machines are also being used in clustered database environments, particularly boxes with four or eight processor cores. Such clusters sometimes replace big SMP Unix and mainframe boxes that support OLTP or data warehousing workloads. (This doesn't happen as much as many vendors would have you believe, but it does happen, and for sound economic reasons.) While midrange Unix boxes have superior number-crunching capabilities, they are not usually used to make high performance clusters for running supercomputer applications. But sometimes, when customers need relatively powerful computing nodes with larger main memories than can be supported on entry one- or two-socket RISC/Unix servers, these machines are used in clusters. A case in point: The 63 teraflops ASCI Purple supercomputer is based on IBM p5 575 servers, a special eight-socket midrange box aimed at HPC workloads.

Gauging relative performance and price/performance in the Unix server market is a bit tricky. No vendor runs a complete set of benchmarks that describe all manner of workloads across its entire product line. And vendors try to show as many different benchmarks as possible without allowing direct comparison to competitor's boxes. So, for instance, IBM and HP will show results on the TPC-C online transaction processing test or the Linpack Fortran benchmark for supercomputing workloads, but Sun will use SPECjbbApp and SPECfpRate to show off the OLTP and number-crunching capabilities of its boxes. What this means is that there is a lot of guesswork coming up with a consistent set of metrics on which to compare the performance and therefore the price/performance of servers. IT managers and vendor salespeople have to do a lot of math on the back of envelopes.

The comparisons in this report are either real or estimated performance for midrange Unix machines running the TPC-C OLTP benchmark. Wherever possible, I have used actual benchmark results, and where not possible, I have made the best estimates I can for how a particular machine would perform based on past benchmark results, other relative performance metrics out there, and general guidance provided by vendors. In the hypothetical TPC-C comparisons shown in the table accompanying this article, I have included the cost of a reasonable base configuration of a server, including memory, disk drives, a Unix operating system, a version of the Oracle 10g database, and an AIT-3 tape drive for backup since data backup is not an option. The prices shown are for initial acquisition costs at list price, and the bang for the buck is that price divided by the estimated performance. For each vendor, I have created four general configurations: Small, Medium, Large, and Extra Large. While TPC-C is an interesting metric for comparing relative performance as well as relative changes in performance and price/performance over time, it is certainly not a suitable benchmark for all workloads. But we have to start somewhere.

How Midrange Unix Servers Stack Up

If Sun has been able to catch up to Big Blue in terms of bang for the buck in the entry Unix server market by shifting to Opteron processors and launching the "Galaxy" servers, it has not done so with its Sparc-based Sun Fire servers, at least when it comes to OLTP workloads like the TPC-C test.

As the table accompanying this report shows, for any given power class in the Unix midrange, IBM is absolutely setting the price and price/performance pace. Both Sun and HP can field machines with equivalent performance to the p5 570 Express and p5 570 machines that are IBM's key midrange Unix boxes, but IBM can do so with fewer processor cores--sometimes a lot fewer cores. And because systems software, database software, and application software is generally priced based on processor sockets or cores, IBM has a distinct price/performance advantage.

The change in Oracle's pricing for its 10g database when used on dual-core systems has helped Sun a great deal in comparisons against the single-core HP Integrity machines. This summer, Oracle announced a pricing scheme that offers customers Oracle 10g at 25 or 50 percent off, depending on the number of cores and sockets in the box. (To be precise, you multiply the number of active cores running Oracle 10g by 0.75, then round up to the nearest whole number, and then multiple by $4995, $15,000, or $40,000 depending on if you are using Standard Edition One, Standard Edition, or Enterprise Edition.) So as Sun has rolled out its faster dual-core Panther UltraSparc-IV+ processors this year, it has been able to significantly reduce the number of cores necessary to support a given TPC-C throughput. In fact, the 1.5 GHz Panther chip is, core for core, just a little bit behind the 1.5 GHz Madison Itanium processor when it comes to raw OLTP performance. This time last year, Sun and HP were both offering substantially less bang for the buck than IBM in the Unix midrange, but Sun is now about halfway between IBM and HP, and more importantly, is in the range where it can discount its way into preserving accounts.

If you think that the Montecito delays have not had an effect on HP's Integrity business, you are wrong. If the 1.6 GHz Montecitos were here right now, HP could cut the number of cores in the machines shown in the table in half and maintain the same performance. Assuming HP charged exactly the same amount for the iron and configured exactly the same memory and systems software--and charged for HP-UX 11i not based on core count, but socket count--then the cost of the resulting Integrity machines would drop by anywhere from 18 to 34 percent for the four configurations shown (the gap widens as the machines get larger). The biggest component of this price decrease would be the substantially lowered cost for Oracle 10g and HP-UX. That said, even had the Montecitos been available now, the Sun boxes would offer better bang for the buck, which is why I think HP, like Intel, was counting on being able to deliver dual-core Itaniums that ran a lot faster than 1.6 GHz. Had Intel been able to deliver a Montecito chip in the 2 GHz range today, as many had expected, then the core counts for a given amount of performance would have dropped by about 25 percent, which would have lowered Oracle 10g and HP-UX pricing even further. In fact, such a Montecito HP-UX midrange line would have a 35 to 41 percent improvement in price/performance compared to the current HP-UX Integrities, and would be delivering roughly the same price/performance on the TPC-C test as the current Sun Fire Sparc-Solaris line from Sun.

But, there is no way either company can catch IBM on this OLTP test. IBM has been criticized by its rivals for massively tuning the very design of the p5 line to run the TPC-C test very well and thus get a massive price/performance lead. While the indications are that the Power5 and Power5+ processors offer a similar advantage on other workloads, this is not always the case.

On the TPC-H data warehousing test, running against the 1 TB TPC-H database, an HP Integrity Superdome with 64 of Intel's 1.6 GHz/9 MB cache Itanium 2 processors running HP-UX and Oracle 10g was able to handle 68,100 queries per hour (QPH) at a cost of $59 per QPH. This machine had a 43 percent discount, so the list price was more like $102 per QPH for the full configurations. (Unlike my TPC-C estimates above, which gauge system peak performance against a reasonable base configuration to try to reckon the price/performance of the core systems against each other, these TPC-H numbers include a huge amount of disk storage, peripherals, and maintenance.) This setup also includes special pricing on Oracle 10g and HP-UX 11i, which artificially lowers the price. On an HP Integrity rx8640 with 16 of the 1.6 GHz/9 MB cache Itaniums, the box can handle 17,726 QPH running the Windows stack and the new SQL Server 2005 database. It is hard to say for sure, but the performance of a Unix/Oracle setup should not be all that different, although the cost could be higher than the $29.63 per QPH (after a 19 percent discount) since HP-UX and Oracle cost more (at least on the TPC tests) than Windows and SQL Server. A cluster of four p5 570 servers, each with four 1.9 GHz Power5 processors, was able to handle 26,156 QPH at a cost of $53 per QPH. Why IBM chose a cluster of p5 570s instead of a real 16-way p5 570 is unclear--it may have been trying to prove the value of its DB2 UDB database on clustered rather than SMP configurations. If you assume the running of the TPC-H data warehousing test on a cluster caused somewhere between 25 to 35 percent of the aggregate performance of the processors to be lost to the cluster, moving to a 16-way p5 570 setup might boost performance on the TPC-H test of such an SMP configuration to around 35,000 QPH. That is a lot more oomph than the Itanium box is offering. Sun has been careful to avoid direct comparisons on the TPC-H test for midrange Unix boxes, but has run a test recently on a Sun Fire V490 with four dual-core UltraSparc-IV processors running at 1.35 GHz. This machine could handle 3,446 QPH at a cost of $41 per QPH. By moving to a Panther UltraSparc-IV+ processor, Sun might be able to boost the performance on the 1 TB TPC-H test on such a small machine to around 7,000 QPH--provided the scalability promised Sun has made for the Panther chips are true. It is a reasonable guess that a 16-core Sun box running Panthers might do somewhere in the neighborhood of 25,000 QPH. But of course, the key word is might, because vendors are very careful to avoid such direct comparisons.

On the NotesBench Notes/Domino groupware test, an eight-core p5 570 server with 1.5 GHz Power5 processors was able to handle 14,740 NotesMarks on the NotesBench R6 iNotes groupware serving test. A Sun Fire V890 with sixteen 1.5 GHz UltraSparc-IV+ cores handled 21,904 NotesMarks, which is pretty good. IBM is delivering about 1,842 NotesMarks per core, compared to 1,369 NotesMarks per core for the Sun box. It would be easy to jump to the conclusion that adding cores to the Sun box would yield a lot more performance, but an E2900 with 24 of the same cores only did 28,268 NotesMarks. Those eight extra cores really were not worth the money or the trouble. It would seem that IBM can do with 16 cores what it takes Sun 24 cores to do. HP has not tested its Itanium servers on the NotesBench test, so it is very hard to reckon where these boxes might end up.