The Four Hundred

No matter what technology you buy--a computer, a car, a cell phone--as a prospective buyer you want to be able to rank various options, and those who have already purchased a product want to reaffirm the choices they have already made and are living with day to day. That's why analysts at Gartner created a simple mathematical model that compares and contrasts the dominant server platforms. That model confirms what many OS/400 shops know all too well: the iSeries has great technology, but its market momentum is not so hot.

Gartner created a spreadsheet-based server-evaluation model for its own internal use in 1999 to help gauge the relative strengths and weaknesses of various server platforms. The company has since published six updates of that server model, the past several of which have been available to the public and to Gartner customers, provided they know where to look for them. In mid-October 2002, Hewlett-Packard got permission from Gartner to publish the online transaction processing (OLTP) server evaluation model for August 2002 on its Web site.

The Gartner server evaluation model is comprised of raw server rankings in 23 categories that are grouped into three major areas. For each server running a specific operating system--such as an iSeries running OS/400 V5 or an Intel-based server running Windows 2000--each category is given a raw rating of between 1 and 10. The total raw best rating that any server could have, then, is 230 points. However, since different technology factors, market dynamics, and vendor/channel business practices are more or less important, depending on how a server platform is used, this raw data is weighted by Gartner analysts to rank servers for different workloads. Gartner has created rankings for OLTP, ERP, and data warehousing workloads, and it could easily create one for various infrastructure workloads as well. The latest Gartner server evaluation model, and the one that we are citing for this story, has been generated for OLTP workloads, the very kind that are the bread-and-butter jobs done at OS/400 shops.

Because comparing all the different servers from all of the vendors would be time consuming, the Gartner server comparison model looks only at the flagship machine available in each category from each vendor. With the HP-Compaq merger, Gartner knocked out the Compaq AlphaServer running its Tru64 Unix from the comparisons, for instance. There's nothing wrong with this machine, except that HP has sunsetted the product and will stop selling it in a few years. In the past, Gartner kept proprietary midrange and mainframe servers and Unix servers in one section of the report and discussed generic Intel platforms and 32-way Intel-based boxes from Unisys separately, using slightly different criteria. This time around, all the servers are put side by side, and can be cross-examined. This is exactly the kind of comparison that vendors love when they are winning, and hate when they are losing, but IT managers have to make these assessments no matter how the vendors feel, so Gartner is sticking its neck out a bit here.

The three broad areas that Gartner ranked servers by are related to technology, market momentum, and business practices. In the technology area, Gartner ranked the processor technology, system performance, clustering, planned downtime, unplanned downtime, disaster tolerance/recovery, workload management, operating system partitioning, and manageability of the servers and their operating systems. That's ten categories for a maximum raw score of 100 points, which only IBM's zSeries 900 mainframe attained, although HP's Tandem NonStop platform was close behind with a rating of 91.

The market momentum area was comprised of six different categories: architectural viability, ISV enthusiasm, application portfolio, demand creation and fulfillment, client radar (a separate Gartner rating from other studies), and delivery on promises. The platforms from IBM, HP, and Dell running Windows Advanced Server swept this category, coming up only a little short on demand creation and delivering on promises. Momentum, at least as Gartner's raw data is concerned, is with the Windows platform, followed close behind by the well-established Unix platforms (Solaris, AIX, and HP-UX). The Unisys ES7000 fared worse than the generic Windows boxes because of its dependence on Windows 2000 Datacenter Server, which has not taken the market by storm even though it is a credible alternative to Unix or proprietary platforms in many cases. IBM's zSeries and iSeries platforms had relatively low ratings in the market momentum category, with a rating of 38 and 37 out of 60 respectively, and the HP NonStop boxes had the lowest score in this area with a ranking of 28.

The final broad area in the Gartner server evaluation model was business practices, which was comprised of seven categories: ease of doing business, support, capacity on demand, vendor financial strength, negotiation opportunity, professional services, and availability of skills. Out of a total of 70 points in the raw data rankings, most vendors came in at between 44 and 51 points. The key differentiator on OLTP workloads, says Gartner, is support, where there were some differences, which explains why the ratings for OLTP workloads vary significantly across the platforms, even though the business practices category did not have much variance across those seven categories.

Tweaked for the OLTP workloads, IBM's zSeries mainframe got a rating of 370 on technology, 164 on market momentum, and 151 on business practices. That's a total of 685 points, giving the zSeries running z/OS the top ranking among the eleven servers that were compared in the model. The iSeries "Regatta-H" Model 890 server running OS/400 V5R2 came in with a respectable fifth ranking, getting a total of 322 weighted points for technology but only 157 points for market momentum and only 147 points for business practices. That's a total of 626 points. IBM's pSeries 690 server, which is essentially the same server as the iSeries Model 890 (excepting peripheral support and OS platform), got a higher weighted score with 646 points. The pSeries 690 was rated running AIX 5L 5.1, which has less capable partitioning and workload management capabilities than OS/400 V5R2, but the pSeries 690 has much higher market momentum and business practices scores than the iSeries.

The number-two ranked machine in the Gartner OLTP server evaluation, and the reason why HP was reprinting the results of the Gartner research, was the HP Superdome platform running HP-UX 11i. It had a composite weighted rating of 648 points, just edging out the pSeries 690, which ranked third overall. The Superdome has virtual partitioning capabilities that are better than the relatively rudimentary logical partitioning capabilities in AIX 5L 5.1. In IBM's defense, AIX 5L 5.2, which was just announced in October 2002, after the Gartner report went to press, has more sophisticated partitioning, but one of the criteria for evaluation servers in the Gartner model is that features have to be in use in the field for a while before they can be weighed in. That's why the Sparc-based servers from Sun Microsystems and Fujitsu Siemens were rated using Solaris 8. Sun's latest Solaris 9 operating system may have better logical and physical partitioning, workload management, and other features, but it has not been widely used in the field. Somewhat to the embarrassment of Sun, Fujitsu-Siemens' PrimePower servers came in with 629 weighted points, giving it a fourth ranking. Sun's own "StarCat" Sun Fire 15000 servers came in with 613 points, with a sixth pole position and a ranking that was significantly behind the PrimePower servers that are its clone. Gartner gives the Fujitsu Siemens box higher scores on raw technology and business practices, which offset the significant market momentum that Sun enjoys with the Sun Fire line. In the overall rankings, the HP NonStop platform came in seventh with 579 weighted points, and it trailed the IBM zSeries and iSeries proprietary platforms as well as the several Unix platforms.

The HP NonStop platform did, however, outrank the four Windows platforms compared in the evaluation. While the IBM xSeries, HP ProLiant, and Dell PowerEdge servers running Windows 2000 Advanced Server got the highest raw marks possible for just about all of the market momentum characteristics, the technology in the Wintel platform is clearly inferior in many ways to the current crop of proprietary and Unix machines. The technology scores for these generic eight-way Wintel platforms from IBM, HP, and Dell were half or a third of the scores for proprietary mainframe or midrange servers and Unix servers. The business practices are basically a wash across all vendors. The enthusiasm for Wintel platforms in the market almost makes up for the relatively weaker technology in Wintel machines, as far as Gartner's rankings are concerned, but when the math is done, Wintel boxes still come up a little short. This ranking fits experience, certainly. But the reality is that a Wintel box of 2002 vintage is a great machine for a lot of companies on a tight budget who cannot afford to pay a big premium for servers. Unix and proprietary machines of equal power to Wintel machines often cost twice as much money. Moreover, Windows 2000 is arguably as good as any 32-bit Unix platform was in the mid-1990s--before these platforms got sophisticated workload management, partitioning, and other advanced features--and that is good enough for almost all small and midsized businesses that either don't yet need these technologies or can't afford them. Hence, the enthusiasm for what seems to be, on paper, an inferior solution.

Oddly enough, the Unisys ES7000 platform scored just under the same levels as Unix servers and did considerably better than the standard Wintel machines from IBM, HP, and Dell when it came to technology, but its overall weighted score was lower than the standard Wintel boxes. The ES7000 scored lower because enthusiasm and market momentum for Windows 2000 Datacenter Server has not been all that great. This is not the fault of Unisys, of course, but rather of Microsoft. Ironically, as Datacenter Server takes off and scores for the Unisys improve as momentum for it grows--and Datacenter Server surely will become entrenched in the IT world--the scores of IBM, HP, and Dell will also improve as they embrace this platform and deliver more scalable 16-way, 32-way, and 64-way Intel-based servers, and the gap between Unisys and these vendors will close. But Unisys will be less differentiated from the pack.

Gartner has not, as yet, seen fit to put a Linux platform in its server evaluation model, but odds are that a Lintel machine running Red Hat Advanced Server can't be far off in the future. Gartner says that Linux has not become an established OLTP platform yet, but Red Hat Advanced Server is making some serious inroads and will soon have to be considered a contender against Wintel and other platforms.