An Update from the X64 Server Battlefields

Published: October 2, 2007

by Dan Olds

Four major vendors--Hewlett-Packard, Dell, IBM, and Sun Microsystems--are engaged in an often brutal head-to-head competition in the X64 server market. As vendors race for the biggest piece of the pie, customers are faced with conflicting claims and constantly changing technology. We believe the best way to track technology advances, figure out which new technologies are actually beneficial, and assess shifting customer perceptions about the major vendors is to go directly to the data center for answers via our ongoing Server Vendor Preference Surveys.

Gabriel Consulting Group (GCG) began surveying X64 customers in the first quarter of 2006. From the end of the first quarter of this year through the second quarter, we surveyed more respondents than ever--297 in total--on the widest range of topics and questions to date. (See the appendix of this article for demographics.) These are real-world enterprise customers who work with the systems--IT managers, architects, and administrators. They're intimately familiar with what's happening on the data center floor and can tell how they're dealing with current problems and what they plan to do in the future. As you'll see in the demographics data, 83 percent of our survey participants have servers from three or more X64 vendors, and almost a third run systems from five or more X64 vendors. This makes them supremely qualified to rank, compare, praise, condemn, and generally nitpick Sun, Dell, HP, and IBM.

You'll notice that we use something called a Vendor Preference Index (VPI) score in the charts. The VPI compares the number of "votes" a vendor gets to the number of respondents who said they have standardized on that vendor. It sounds a bit complicated, but it's actually a very simple calculation designed to level the playing field between vendors and remove respondent bias in the survey. A score of 100 on any given question is 'par', meaning that the same number of customers who have standardized on a particular system also 'voted' for that vendor on that question. A score significantly under 100 is not so good, because it means customers who have standardized on systems from one vendor voted for another vendor. A score significantly over 100 is, of course, great. This method allows us to understand how participants rate server brands against each other, not just measure how prevalent each vendor is in their data centers. A full explanation of our scoring and methodology can be found in the appendix of this report.

Our overall Server Vendor Preference survey is broken into several sections. Some cover IT trends and usage patterns, and others examine customers' perceptions of differences between vendors. A deep dive section tests customers' loyalty to their favorite brand.

This report focuses mainly on results from the Vendor Face-Off section of the survey. Subsequent reports will explore X64 customers' future purchasing plans, their approaches to the current virtualization trend, their assessments of power and facilities challenges, and even how they view the battle between Intel and Advanced Micro Devices.

The Vendor Face-Off section of the survey is designed to give customers a chance to rate the vendors head-to-head. The chart below shows the overall Vendor Face-Off results from the second quarter of 2007.

It's clear that customers see HP and IBM as the market leaders. The race between IBM and HP has become even tighter this year, with a slight lead for IBM overall on the strength of its performance in the technical categories. HP has the edge in the customer support categories, as we'll see in more detailed results below. Both companies have significant leads over Sun and particularly Dell in the overall results.

Sun turns in a respectable third place finish, with scores in most categories about the same as last year. We continue to be impressed by just how many customers have Sun X64 gear in their data centers; we weren't sure that we'd see enough Sun responses to be able to include them in the survey. Even more surprising is how consistently Sun is ranked ahead of Dell on almost every survey criteria. This is pretty good progress for a vendor who has only been in the X64 server market for a few years and still offers a limited product line.

Dell is still mired in fourth place, receiving low scores from our enterprise customers on most criteria with little change from last year. Some of this may be explained by the fact that this survey was in the field in the second quarter, during a protracted period of bad Dell publicity concerning both financial and customer satisfaction problems. While much of this doesn't directly impact enterprise product or customers, it can influence how survey respondents view a particular vendor or issue at a given point in time. Since the completion of this survey, Dell has rededicated itself to customer service and product excellence. If the company sustains this focus, we may see higher scores in subsequent surveys.

Vendor Face-Off: Technology Ratings

The chart below summarizes the scores from the technology-related questions on our survey, comparing results between the first quarter of 2006 and the second quarter of 2007. Our complete battery of questions in this section covered 14 different technology criteria; what you see here is the average result for the category as a whole.

IBM has always had a strong showing in the technology ratings, and this year was not an exception; the company managed to increase its VPI score from 103 to 108. HP also received a solid vote of confidence, moving from 98 to 103. Sun lost some ground, moving from 87 to 80, but it did show improvement on a few key criteria and even won a couple of them outright. Dell's results were essentially flat in the technology section of the survey.

Drilling down into specific questions, IBM's strength in technology was apparent with its VPI score of 132 on the Best Overall X64 Server Technology questions, compared to HP with an 85, Sun at 77, and Dell at 63. IBM also won the Highest Raw Performance category with a score of 112, although Sun managed to almost tie with its score of 110. IBM took the Observed Performance crown with a narrow (112 versus 108) win over HP. The song remains the same in the system management categories, with IBM topping all competitors by a significant margin.

HP notched wins in the hardware quality categories, beating everyone on the Hardware Build Consistency, Best Initial Quality, and Easiest to Set Up and Use questions. HP also took the lead on several of the questions dealing with reliability and availability, narrowly beating IBM on the Best Availability & Reliability Features question, with a score of 115 vs. 112. Sun and Dell trailed with scores of 67 and 64 respectively on this question. HP also gets high marks for system serviceability features, beating IBM 115 to 109 and maintaining much wider margins over Sun and Dell.

Vendor Face-Off: Customer Support/Services Ratings

This chart shows the average results on the 20 separate questions that make up the Customer Support and Vendor Business criteria.

HP sees a big improvement in this section of the survey, raising its average score from 90 to 100. IBM slips slightly, Sun remains flat, and Dell takes a bit of a nosedive in this category. HP was the only vendor to get high marks for following through on its roadmaps and delivering products on time. Its score of 100 on this question soundly trounced IBM, Sun, and Dell--all of whom were clustered in the 70 to 80 point range. HP also dominated the pack in its ability to engineer and implement Windows solutions, posting a VPI of 125; IBM and Dell scored in the 70s, and Sun trailed the pack at 57. This may change in the future, as Sun establishes ties with Microsoft. Of course, turnabout is fair play; Sun beats the field in Linux capability with a score of 107 vs. 94 for IBM, 83 for HP, and 52 for Dell.

Customers also viewed HP as the vendor who does the best job of helping them both increase their IT efficiency and reduce their costs. As a large share of X64 servers are sold and installed by channel partners, these scores may be at least partially attributed to the quality of HP's business partner organization and how it interfaces with customers.

HP and IBM tied on how well they are helping customers to virtualize and consolidate their X64 infrastructures, but a significant portion of respondents (24 percent) said there wasn't much difference between the vendors, or that they weren't sure who did the best job. HP scored highest on its desire to fuel X64 server innovation, while HP and IBM tied in terms of their X64 research and development prowess. This may be an interesting space to watch in subsequent surveys--Sun is making noise with its recent introduction of the first four-socket, quad-core X64 server, and Dell just may pull off a transformation that makes enterprise users take notice.

Conclusions?

Enterprise X64 isn't getting any less competitive, and the stakes are rising as these systems become more sophisticated and tackle increasingly critical enterprise computing chores. The largest vendors (in terms of experience and range of products), IBM and HP, still receive the highest scores from enterprise customers--even customers who have standardized on another brand.

This report is only a brief summary of data from the Vendor Face-Off section of the survey. We will be presenting much more detailed reports on individual vendor results along with data on how enterprise X64 customers view power/cooling/facilities, Intel vs. AMD, future purchasing plans, virtualization, and other timely topics. We will be posting subsequent reports on our Web site, along with authoring other articles that will appear in IT Jungle newsletters.

Appendix

Scoring & Methodology

All responses, unless noted, are compiled and normalized so that vendor scores are not skewed by the sheer size of a particular vendor's installed base. To level the playing field and determine customer preferences, not customer purchases, all survey respondents are asked to specify which X64 server vendor is their particular corporate standard, or the dominant vendor in their organization. The total number of respondents who have standardized on a particular brand is then compared to the number of "votes" for that vendor on a particular factor and scored.

For example, assume the survey had 1,200 responses, 500 of whom have standardized on Dell, 200 are strong HP customers, 300 have chosen IBM as their dominant vendor, and 200 have standardized on Sun. When asked which vendor had the best dressed salespeople, 400 participants responded Sun, 300 picked IBM, and 300 said that HP salespeople were particularly natty dressers.

While the raw scores favor Dell, the normalized score (which is simply the number of "votes" divided by the number of respondents who have standardized on that brand) shows that HP is the winner of this beauty contest. HP wins because it captured a larger number of first place votes than the number of HP respondents.

In this simple example, IBM and Sun scored at par--they were voted number one in this category by the same number as those who selected those brands as their corporate standard. Dell underperformed with its 'voters' defecting to HP.

2Q 2007 X64 Server Vendor Preference Survey Demographics

There were a total of 297 enterprise X64 respondents to this survey. Small and mid-sized businesses (SMBs) were well represented in the survey, making up 46 percent of total participants. This survey also had a reasonable number of very large enterprise participants, at a little over 13 percent. The "Servers Managed by Respondent" chart refers to the number of servers that the individual participant is responsible for, or has detailed knowledge about. An interesting data tidbit from this survey is that some of the SMBs with relatively few employees had fairly large server counts, in some cases 100 or more X64 servers. Given this, it isn't hard to understand why the server vendors are rushing to produce SMB-friendly offerings.

Over 83 percent of our respondents own X64 servers from three or more vendors. Almost a third have systems from five or more vendors, including white box or 'built it ourselves' systems. Only a very small portion--6 percent--have managed to completely standardize on a single X64 vendor. Drilling down a little deeper, we find that the major vendors are present in pretty much every account. This isn't too surprising, given the fact that such a large proportion of customers have servers from three or more vendors.

Dan Olds is the founder of Gabriel Consulting Group, an IT industry analyst firm that focuses primarily on IT as it relates to business, showing companies how technology can be deployed in a more efficient and effective manner. You can contact him at gcginfo@gabrielconsultinggroup.com.

RELATED STORIES

Unix Is Dead? It Isn't Even Sick. . .

Latest Dispatch from the Unix Server Wars

Who's Ahead in the X64 Server Wars?

Who's Ahead in the Unix Wars?

Stop Arguing About Cars and Start Managing Fleets