Yankee Says Windows Improves, But Still Not Unix

Published: June 29, 2006

by Alex Woodie

A recent study by the Yankee Group found that users have experienced considerably less downtime with Windows Server 2003 compared to Windows Server 2000. The study also looked at how Windows reliability fared against other operating systems, and found that the Microsoft product was up and running more than the leading Linux distribution, although it still can't match the superior uptime of Unix systems.

Reliability is one of the most important characteristics that users should consider when making a decision on which server platform to buy. In the past, unscheduled downtime was a real thorn in Windows' side, and gave plenty of ammo to backers of more established business systems, such as the IBM mainframe and OS/400 platforms and the Unix variants that companies historically relied upon for their most critical applications.

Times have changed, and Windows has improved considerably in the reliability department. It still is not equal Unix in terms of unscheduled downtime, and isn't in the midrange and mainframe realm, where servers have been known to run continuously for years and downtime is often measured in minutes per year. But anybody with experience using the old Windows can tell you the newest Windows operating systems have made some pretty big strides in reliability.

Yankee's "2006 Global Server Reliability Survey," completed in March, provides some science and statistics to these observations. The analyst group contacted 400 businesses across the world, ranging from small businesses to large enterprises, and asked them about the scheduled and unscheduled downtime they have experienced with their servers, including the types of servers, the number of incidents, and the length of the actual outage. The results were compiled by Yankee analyst Laura DiDio, and provide a snapshot into the reliability issues facing Windows, Unix, and Linux data centers today.

According to Yankee, Windows Server 2003 recorded a 20 percent increase in reliability compared to its predecessor, Windows 2000 Server. The study shows that Windows Server 2003 users reported an average of 770 minutes of unplanned downtime per year, or about 12.8 hours, compared Windows Server 2000 users, who reported an average of 970 minutes of unplanned downtime, or 16.5 hours. These two operating systems accounted for 65 percent of the servers encountered by Yankee in the survey, the group says.

When put in percentage terms, neither of these Windows operating systems hit the 99 percent uptime level. Considering there's 8,760 hours in a year, being down for 12.8 hours of them (in Windows Server 2003's case) gives it a reliability rating of 98.54 percent, while Windows Server 2000's reliability rating is 98.12 percent.

Unix continues to be more reliable than Windows, according to the survey. About 10 percent of the respondents, or 40 organizations, reported using one of the three dominant Unix operating systems (AIX, HP-UX, and Solaris), and these users reported an average of 600 minutes of unplanned downtime, or about 10 hours, per year. This also falls short of the 99 percent uptime level as it translates into a 98.86 percent reliability rating.

There were some surprises among the Linux downtime figures. Yankee split the Linux figures between those that were using standard implementations of the distribution, and those that were using modified versions; about half of the Linux servers were using customized OSes. Red Hat Enterprise Linux (RHEL) is the big dog in the Linux camp, with a 75 percent market share. Unmodified RHEL servers recorded an average of 950 minutes of unplanned downtime, or 16 hours per year, which translates into a 98.17 percent reliability rating. Compare that to the 750 minutes, or 12.5 hours, of downtime experienced by customized versions (a 98.58 percent reliability rating). Yankee says shops that modify their Linux implementations fare better because they usually have an extremely proficient guru on the scene.

Yankee found several other Linux distros in use, and reported on their reliability numbers. However, these other Linuxes were deployed in such small numbers that it deemed the numbers statistically invalid. Just the same, they're interesting to examine, especially considering that Novell's SuSE Linux fared considerably better than Red Hat, with about 800 minutes per year of downtime, or about 13.3 hours per year (a 98.49 percent reliability rating), for the vanilla version, and about 470 minutes per year, or about 7.8 hours per year (or 99.11 percent uptime), for customized implementations. Turbolinux and Mandriva each scored about the same in terms of reliability: about 960 minutes of unplanned downtime per year for the basic versions, about the same as Red Hat. Interestingly, customizations actually hurt reliability slightly for these Linuxes. The open-source Debian fared the worst, with an average of 1,170 minutes, or about 19.5 hours, of unplanned downtime per year, which translates into a 97.78 percent reliability rating. While interesting, these figures can't be fully relied upon due to the small numbers of customers using them. Considering that Linux is the fastest growing operating system, it would behoove analyst groups like Yankee to more thoroughly flesh out these figures, no matter what it takes, but this is the best they could do with 260 of the 400 survey respondents using Windows.

Yankee analyst DiDio took the survey results as an opportunity to enthusiastically toot the Windows horn. "The Microsoft Windows server operating system platform has come of age," she writes. "It is truly enterprise worthy; its reliability is equal to or better than any mainstream platform with the present exception of the Unix distributions."

In particular, DiDio called out the downtime statistics of Windows Server 2003 versus Red Hat Enterprise Linux, writing that the Microsoft product "trounced" the Linux leader in head-to-head comparisons. The analyst concludes that the current reliability figures, combined with the improvements Microsoft has made with security and performance, make Windows Server 2003 "the most reliable, all-around server operating system in the majority of real-world scenarios."

This isn't the first time DiDio has written a report favorable to Windows. About a year ago, she authored a study comparing the total cost of ownership (TCO) of Windows and Linux systems (see "Windows Trumps Linux in Key Areas, Yankee Group Finds"). Yankee says the new study, like the one from last April, was not sponsored by anyone, and is the result of empirical research. Just the same, IT Jungle recommends you take this study--and all such studies--with a grain of salt.