New Report Picks Apart Linux, Windows Security Claims
by Timothy Prickett Morgan
The Linux community, lead by legions of coders and a handful of industry luminaries who talk up the capabilities of the Linux platform, and the Windows community, led by Microsoft and its mighty marketing machine, have been in a war of words over which platform provides better security. Nicholas Petreley, a Linux analyst at Evans Data and a columnist for ComputerWorld, has painstakingly picked apart the claims of both sides in an independent report.
Rather than publish his findings in an Evans Data report or in a story at ComputerWorld, Petreley decided to release his report, "Security Report: Windows vs Linux," through the British IT publisher, The Register. Regardless of where the report is, it bears examination for anyone who is concerned with the relative security of these two competing operating system platforms.
The opening salvo of this Linux-Windows security war was fired by Forrester Research back in March, when the company released a report that showed that Windows was more secure than Linux. Forrester analysts looked at the number and frequency of publicly reported high-severity vulnerabilities and the time it takes Microsoft or the open source community to make patches for those vulnerabilities; they compared all Windows platforms between June 1, 2002, and May 31, 2003, with all variants of the Linux distributions from Debian, MandrakeSoft, Red Hat, and Novell. While the vulnerability tracking and patch distribution methods of these five platforms are quite different, and the definition of a high-severity vulnerability also has varied over time, Forrester said that it normalized these differences as much as possible to try to measure the platforms against each other. It used the National Institute of Standards and Technology's ICAT definition of high severity to classify vulnerabilities. Basically, if a vulnerability allows a hacker to violate the security of a system, or to gain control of it, or the Computer Emergency Response Team (CERT) issues a warning, then Forrester said it was a high-severity vulnerability. Forrester looked at two metrics to compare relative security between Linux and Windows: the total days that a system is at risk (from the moment that vulnerability is disclosed until the fix is deployed to the user community or Microsoft) and the time between when a fix is available and when it is deployed.
In that time frame, which is very outdated now, the Windows platform had 126 security flaws in its stack, with 67 percent of them being high-severity vulnerabilities (that's 86). Microsoft fixed all 128 flaws in an average of 25 days. Red Hat, by comparison, had 229 flaws, of which 56 percent (128) were high severity flaws. Red Hat fixed 99.6 percent of all flaws during that time, and the average days of risk for the Red Hat platform was 57, with 47 days of distribution risk. Other Linux suppliers were roughly in the same category.
As you might imagine, the Linux community didn't take this sitting down. Immediately after the Forrester report was published, the Linux distributors cited in the report released a joint statement that said Forrester was misleading, particularly since its report "erroneously treats all vulnerabilities as equal, regardless of the risk they pose." The Linux community takes the position that critical flaws that are pervasive and can cause great harm are fixed as quickly as possible--within hours if possible--and other flaws that are not as critical, but which may be given such a notation by a third party, are fixed at a much slower pace.
"Ask yourself this question," writes Petreley. "If you experienced a heart attack at this very moment, to which hospital room would you rather be taken? Would you want to go to the one with the best average response time from check-in to medical treatment? Or would you rather be taken to the one with a poor record for average response time, but where patients with the most severe medical problems always get immediate attention?" When he puts it that way, the choice is easier. But using this analogy, Petreley says what you would really want to know is mortality rates, how cases are prioritized, average physician skills, and other data.
And the Forrester report did not even acknowledge that there are known flaws in the older Windows stack that Microsoft simply refuses to fix; this is presumably an enticement for customers to upgrade--it is hard to say if it is an enticement for Windows NT and 2000 shops to move to Windows 2003 or to Linux.
In his report, Petreley basically says that the problems with Windows security are not due to its ubiquity on the desktop and in the data center, but rather are due to the monolithic design of Windows and its nature as a desktop platform where one user and his applications are given full administrator privileges, whereas in a real multiuser system like Linux and Unix, each user has his own privileges and any hack attack is isolated to his own account on that Linux or Unix system. By design, argues Petreley, a vulnerability in any program running on Windows can be used to take down Windows because of the way Windows uses remote procedure calls to link programs to each other inside a single machine or across networks. This is a fundamental difference.
So is the fact that Microsoft encourages Windows administrators to log onto servers locally with full administrator privileges, usually with Internet Explorer, whereas Linux shops almost never administer a server on that server itself (but rather go in through SSH from another machine), and Linux administrators almost never do it with root access. This main difference in the philosophy of administration is something that no security metrics have properly taken into account.
Petreley says that a more rigorous means of measuring the relative security of any platform has to include metrics for gauging the damage potential of a security vulnerability (what can someone do once they hack in), the exploitation potential of the vulnerability (how hard is it to code a hack to exploit it), and the exposure potential (can you hack from the Internet, or do you have to be a real user on the inside on the system). You can have a critical flaw, he argues, that can do a lot of damage and that is easy to code, but if it has to be done by a real user on the network and they can be caught doing it, then it may not be as critical as a flaw that can be exploited through holes in a Web browser or an email client that can be accidentally triggered by end users who are unaware of malicious code in emails or on Web pages.
Having set up a better criteria for gauging the relative security of Linux and Windows, Petreley went to the Microsoft and Red Hat sites and grabbed the latest 40 security patches for their respective Windows Server 2003 and Enterprise Linux 3 AS. Of the latest 40 Windows Server 2003 security holes, by Microsoft's own rating system, 38 percent of them are critical, the highest severity level; Petreley says he would rate between 50 percent of them critical, mainly because Microsoft turns off ActiveX controls, Java, and other features in the server version of Windows that it has to leave on in Windows XP. (But it is essentially the same Windows code, so if you turn these features on, you elevate a flaw to a critical state, even if Microsoft's scale doesn't.)
On the Red Hat software, Petreley found that only 4 of the last 40 vulnerabilities were critical, which is 10 percent. And he says that it is arguable that two of the four should be listed as critical, since they both involve the Ethereal network-sniffing tool, not Linux itself. So, in Petreley's analysis, Linux has anywhere from one-fifth to one-tenth the vulnerabilities of Windows Server 2003.
However, Petreley's report will not be the end of it. There is a fair argument that Petreley's analysis leaves out a time factor--how these platforms do in terms of vulnerabilities over time as new releases and versions come out. And if stacking and counting the vulnerabilities is important, it is also important to rate how quickly fixes for critical and less-critical vulnerabilities are made available to users. If Forrester's methods for counting and rating vulnerabilities was flawed, as many have argued, the overall security metric that Petreley proposes in his report is missing the time factor, which shows if a platform is getting better or worse over time and how frequently users have to deal with what are truly critical issues. Taking the top 40 recent vulnerabilities is interesting, but it is insufficient. And someone needs to keep track of the flaws that won't ever get fixed in an operating system platform, and those flaws have to be weighed against these platforms as well. Finally, if someone wants to be really fair, they should configure a standard Linux and Windows stack and track the vulnerabilities for those entire stacks over time using the metrics Petreley suggests.