PowerVM Outshines Other VMs In IBM-Backed Report

September 17, 2012 Alex Woodie

There are many things that set PowerVM apart from other hypervisors on the market. For starters, the server virtualization products from VMware and Microsoft have a much, much bigger installed base than IBM‘s Power Systems hypervisor. But does that make them better? Not according to a recent report commissioned by IBM, which found that PowerVM is quicker to deploy, more stable at scale, requires fewer people to manage, costs less, consumes fewer resources, and delivers higher availability than the other leading brands.

IBM hired Solitaire Interglobal Ltd. (SIL) to gather operational data from companies in various industries with the goal of getting a deeper understanding of their use of virtualization products for servers, including their raw performance and their impact on the business. The company published the results of the study in a report titled Does Your Virtualization Platform Matter?, which can be downloaded here.

SIL drew information from more than 61,000 virtual environments (VMs, LPARs, WPARs, etc.) hosted by seven different hypervisor products, including PowerVM running on AIX platforms as well as VMware’s ESXi, Microsoft’s Hyper-V, Oracle‘s VM for Sparc (also known as Logical Domains or LDoms) and VM for x86 (a Xen variant), the open source KVM hypervisor controlled by Red Hat, and the similarly open Xen hypervisor controlled by Citrix Systems.

SIL attempted to qualify and quantify the performance of the hypervisor environments in various ways, including total cost of ownership, performance, staffing requirements, patching, availability, agility, system efficiency, and security. SIL repeatedly emphasizes that the data was drawn from the real production systems of the companies and government agencies that it works with in its ongoing performance survey and is not based on theoretical models. The company runs an ongoing computer performance study, which has touched more than 4,100 organizations in the last 34 years, and resulted in a massive 75 petabyte database.

While SIL’s analysis is mostly data driven, it does leave room for the subjective impressions of the hypervisor’s impact on the business, and does not rely entirely on objective performance figures. This qualified approach (“But how does VMware make you feel?” “Why, it makes me sad!”) is an interesting departure from the usually staid, data-driven world of product and performance comparisons. We’ll play along with SIL on this, with the recognition that it’s difficult to make an apples-to-apples comparison in the incredibly complex and diverse world of virtualization, and that the impression that a human has of a product is something that matters. At the same time, subjective impressions should be taken with a grain of salt–or perhaps two grains, when you consider who paid for the study.

SIL leads with the most subjective measure, customer satisfaction, which it says “is the ultimate metric of a successful implementation.” In that category, SIL finds that PowerVM has a higher customer satisfaction rating than all other hypervisors in two out of three customer groupings–large and very large. This trend repeated itself throughout the paper, where PowerVM scores very well amongst the bigger companies and sometimes had lower scores than other hypervisors among mid-sized businesses.

Microsoft’s Hyper-V had the highest score among medium-size companies in the customer satisfaction category, where PowerVM took fifth place. VMware took second place in overall satisfaction among medium-size companies, and had similar scores to Hyper-V, OVM Sparc, and Xen in large and very large segments. KVM and OVM scored well with medium sized companies, but both had poor satisfaction ratings among large and very large customers.

Similarly, PowerVM had the fewest number of end user complaints among large and very large customers, and it was basically tied with Hyper-V and ESXi in the medium-size market. PowerVM again had a middling score among mid-size companies in SIL’s next metric, operational satisfaction, which is based on the amount of downtimes, number of complaints, availability of automated tools, and efficiency of user interface; KVM took the prize in this department. But PowerVM shined in this metric among large and very large companies, while Hyper-V, VMware, and Xen also delivered decent results.

The trend continued in the total cost of ownership (TCO) category. The open source products Xen and KVM, had low TCOs among medium-size businesses, as you would expect. KVM maintained a slight edge over PowerVM among large businesses, while PowerVM had the lowest TCO among the largest companies, according to SIL. ESXi and Hyper-V delivered parity in TCO across all three customer-size segments, the report shows.

SIL attempted to demonstrate how efficiently the various hypervisors scale with its “Efficiency of Scale,” or EOS, metric. This metric analyzes how well a given hypervisor implementation scales, and is based on the availability of good scripting tools, efficient resource sharing, and a non-punitive licensing structure. SIL found that, as a PowerVM implementation scales up, very large customers realized a 19 percent savings in the cost per VM, while Hyper-V, ESXi, Xen, and the Oracle products resulted in a cost-per-VM increase of 50 to 100 percent.

PowerVM displayed better overall system availability than other hypervisors, although it scored very close to Hyper-V among midsize organizations, with about a 98.5 percent availability score. PowerVM availability dropped to 98 percent among midsize companies, before returning to about 98.5 percent for very large installations. OVM Sparc and ESXi were the only other hypervisors to deliver availability in the 96 to 97 percent range among very large installations (Hyper-V dropped to below 93 percent).

Hyper-V actually outshined PowerVM in the staffing category, at least among very large companies. SIL took the standard-bearer in the hypervisor category–VMware’s ESXi at a midsized company–and looked at how much time it took administrators to do standard hypervisor management tasks. It then compared the other hypervisors against this VMware baseline and calculated the variances. PowerVM required about 20 percent less work to manage in a mid-size environment than ESXi, while Hyper-V required slightly more work than ESXi. As the company size scaled up, PowerVM administrative tasks increased, while Hyper-V’s actually decreased.

The learning curve for PowerVM is also shorter, SIL found. It took just over a month to gain a base level of competency with the software, compared to nearly two months with Hyper-V and Xen, nearly three months with VMware, and more than three months with Xen, OVM for x86, and OVM for Sparc.

Implementation times were marginally better with PowerVM among medium and large organizations, according to SIL, with a typical PowerVM implementation taking between about five and 10 days. A very large PowerVM implementation takes about 15 days, according to SIL, while VMware takes about 20 days and HyperV takes about 25 days. The open source Xen and KVM hypervisors take more than 35 days to implement, according to SIL.

PowerVM smashed the competition in the “VM density” category. Medium-sized PowerVM customers run an average of eight VMs per server, compared to fewer than four for everybody else. Very large PowerVM shops run about 16 VMs per server, while only ESXi was able to approach eight VMs per server.

The higher VM density allowance of PowerVM contributed to a lower deployment cost per VM for the IBM hypervisor. SIL found that, compared to a VMware baseline, PowerVM’s cost-per-VM went down as the company (and system) size got bigger, while the costs went up with all the other hypervisors as the systems (and company sizes) got bigger.

SIL also tried to quantify how much overhead the hypervisor consumes. Again, it used VMware running in a medium-sized shop as the baseline for this calculation. It found PowerVM consumed about 30 percent less CPU and memory than ESXi in the mid-size shops. And while each of the other hypervisors consumed greater amounts of resources as the size of the company (and underlying load) increased, PowerVM stayed flat, at 70 percent of the ESXi baseline, which ultimately means that Hyper-V, ESXi, and others consumed up to twice as much resources at scale than PowerVM in SIL’s analysis.

In its report, SIL concludes that “there is a substantial advantage to incorporating the IBM PowerVM offering within an organization’s IT architecture, based on a broad set of business and performance metrics,” and that PowerVM adoptions “translate to real-world positive results experienced and reported by the businesses in this study.”

It’s hard to argue with these conclusions. However, the wide disparity in types of systems that the hypervisors are running on make a straight apples-to-apples comparison difficult. Many of the advantages of PowerVM cited by SIL–greater availability and scalability, simplified management, and lower cost of ownership–are also attributes of the Power Systems machines themselves.

Hyper-V and ESXi are restricted to running on machines that use X86 processors from Advanced Micro Devices or Intel. Since PowerVM runs only on the Power Systems boxes, it’s impossible to separate the supposed benefits from the underlying platform. IBM has done a fantastic job architecting its midrange line of servers over the past 30 years. It has implemented mainframe-class reliability systems into the machines, which make them more reliable than typical X86-class machines, but also more expensive.

PowerVM is another example of excellent system engineering by IBM. The software or its precursors have been used for decades, delivering higher utilization rates in servers long before VMware came along. However, to say than PowerVM is now the source of those reliability, scalability, and cost of ownership benefits would be wrong. The IBM hypervisor, instead, should be viewed as an integral part of its excellent Power Systems platform, and another reason why a CIO might consider that platform for his company’s IT workloads.