IBM Broadens Use of Opterons in System x Servers

Published: August 2, 2006

by Timothy Prickett Morgan

As has been rumored for a number of weeks, IBM yesterday unveiled a System x server line based on the "Santa Rosa" Rev F Opteron processors from Advanced Micro Devices. The five new machines will give IBM the means of better competing with Opteron-based solutions from Hewlett-Packard and Sun Microsystems, but IBM is by no means abandoning or even diminishing its use of X64 processors from Intel.

IBM didn't announce its new Opteron-based System x machines yesterday so much as pre-announce them in a way that allowed its key X64 server executives to take center stage with Hector Ruiz, AMD's chairman and chief executive officer, and to get IBM's new Opteron-based line as the first official Rev F Opteron machines in the limelight. While Sun Microsystems made a lot of noise three weeks ago with its own "Galaxy" server announcements, fleshing out its entry Opteron-based X2100, X4100, and X4200 servers from last September with the X4500 eight-socket box, the X4600 data server (a server with lots of integrated SATA storage, thus obviating the need for SAN or NAS storage), and the X8000 blade servers, these machines use the current Rev E Opterons and will require some engineering of processor-memory boards to use the Rev F Opterons, which have 1,207 pins instead of the 939 or 940 of the current Opterons. The Opteron Rev F chips were expected in July, then the date slipped to August 1, and now they are expected to come out on August 15, if the rumors are to be believed. With AMD slipping on the delivery date of the Rev F chips, server makers who have invested heavily in making new Rev F machines--and particularly IBM and Sun, which had a much skinnier Opteron line than Hewlett-Packard until now--do not want to wait for the chips to start selling their machines against the competition and to start chasing the remaining 2006 IT budget dollars. Hence, IBM seems to have turned what would have probably have been a generic Opteron announcement into an exclusive showcase, and because of IBM's strong backing of Opterons this time, AMD was in no position to argue.

Still, the event reflected well on AMD. Bill Zeitler, general manager of IBM's Systems and Technology Group, cited the "excellent" architecture of the 64-bit Opteron processors and their compatibility with existing 32-bit X86 code, and said that the Opteron "reset the agenda for the X86 part of the market and the results speak for themselves." According to the latest count from Mercury Research, AMD captured 25.9 percent of the server market in the second quarter, an increase of 133 percent. Zeitler said that 80 of the top 500 supercomputers in the world in the last ranking were based on Opterons, and that while 17 of the 18 largest financial firms had chosen its BladeCenter blade servers to deploy at least some of their applications, a third of them have deployed IBM's LS20 blade server, which uses the Opteron chip. Zeitler also said that AMD and Big Blue had extended their collaboration to develop advanced chip-making technologies, extending it to 2011 and now including 65 nanometer, 45 nanometer, 32 nanometer, and if possible, smaller wire geometries. "The word partnership is used loosely in this industry, and anyone who buys anything from anyone is called a 'partner'," said Ruiz. "This is different from that," he said, referring to AMD's partnership with IBM. "This is more than supplying silicon to IBM."

While IBM previewed the five new System x Opteron boxes, it did not provide the detailed feeds and speeds of the boxes, nor did it provide pricing on its new machines because, according to sources, AMD has yet to finalize pricing on the Rev F chips themselves. But because so much of the Opteron story for the past three years has been about efficient computing and the consequent performance per watt comparisons with Intel X86 and X64 processors as well as with various RISC/Unix alternatives, it was perhaps fortuitous that Susan Whitney, general manager of IBM's System x server division, chose yesterday as the launch day, since temperatures in New York were expected to hit 100 degrees. (Indeed, as I was writing this story back in my home office, my PC shut down because it had overheated.) With the heat raging across North America and Europe in the past several weeks, power and thermal issues are something we are all dealing with. But data center managers, who have been running out of electricity and cooling as they cram more and more computing into their data centers, have been wrestling with these issues for a lot longer and in a much more extreme way. And that is one reason why IBM is expanding its Opteron server portfolio.

The other reason--and one that no IBMer will ever say publicly--is that giving Intel some competition has been good for both Intel and its partners, including server makers such as IBM. AMD has, on both the technical and economic fronts, in the past three years defined, more than Intel, the X64 architecture. What Intel does is now measured against what AMD did and what AMD will do. And to its credit, IBM was part of that when, three years ago, it stepped up to be the first tier one server maker to support the Opteron processor. That move by IBM, more than any other event, set the stage for Intel capitulating on adding 64-bit memory extensions that were compatible with the Opteron chips to its Pentium and Xeon processors. While IBM launched an Opteron blade server a year ago, it has been slow in building out its Opteron server line, sticking with the two-socket eServer 325 and 326 and the LS20 blade server. But now that HP and Sun have a relatively broad Opteron product line, IBM is jumping in to offer competitive products. The success of the LS20 blade server, which went from zero to 30 percent of its blade server sales in a year, seems to be the reason IBM is being more aggressive. As many of us suggested to IBM three years ago, the market was ready for a full Opteron product line then, and it is certainly even more ready for it now.

The new System x servers, code-named "Armada" (reviving a code-name that IBM once used for the early Power5 server designs, but changed to "Squadron"), are about giving customers what they want and going after those customers who want a wider variety of Opteron servers. "Customers have asked us for additional AMD systems and we are responding to that," explained Whitney. And she, Zeitler, and Ruiz all explained that IBM was doing more than using AMD as a second source for X64 processors. IBM is innovating on top of the X64 architecture, much as its BladeCenter blade servers and "Summit" and "Hurricane" high-end System x Xeon MP boxes added engineering on top of Intel chips.

There are three rack-mounted Armada Opteron machines and two blade servers, and Jeff Benck, vice president of development in the System Technology Group who managed the creation of the five machines, did not want to talk too much about what other Opteron boxes IBM might have up its sleeve. But he did provide some details on the boxes. First of all, they all use the Rev F Opterons, they all use DDR2 main memory, and they are all based on the ServerWorks 2100 series chipsets from Broadcom. The System x3455 is a 1U, two-socket box that is aimed at HPC clusters as well as for Linux clusters and database clusters. It is a kicker to the existing eServer 326 machine, which uses the Rev E Opterons. The System x3655 is a two-socket, 2U server that is aimed at more generic workloads. It can scale to 64 GB of main memory. The System x3755 is a four-socket, 4U box that can have up to 128 GB of main memory, and with dual-core Rev F Opterons, it will support up to eight cores.

This System x3755 machine, like Sun's X4500, has a uniboard-style concept, putting each processor socket and its main memory on a board that plugs into the system motherboard. Customers can add up to four of these boards to the System x3755, or they can put a special card that uses the glues the three HyperTransport links in the three other processor/memory slots into a triangular configuration. What this means is that the workloads running on the server only have to make one hop to an adjacent cache memory of the other chips next to it if data it needs is not in its own cache. In a normal four-socket HyperTransport link, every third cache miss would require, on average, two hops to the cache, moving from one chip then to the other. By doing this, IBM says that it can make a three-socket box do the work of a four-socket machine using a more typical NUMA approach, and that means a six-core machine will be able to do the work of an eight-core box--without the need of paying for the extra processor or generating the extra heat. This is very clever engineering.

The new Opteron-based machine also includes a patent-pending technology called Xcelerated Memory, which will allow Opteron machines with more than four DIMM slots per socket to run at the full 667 MHz speed rather than having to drop back to 533 MHz, as other designs apparently require, according to Whitney. This allows the memory subsystems in the new IBM boxes to run with 25 percent higher memory throughput. IBM is also claiming that its Opteron designs offer a third more I/O bandwidth.

The LS21 kicker to the current LS20 blade is a two-socket blade that will fit into the older BladeCenter chassis as well as the new BladeCenter H chassis, which has a little more breathing room to deal with heat and which is a little bit taller top and bottom. The interesting new blade is the LS41, which is an LS21 blade that has some extra HyperTransport ports on it. If you want to expand a blade from two sockets to four sockets, you buy a companion LS41 blade that snaps on top of it and makes a four-socket machine. The resulting blade occupies two blade slots in the chassis. And if you need to expand I/O even more, Benck says IBM will allow other I/O expansion units to snap on top of this.

IBM said that it will announce the pricing and availability of the new System x machines when AMD launches the Rev F machines, and hedged on that date by only saying that these chips were expected to be available in the third quarter.