IBM Gooses Power Systems Storage and Networking

Published: October 24, 2011

by Timothy Prickett Morgan

As we discussed in last week's issue, on October 12 IBM rolled out some new entry and enterprise Power7-based systems with double the maximum memory capacity and with support for PCI-Express 2.0 peripheral cards, doubling up the bandwidth per slow. But there were a bunch of other enhancements to storage and networking adapters, too, and these are useful for all Power Systems shops on reasonably recent machinery.

The new SAS disk and flash drive controllers are the most interesting new devices that IBM put out in the October announcements. With solid state drives now coming into vogue to boost the I/O processing capacity of systems, you have two options. One, you can put a relatively dumb disk controller in the system and let the central processor handle most of the heavy I/O coming through flash drives--I have seen this in a flash-based supercomputer called Gordon that is being built at the San Diego Supercomputer Center. (So I ain't just making this up.) Or two, you can put more cache memory on a fatter and more capable disk controller to handle the increasing amount of I/O that an SSD can push compared to a similar number of disks. Given the relative cost of processors, I can see why you wouldn't want to do a lot of I/O processing on a Power7 machine and would want to offload as much as possible to a smart peripheral card. (And one most likely based on a PowerPC processor, in this case, if I were IBM.)

The new feature 5913 large-cache SAS controller plugs into the PCI-Express 2.0 x8 slots that are available on the new Power 710, 720, 730, 740, 770, and 780 machines that were announced on October 12 and that started shipping on October 21. It will also be available for older Power7 and Power6+ and Power6 servers through EXP12S or EXP24S external I/O drawers. This feature 5319 controller is a full-height, single-wide controller that allows customers to mix and match hard disks and SSDs on the same controller and offers three times the SSDs per slot and two times the disk drives per slot as prior SAS disk controllers; it also packs 1.8 GB of write cache, which helps boost system performance. The 5319 controller offers 6 Gb/sec SAS links to disks and controllers, which can handle data transfer rates of up to 4GB/sec; a pair of them can handle 72 disk drives, but you have to get the right enclosures to max it out. Some enclosures top out at 42 or 50 drives.

Here's how the new large-cache SAS adapter for PCI-Express 2.0 slots stacks up to the two other key SAS adapters, which plug into PCI-Express 1.0 slots (and are therefore compatible with the Gen 2 slots as well):

Interestingly, the feature 5319 controller also has flash memory as its cache, so it does not need to be backed up by memory as the prior generations of cached disk controllers in the Power Systems lineup required. So now, you don't need to go mucking about changing the batteries on the controllers when they drop dead, and if a system goes haywire, the flash memory on the controller will hold the data until the power comes back on. As has been the case for many years, IBM recommends that disk controllers be paired on mission-critical back-end systems where data loss cannot be tolerated.

The feature 5319 controllers cost a little more than three times that of the feature 5805 and 5903 controllers they replace, so having that I/O expansion does not come cheap. But at many companies, PCI slots are scare and storage capacity and I/O bandwidth demands are growing a lot faster than CPU capacity requirements, which is why IBM had to make this beefier controller and move to PCI-Express 2.0 peripheral slots. Moreover, IBM warns customers that the feature 5805/5903 controllers were really designed for disk drives and will get bogged down once you have three to five busy SSDs hanging off them; if the SSDs are not particularly busy, you can probably hang eight or nine of them off one of these older PCI Gen 1 controllers.

You may be using the old feature 5904, 5906, and 5908 PCI-X large-cache adapters in your Power Systems machines, which are very respectable as it turns out. But the new controller is still better and cost less money, as you can see:

And, as with the older PCI Gen 1 SAS controllers, this PCI-X controller was not designed to take the heavy I/O load of SSDs. It is also a double-wide card and you cannot mix and match disks and SSDs on the old PCI-X controllers. The other thing to consider is that read cache is basically useless because SSDs can just pipeline data straight through the controller into the system bus at full tilt boogie. You don't have to compensate for the slowness of a disk drive. And IBM says that for customers using disks with this fat memory PCI Gen 2 controller, don't worry about the lack of read cache on the controller unless you are getting more than 35 to 40 percent hit rates.

In addition to the new large cache SAS adapter, IBM put out a slew of new PCI-Express 2.0 adapter cards to take advantage of that extra I/O. Feature 5729 is a four-port 8 Gb/sec Fibre Channel adapter card for linking out to SAN storage; this card has twice as many ports as the previous PCI Gen 1 adapter and costs 14 percent less, too. On the entry Power Systems machines (by which I mean the Power 710, 720, 730, and 740 machines), this new card costs $6,000; on the new enterprise models (that's the new Power 770 and 780 boxes) it costs $7,859.

There is also a new Quad Data Rate (QDR) InfiniBand networking card, which runs at 40 Gb/sec; that's feature 5285 and it is a two-porter that costs $1,000 on the new entry machines and $1,310 on the enterprise boxes.

IBM also kicked out a bunch of new Ethernet adapters, which again double up the port count over their PCI Gen 1 predecessors. Feature 5287 and 5288 are two-port Ethernet adapters running at 10 Gigabit Ethernet speeds; the feature 5287 card is more expensive because it uses SFP+ cables. On the entry machines, the feature 5287 costs $3,000 and on the enterprise boxes, you're in for $3,930. Feature 5288 uses cheaper SFP cabling and costs $1,600 on the entry machines and $2,096 on the enterprise boxes.

IBM has also kicked out new low-profile features that put two 10 Gigabit Ethernet and two Gigabit Ethernet ports on a single card; this device is only supported with the Linux operating system at the moment and plug into the new entry Power Systems machines. Feature 5279 comes with SFP+ cables on the 10 GE ports (it costs $2,499), while feature 5280 has two optical ports (and costs $3,499). Feature 5287 is a two-port, full height 10 GE card for PCI Gen 2 slots that has SR cables (it costs $3,930) while feature 5288 has two 10 GE ports using SFP+ cables (it costs $2,096).

And last but not least, IBM is doubling up the capacity on the 2.5-inch SAS disks spinning at 15K RPM for IBM i, AIX, and Linux. Feature 1948 is a 300 GB disk formatted down to 283 GB for the IBM operating system that fits into first-generation 3 Gb/sec SAS bays, while feature 1879 is the same capacity but plugs into 6 Gb/sec SAS slots. Either way, the disk costs $950 in entry machines and $1,244 in bigger boxes. If you want them formatted for AIX or Linux, that's feature 1953 or 1880 (for the different small drive form factor slots) and they cost depending on the machine size.

RELATED STORIES

I/O, Memory Boosted On Entry, Enterprise Power Systems

IBM Readies October Power Systems Announcements

Start Planning For New Systems Now

More Details Emerge on Future Power7+ and Power8 Chips

The Power Systems-IBM i Road Ahead

IBM i Chief Architect Tells Us Where We're At

IBM Is Prepping Power7+ and Pondering Power8

That Faster Power 750 Motor Is Made for IBM i Shops

IBM Doubles Up Power7 Blade Sockets, Cranks Power 750 Clocks

IBM i Dominates the CPW Capacity Budget

The Little Power7 Engines That Could--And Those That Won't

Power 720: Same Entry Price, But More Room to Grow at Less Cost

IBM Ducks i Pricing on Most Entry Power7 Servers

BladeCenter S Express i Edition Gets a Power7 Upgrade

IBM Rounds Out Entry Power7 Server Lineup

IBM's Evolving Power Systems Rollout

Let's Take Another Stab at Power7 Blade Bang for the Buck

IBM's Power7 Blades Pack a CPW Punch

IBM Peddles Baby BladeCenter PS700 Express Blade Box

The Power7 Systems Sales Pitch

i/OS Gets Short Sheeted with Power7 Thread Counts

Power7: Yields Are Good, Midrange Systems A Go

The Power7 Rollout Begins In The Middle

                     Post this story to del.icio.us
               Post this story to Digg
    Post this story to Slashdot