InfiniBand Gets iSCSI Tweaks to Support Storage

Published: October 2, 2006

by Timothy Prickett Morgan

The InfiniBand Trade Association, a standards body that babysits the standards and protocols associated with InfiniBand switched fabrics and interconnections, said last week that it has ratified the iSCSI Remote Direct Memory Access storage protocol for use on the InfiniBand architecture.

The protocol, which is known as iSER, will allow for iSCSI links between servers and storage over InfiniBand fabrics rather than over Gigabit Ethernet or 10 Gigabit Ethernet. And, according to the InfiniBand enthusiasts, such connectivity will even do better than Fibre Channel links, too. Fibre Channel links, which are now available in 4 Gigabit/sec speeds, are commonly used to attach multiple servers to storage area networks, often with redundant paths. InfiniBand can deliver connections with 10 Gigabit/sec or 20 Gigabit/sec bandwidth, and according to the InfiniBand Trade Association, do so with network adapters that cost half as much.

With RDMA, a network adapter--say, a Gigabit Ethernet card running TCP/IP or an InfiniBand adapter using iSER--can take data and directly map it into the main memory of the server, bypassing the I/O subsystem and therefore allowing distributed applications to move data across servers in the network much faster. RDMA transfers do not involve processors or cache memory, which means both are available to do real data processing instead of managing data transfers off the networks from the I/O buses. Ethernet networks running TCP/IP can use RDMA through a technology known as iWarp.

InfiniBand is a switched fabric standard that was created by Intel and IBM several years ago to do away with the PCI peripheral I/O and Fibre Channel storage I/O that was common on servers. Both companies believed that it was desirable to converge all kinds of connectivity to a single standard and to create a single switched fabric linking servers to servers and servers to storage. While InfiniBand has not exactly taken the market by storm--there are very few suppliers of InfiniBand adapters and InfiniBand is not native to server motherboards--IBM is nonetheless expected to offer native InfiniBand support on its Power6 generation of servers. And that is why the iSER ratification by the InfiniBand Trade Association matters.

The iSER code running on InfiniBand is available through the OpenFabrics Enterprise Distribution of the OpenFabrics Alliance, which was founded as the OpenIB Alliance in June 2004 to promote InfiniBand standards. In March 2006, that organization expanded to include iWarp (RDMA over Ethernet) and changed its name to the OpenFabrics Alliance. Support for iSER is in the Linux 2.6.18 kernel and is available in Linux distributions from Red Hat and Novell.