Pogo Linux to Ship First Commercial 8-Way Opteron Server
by Timothy Prickett Morgan
In December, Linux server and storage specialist Pogo Linux will become the first server maker to stop messing around and actually deliver an eight-way server based on the Opteron 800 series processors. Pogo Linux, which is cheekily based down the street from Microsoft in Redmond, Washington, has adopted motherboard technology from Taiwanese motherboard maker Iwill for its PerformanceWare 5864 server.
The original intent of the HyperTransport processor interconnection scheme that was developed alongside Advanced Micro Devices' 64-bit Opteron processors was to be able to create eight-way systems with a NUMA architecture without the need of a sophisticated chipset for keeping L2 caches coherent, which is necessary for regular SMP servers. AMD wanted server makers to take two-way SMP boards and use HyperTransport to glue together four of these two-way boards to make an eight-way system. For some reason, this is not how server makers have been designing Opteron servers, possibly because AMD's NUMA scheme did not yield the efficiencies that server makers wanted and their customers would expect. So Opteron chips have ended up in more traditional SMP designs with two or four processors. But just possibly, the server makers who have created Opteron designs to date--IBM, Hewlett-Packard, and Sun Microsystems (which rebadges AMD boxes from the Newisys subsidiary of Sanmina-SCI as its Sun Fire V20z and V40z)--just didn't want to listen to AMD and wanted to do things their way.
Pogo Linux was showing off the future PerformanceWare 5864 server at the Supercomputing 2004 trade show in Pittsburgh a few weeks ago, and it looks like its motherboard supplier, Iwill, is augmenting the NUMA approach that AMD had suggested for making Opteron servers and creating a modular design that AMD did not envision. The Iwill QK8S and QK8S-8P motherboards can be used to make an eight-way server, or they can also be used to create two-way and four-way machines in various form factors.
The four-way QKS8 motherboard that is at the heart of the new eight-way Opteron board is based on the AMD 8000 series chipsets (the 8131 PCI-X tunnel and the 8111 HyperTransport I/O hub) and has three components. The CPU board has four Opteron 800 series sockets and 16 DDR SDRAM main memory slots. Right now, this board supports 4GB DIMMs running at either 266MHz or 400MHz or 8GB DIMMs running at 266MHz, providing a maximum main memory per four-way board of either 64GB or 128GB. This main CPU board has HTX-Pro interfaces, short for HyperTransport Extended, that allow an I/O board and a video board to be plugged into the CPU board via the HyperTransport high-speed bus and actually reside, as equals to the CPUs, main memory, and I/O, on that bus. The HTX-Pro slots on the CPU board can also accommodate links to a dual channel InfiniBand host controller from Mellanox and an SMBus for system environmental monitoring (temperature, voltage, fan speed, and so forth). The I/O board hooks into a separate HTX-Pro port, and offers four 64-bit PCI-X slots running at 133 MHz (each on its own PCI bus) as well as two dual-port Gigabit Ethernet ports. The video board connects to the system through a cable (not the HyperTransport bus), and provides video out from the integrated VGA card on the system board as well as two serial ports, two PS/2 ports, and six USB ports. To make an eight-way server, which is called the QK8S-8P, you buy a second four-way CPU board and a second I/O board and plug them into the HTX-Pro backplane.
By separating the I/O board from the CPU board, Iwill says that it will be able to upgrade from PCI-X to PCI Express peripherals without requiring customers to buy whole new systems. Moreover, the Iwill design means that customers who only need one set of video and basic I/O outputs don't have to double buy those components to make an eight-way server. The Iwill Opteron board is very cleverly designed.
The question now is how well does an eight-way machine with 2.4GHz Opteron 850 processors with 256GB of main memory scale? That will be something that Pogo Linux will be trying to prove when it begins shipments in December of its PerformanceWare 5864. This server has been certified to run Red Hat's Enterprise Linux 3 and Novell's SuSE Linux Enterprise Server 9. Pogo Linux has not yet released prices on the machine.
Pogo Linux sells a line of two-way and four-way servers based on Intel Xeon and AMD Opteron processors. In August, the PerformanceWare servers that the company has already been shipping were updated to use the new 64-bit "Nocona" Xeon DP processors. Pogo Linux also sells uniprocessor Pentium 4 and dual Xeon Web servers, which are branded WebWare. In addition to these machines, Pogo sells preconfigured Linux Beowulf clusters with 16, 32, and 64 processor configurations, which is called the ComputeColony. The company also sells the StorageWare S548 storage array, which is based on its four-way PerformanceWare 3564 Opteron server and which crams 48 250 GB SCSI disk drives (12 TB) into a 5U space.