Linux Networx Chases HPC Users with Supersystems

by Timothy Prickett Morgan

The commercialization of Linux started at two different, but related, ends of the computer business--at the low end in the Web server farm and at the high-end in high-performance supercomputer clusters. In both cases, Linux was deployed because it was essentially free, tweakable, and ran on cheap--even old--X86 iron. While customers don't expect much more of a Web server than they did seven years ago, this is certainly not true of HPC clusters running Linux. Customer may choose Linux clusters for their supercomputers, but they expect a level of integration and management tools akin to those found on SMP systems. In short, they want tailored hardware and software, not raw commodities.

While there are still plenty of companies who are building their first Linux cluster to run data warehouses, simulations, engineering and design, digital rendering, and other similar applications that can take advantage of parallel processing, there are thousands of customers who have been there, done that, and they not only don't want to build their Linux clusters. They don't even want to think about them as clusters, and they certainly don't want to manage them one node at a time. There are a number of vendors who are trying to take the Linux clustering market up another notch, and one of them is Linux Networx, which just rolled out a new generation of Linux supercomputers at the SuperComputing 2005 trade show in Seattle.

According to Ben Pasarelli, vice president of product marketing at Linux Networx, the company is not just trying to catch a Linux wave and make money, but is in fact one of the companies that is making a wave with commercialized, enterprise-class Linux clusters and then riding that wave up. According to market research from IDC, the HPC server market grew over 30 percent in 2004, driving revenues to $7.25 billion worldwide, the highest level ever attained in this segment of the server market. Linux and X86 iron has made supercomputing more affordable, and this has expanded the market. But that doesn't mean you can just sell any old collection of Linux boxes and make demanding HPC customers happy. "The needs of supercomputing are enough different from traditional enterprise computing that HPC users want someone who will focus on compute, storage, and virtualization," explains Pasarelli.

Linux Networx was founded in 1989 and is headquartered in Bluffdale, Utah, and has received $40 million in venture capital funding from Oak Investment Partners and Tudor Ventures. It is focused solely on Linux-based supercomputing. The company has 16 machines on the current Top 500 supercomputer list, announced this week at SuperComputing 2005. To put that into perspective, that is almost as many machines as established supercomputer players Cray and Silicon Graphics have on the list, and it is just behind the number that Dell, which is basically a box vendor and kind of the antithesis of what Linux Networx is espousing, has on the list. The Linux Networx machines have 14,706 processors and an aggregate of 52.4 teraflops of computing power, which is about half of what Dell has, though. So don't count Dell out quite yet. It would be foolish to count Linux Networx out either, however. It was the first company to deliver a commercial Linux cluster in 1997, and it created its Clusterworx Linux cluster management software that year, too. In 2001, it broke into the top five of the Top 500 list with a Linux cluster, and this year, it delivered the largest Linux machine installed to date, a 5,000-processor machine that is being built at a Department of Energy lab in the States and which is not on the Top 500 list yet. The company has a total of 150 customers and over 400 supercomputers installed, and has had a 50 percent compound annual growth rate in the past five years. In its last quarter, the company set its highest revenue record, and hopes to double its business in Europe before the end of 2006 and build out its operations in the Asia/Pacific region. One in three of its sales are to new customers, which is a pretty respectable ratio for any server maker.

The new LS Series of machines, which include the LS-1 and LS/X models, are going to be the engines for growth at Linux Networx, says Pasarelli. LS is short for Linux Supersystems, and these machines are aimed at midrange and high-end HPC customers. Both machines are based on AMD's Opteron processors. The LS-1 can have from 16 to 256 single- or dual-core Opteron 200 Series processors, and can deliver 19.2 gigaflops of peak theoretical performance using the dual-core Opteron 280 per node and up to 128 nodes (with a maximum of 512 cores) can be put into a single machine. Each node can have from 1 to 8 GB of main memory per socket. Customers can use Gigabit Ethernet, Myrinet, or InfiniBand to lash the nodes together, which also have multiple PCI-X and PCI Express slots. The LS-1 is configured with Novell SUSE Linux Enterprise Server 9 on the nodes, including HPC optimizations added by Linux Networx, as well as Scali MPI for connection between the nodes and the Clusterworx 3.3 system management software. A wide variety of compilers work with the LS-1, too, and the whole thing comes preconfigured and ready to rock. An LS-1 with 64 cores and 128 GB of main memory will sell for about $130,000. The LS-1 will be available in the first quarter of next year.

The LS/X is the high-end box, and the "X" is short for "Extreme." This machine is built using the Opteron 800 Series of processors, and uses a hybrid Opteron HyperTransport-InfiniBand interconnect called, appropriately enough, Infinipath, to lash the nodes together with a lot more bandwidth and a lot lower latency. This Infinipath interconnect was created by PathScale. The LS/X 4310 model scales from six to 144 nodes and has a latency of about 1.8 microseconds, while the LS/X 4510 model scales from 24 to 1,728 nodes (that's 6,144 Opteron cores driven by a single tier of InfiniBand switches) with a latency of about 2.2 microseconds. Gigabit Ethernet is running around 40 microseconds of latency, by comparison. Each node can do about 38.4 gigaflops using the dual-core Opteron 880s, which yields about 66 teraflops of computing power in a single LS/X machine. Oddly enough, this machine runs SUSE Linux 10 with a free upgrade to SLES 10 when it comes out next year. Pricing and availability of the LS/X was not available at press time, but the machines are going into the sites of a few adopters in December.

Pasarelli says that further enhancements to the LS line of machines are due in 2006, and that one of those enhancements will be the addition of field programmable gate array (FPGA) co-processors for these machines, code-named "Jake" and "Elwood," which can significantly speed up the running of certain algorithms. Customers in the oil and gas exploration, financial analytics, and signal and image processing areas are expected to be early adopters of FPGA co-processors. These are becoming more popular as HPC shops are figuring out how to program them and make them more malleable.