ScaleMP Makes Big SMPs Out of Little Ones

Published: April 15, 2008

by Timothy Prickett Morgan

This is an idea that has been around for a long time, and one that has been implemented in various ways before: take a bunch of small, expensive servers and lash them together by their memories and make them look like a big server, but one that does not have such a big price tag. A startup called ScaleMP, which has been around for a few years in stealth mode, is just beginning to roll out the commercialized versions of its software, which glues together servers into symmetric multiprocessing (SMP) clusters using software and InfiniBand or Gigabit Ethernet interconnections.

ScaleMP was founded at the same time as Katana Systems, now known as Virtual Iron, and both companies had basically the same idea of creating a giant, virtualized SMP server out of smaller components and using fast interconnections as a backplane to link together server nodes. ScaleMP was founded by Shai Fultheim, who is the company's president and chief technology officer and who did IT development for the Israeli Defense Force as well as working as CTO for a number of Israeli startups. Shai is a contributor to the Linux kernel project as well as a number of other projects, such as Apache and Tomcat. ScaleMP's Versatile SMP software is, according to Fultheim, a derivative of work done at the IDF, and between 2003 and 2006, ScaleMP further developed the SMP clustering and virtualization technology so it would not require any rewriting of operating system or application software to run. In late 2006, although few people knew it, ScaleMP launched an embedded version of the product, now known as vSMP Foundation Embedded, which was sold by supercomputer maker VxTech, system board and OEM system maker Flextronics, and server makers Dell and Silicon Graphics. The initial vSMP Foundation Embedded product scaled up to a maximum of 32 processor cores and 256 GB of main memory in a single system image, and it required InfiniBand interconnections between the server nodes. Today, the software can scale up to 32 processor sockets, or 128 cores using Xeon or Opteron quad-core chips, and address up to 1 TB of main memory. Fultheim says that its partners have sold approximately 60 machines to date using Versatile SMP in one form or another.

Here is how Versatile SMP works. First, it does not require InfiniBand any more, and it does not make use of the Remote Distributed Memory Access (RDMA) protocol inside InfiniBand to do its magic of turning a bunch of standalone servers into an SMP cluster. As it turns out, to hide the delays in the InfiniBand or Gigabit Ethernet backplanes used to link nodes together into a single system image, ScaleMP has come up with 15 different caching algorithms; if Fultheim told me how these worked, he would have to shoot me. (I am kidding. Well, maybe not, because this is the secret sauce of the Versatile SMP architecture.) Each processor core inside each node of the vSMP cluster is loaded with a virtual machine monitor, which is loaded at boot time from a USB memory stick. (He did not say which hypervisor it uses, but he did say that it does not require any of the hardware-assisted virtualization electronics that Intel and Advanced Micro Devices have put into their server chips, such as VT or AMD-V.) The Versatile SMP software can be configured on anywhere from four to sixteen two-socket servers, and the servers can be equipped with processors with one, two, or four cores.

Starting last week, ScaleMP started selling its vSMP Foundation Standalone commercial product, which the company is using to attack the market for four-socket X64 servers. This version of the software is only intended to glue two two-socket machines together to make a four-socket box; the two USB sticks containing the Versatile SMP code and two InfiniBand cables costs $2,750. Depending on the configuration of the servers being compared, the vSMP setup of equivalent performance will cost anywhere from 30 percent to 60 percent less than an actual four-socket server. (The tough economics of four-socket and larger servers, which require sophisticated and expensive chipsets, is one of the reasons why server makers embraced multicore processors to begin with, of course.) Basically, you can create a four-socket server with a usable configuration for under $10,000.

ScaleMP is supporting Red Hat Enterprise Linux 4 and 5 and Novell SUSE Linux Enterprise Server 10 atop its Versatile SMP clustering, and according to Fultheim the company is working to support Windows Server 2008 sometime in the second half of this year; support for Windows Server 2003 will come after that.

While the virtual four-socket product is interesting, it seems a bit odd that ScaleMP isn't just launching the full product and letting customers have at it. (The odds are that this has more to do with certification of software stacks than anything else.) But when that does happen--and it will, no doubt--then the Versatile SMP product becomes very interesting for both data centers and high performance computing centers. The reason is that Versatile SMP will allow a cluster of two-socket servers to be reconfigured more or less on the fly to suit whatever workloads are necessary on the machines. Take an HPC example. Usually, before you run a simulation, you need to do some preprocessing, and this tends to work best on big SMP boxes with a shared memory space. Then, when you do the number crunching at the heart of the model, this works best on a lot of smaller computing nodes that are linked together using the Message Passing Interface (MPI) protocol. Versatile SMP allows for MPI workloads to run unchanged on a cluster, as if MPI had control of the machines. So this looks just like a normal cluster. Now, once the simulation is done, HPC centers usually have to do a big image processing job (again, running best on a big SMP box with a big shared memory space) to render the banks of numbers into something human beings can interpret. At best, this would take two or three different HPC machines today, but with Versatile SMP you can do it with one.

"You get the ease of use of a shared memory system, you get the performance of a cluster or a share memory system, depending on your workload, and you get the low price of a cluster," says Fultheim.

Sounds pretty ideal, doesn't it? Maybe they should have called it Ideal SM(H)PC? Considering how ugly that looks, it is surprising that the marketeers didn't try it. . . .

ScaleMP also announced last week that it had received $8 million in its third round of venture funding, bringing its total take so far from the venture capital community up to $26 million. Sequoia Capital, Lightspeed Venture Partners, TL Ventures, and ABS Ventures all kicked in dough.

One last note: When Virtual Iron shifted from its own hypervisor to the open source Xen hypervisor two years ago, it dropped very similar virtual SMP capability from its architecture for its Virtual Iron product. The original products created by Virtual Iron, when it was Katana Systems, could lash up to 16 servers together and then virtualize the giant SMP space any way customers wanted to. If ScaleMP takes off, it will be interesting to see if Virtual Iron dusts off its virtual SMP code and tried to merge it somehow with the Xen hypervisor. That would very likely require the open sourcing of that virtual SMP code, of course, which would then mean anyone could take a stab at commercializing this.

                     Post this story to del.icio.us
               Post this story to Digg
    Post this story to Slashdot