U.S. Energy Department Gives Away 95 Million CPU-Hours on Supers

Published: January 9, 2007

by Timothy Prickett Morgan

The U.S. Department of Energy is not only one of the largest consumers of computing cycles in the world with its array of government-sponsored supercomputing centers and one of the key drivers in the design of future supercomputing systems. The DOE is also a beneficent organization, and since 2004, it has been donating spare supercomputing cycles to good causes in academic, public, and industrial organizations. Today, the DOE announced the largest grants so far, with the INCITE program giving away 95 million CPU-hours to 45 different projects.

Because it is the government, INCITE is actually an acronym, short for Innovative and Novel Computational Impact on Theory and Experiment. And because supercomputing has always been affected by politics as well as a desire to do science, size matters. So it is no surprise that Sandia National Laboratories, Oak Ridge National Laboratory, Lawrence Livermore National Laboratory, Lawrence Berkeley National Laboratory, Los Alamos National Laboratory, Pacific Northwest National Laboratory are the biggest consumers of computing cycles among the 21 labs funded by the DOE, and in the world. Trying to manage the maintenance and continued functioning of existing nuclear weapons and coming up with simulations to help design new generations of weapons that cannot be tested in the real world are the main drivers of the installation of supercomputers at the DOE centers. And while this is a big job, the advent of teraflops-scale computing and the impending delivery of petaflops-class supercomputers means that when these labs have a little spare capacity, it ends up being a lot of teraflops.

So in 2003, the DOE started the INCITE program to give those gigaflops and teraflops away based on a peer-reviewed, scientific equivalent of American Idol. In 2004, three projects were given 5 million CPU-hours to play with on various computers at Lawrence Berkeley's National Energy Research Scientific Computing (NERSC) center. It was about 10 percent of the capacity of the supers at that site. Each year, the number of DOE super centers contributing to the INCITE program has increased. In 2005, 6.5 million CPU-hours were awarded, for three new projects. In 2006, 18.2 million CPU-hours were awarded to 15 projects, and NERSC, Lawrence Berkeley, Argonne, Oak Ridge, and a smaller lab in Richland, Washington were participating. This year, the Pacific Northwest lab is giving away cycles along with the others. (Sandia and Lawrence Livermore are heavily involved with a cooperative nuclear missile design program and are not giving away any cycles.)

According to Raymond Orbach, the DOE's undersecretary of science, the U.S. government believes that any excess computing capacity the labs have left over should be used for big scientific problems that will give the United States a competitive advantage in science, which in turn fuels industry. And since the DOE is rapidly expanding its computing capacity, the odds favor universities and private companies being able to get their hands on very large blocks of computing capacity--if they can run the gauntlet and get picked.

"Back in 2003, when we awarded 5 million CPU-hours and peak supercomputer speeds were an order of magnitude lower than what we have today, when people talked about petaflops computing, I thought that this was just poppycock," explained Orbach. "But if the budge holds, we will hit a petaflops in 2008, and this will give us leadership over the rest of the world."

Boeing and Proctor and Gamble gave presentations at the announcement explaining how they would do fundamental research that would improve the designs of their products, now that they have been given CPU-hour awards. Boeing, DreamWorks Animation, Pratt and Whitney, and General Atomics have gained free CPU-hours in the past, and they got renewed proposals. Corning, Fluent, and Proctor and Gamble were new industrial companies given awards, bringing the total count of companies getting CPU-hours to 11. The remaining 34 who got free cycles were in academia or at government research centers.

A total of 185 million CPU-hours of time were requested in all of the proposals submitted to DOE; only 95 million were awarded, and according to DOE representatives, those who received awards were given the computing time because they could demonstrate that their simulations were "computationally ready," meaning the code is ready to go on an existing super, and that the engineering or basic problem that they were trying to solve could be solved with the requested amount of computer time. Organizations can carve up their CPU-hours in ways that make sense for their particular simulation, either running it on a large number of CPUs for a certain number of hours, or running it on fewer CPUs for more time.

When pressed, Orbach did not answer a direct question concerning the value of the supercomputing time awards, saying that he preferred to think of the awards as the value to users. If you reckon that it is a $1 per CPU-hour, like Sun Microsystems is talking on its Sun Grid compute utility at list price for a contract with no long-term commitment, that works out to $95 million. As the IT director from Proctor and Gamble put it when discussing the computing power he needed to solve some scientific problems concerning soap and oil and turbulent water, "it would take all of the computing capacity at Proctor and Gamble months to get at this problem."

Orbach said that any results that are derived from the 45 projects are apparently going to be available to the scientific community. So it is hard to imagine how much of an edge the free computing capacity will give industrial companies that are participating. The important thing is that the excess cycles are not just going to waste, and considering the dozens of megawatts of power that these supercomputing centers burn up, that would be even more of a crime than just letting those cycles go idle.

                     Post this story to del.icio.us
               Post this story to Digg
    Post this story to Slashdot