Oil and Gas Industry Prefers Personal HPC Capacity, Says Microsoft

Published: March 14, 2007

by Timothy Prickett Morgan

When Microsoft enters a sector of the IT industry, it doesn't just launch a product. Long before a product comes out, it instigates itself into that industry by gaining expertise and developing high-level contacts. So it is with Microsoft, its parallel supercomputing variant of the Windows operating system, and the oil and gas industry.

Last week, Microsoft hosted the fourth annual Global Energy Forum in Houston, Texas, which has been the epicenter of the oil and gas industry for a long time. But considering that the United States reached peak oil production in the 1970s and that industry exploration giant Halliburton this week announced that it is moving its headquarters to Dubai, the capital city of the United Arab Emirates, maybe next year, Microsoft will host the forum in Dubai. Anyway, this year, the event was hosted in Houston and John Hofmeister, president of Shell Oil Company, and James Baker, former U.S. secretary of state and a man well-hooked into both politics and oil, gave the keynote addresses.

But the most interesting part of the day, perhaps, was a report that Microsoft commissioned by Gelb Consulting Group that indicated that companies in the oil and gas industry believed that their ability to discover oil and gas reserves and thereby help their companies increase production was being hampered by limitations on access to high performance computing resources.

This, of course, plays exactly into Microsoft's view of the HPC server market and its desire to bring supercomputing out of the data center and directly into the hands of scientists and other researchers who would almost certainly rather have a smaller cluster of machines to themselves all the time than to be forced to share a big cluster that can do the work a lot faster.

While Windows Compute Cluster Server 2003, which is a streamlined variant of Windows with the Message Passing Interface (MPI) protocol commonly used in parallel supercomputer clusters to allow them to share work embedded it in, can be used to build machines every bit as large as those that use Unix or Linux operating systems, thus far, Microsoft only had a brief moment on the Top 500 supercomputer list before the machine was rebooted as a Linux cluster last fall. When Linux dominates the parallel machines big academic and government research centers, there is not much of a point trying to unseat Linux. But surrounding HPC centers with researchers who deploy baby HPC clusters running Windows is a brilliant strategy--one that worked to move Windows from the PC to the print and file server and eventually into the data center. And the fact that researchers might be more familiar with Windows than Unix or Linux sure helps Microsoft's cause.

If the information presented in the High-Performance Computing Oil and Gas Survey 2007 is correct, then Microsoft is not just tilting at stone walls with the Windows HPC variant. People want their own machines--or at least something that feels more like it.

Gelb surveyed 104 oil and gas industry experts from around the world, and 81 percent of them reported that if they had more ready access to HPC processing capacity, they could increase oil and gas production. Some 86 percent of those surveyed have a respectable amount of computing power on their desktops already, and 69 percent said that if they had their choice, they would prefer to have number-crunching power near their desks, not back in the data center.

This, of course, cuts against the whole trend toward grid computing and shared resources--but, then again, most of us have a PC, not a thin client, too.

According to the survey, 61 percent of the oil and gas industry experts polled said that they can better manage the risks on a project by conducting multiple iterations on a simulation over shorter bursts of time than by having a giant simulation run on a larger, central supercomputer cluster. (About half of those polled said that their applications were not parallel anyway.) Interestingly, 75 percent of those polled said that they had ready access to the computing power they needed to do their jobs. Which means the issue is not flops, but when and how you get to them.

Not surprisingly, 56 percent of those polled said that they wanted to be able to run their jobs when they wanted, not when a cluster administrator works them into the queue. Because of limited access to HPC clusters, researchers have to run bigger simulations that fill up their allotted time, and then hope the results pan out after a simulation is done in a matter of several hours to several days. They would much rather do smaller simulations, and tweak the simulations as interim results are returned on a smaller machine.

While this seems to argue for local HPC processing rather than a centralized approach, what seems to be the case is that HPC system administrators are not using a fine enough granularity to schedule jobs. With the appropriate job scheduling software and provisioning tools to get the right operating system, database, and application software on the right subset of a cluster of servers, plus some supercomputing on demand capacity from IBM, Sun Microsystems, or Hewlett-Packard on tap to cope with peaks, the issues that Microsoft is correctly addressing with its Windows Compute Cluster Server offering could be fixed without having computers scattered all over hither and yon, as in days gone by.

It will be interesting to see what happens. With a discretionary budget of several tens of thousands of dollars, researchers can build a baby HPC cluster. And they can probably add a factor of 1.5 times more performance each year to their deskside clusters every year for the same money. And do it all under the radar of the bean counters.

If you want to read the full report, you can do so here.

RELATED STORIES

Windows Compute Cluster Server 2003 Goes GA

Windows Compute Cluster Server is Ready, Microsoft Says

Microsoft Ships First Release Candidate for Windows Compute Cluster Server 2003

HPC Version of Windows Server Goes to Public Beta

Gates Lays Out Vision of Future of Supercomputing

Microsoft Confirms Windows Server HPC Edition Due in 2005

Windows HPC Edition Is in the Works

                     Post this story to del.icio.us
               Post this story to Digg
    Post this story to Slashdot