The Windows Observer--Google, IBM Partner on Utility Computing Cloud

Newsletters	Subscriptions	Forums	Safari	Store	Career	Media Kit	About Us	Contact	Search	Home

Volume 4, Number 38 -- October 10, 2007

Google, IBM Partner on Utility Computing Cloud

Published: October 10, 2007

by Timothy Prickett Morgan

Search engine and Web advertising goliath Google and server maker and supercomputing giant IBM announced today that they are working with some prominent American universities to set up a so-called cloud computing grid to assist those universities in teaching and developing utility-style applications that may represent the future of corporate computing some day.

By virtue of its growing number of Internet-based applications, Google perhaps knows more about utility computing (meaning systems are utilized on the fly as workloads change) or cloud computing (a much vaguer term that seems to imply utility-style infrastructure as well as parallelized applications that can scale with the size of the computing resources dedicated to them). In theory, cloud computing could encompass many different kinds of utilities, with varying processing and bandwidth characteristics, and a workload would make use of the computing resources available in the cloud (another way to refer to a network of machines connected by Internet-style networking) as the workload progressed. For instance, a Linux-X64 cluster might be used to churn through large data sets, but a vector parallel computer might be used to run simulations based on a subset of the data and then a separate virtualization system might be used to show simulations in 3D over time. Similarly, large-scale parallel systems can be used to search volumes of information (as Google does well), perhaps looking for spam lurking in wiki and forum sites or stolen credit card data involved in business transactions.

To help students get a handle on the underlying hardware and software technologies involved in cloud computing, IBM's Software Group, which peddles a solution called the High Performance On Demand (HiPOD) cloud computing configuration, is the lead integrator on a cloud utility that is being installed at Google's data center in Mountain View, California, IBM's Almaden Research Center in San Jose, California, and at the University of Washington in Seattle. The initial machinery in the cloud has several hundred virtual servers (the exact physical number has not been announced), and eventually 1,600 of IBM's BladeCenter blade servers and System x rack servers will be installed across these three sites. The machinery will run Red Hat's Fedora 7 development release of Linux, and will be automatically provisioned by IBM's Tivoli Provisioning Manager. The cloud will use the integrated Xen hypervisor in Fedora to virtualize the physical servers, and IBM's DB2 database and WebSphere runs on the systems as well. So does Apache Hadoop, which is an open source project that contains two key Google technologies--the Google File System, the file system that Google created to run its search engines, and MapReduce, a means of taking data that is mapped across a large number of systems and reducing it back to the original document. (This is one of many different ways to implement a parallel file system.)

In addition to the University of Washington, Carnegie Mellon University, the Massachusetts Institute of Technology, Stanford University, the University of Maryland, and the University of California at Berkeley are all making use of the cloud cluster set up by Google and IBM. Google and the University of Washington have also developed a cloud computing curriculum for universities to use to teach modern programming techniques on parallel architectures, and this coursework has been open sourced under a Create Commons license. IBM has also created an Eclipse plug in for Hadoop so programmers can more easily create applications that run on Hadoop.

IBM and Google estimate that they will split the cost of setting up and running the cloud, which will cost between $40 million and $50 million. Of course, they could have saved a lot of effort and just bought capacity on the Sun Grid utility from Sun Microsystems. But universities and vendors like to control their capacity, even if they are sharing it.