MIDDLEWARE

Amazon Web Services (AWS) has launched Cluster Compute Instances for Amazon EC2, which makes high-bandwidth, low-latency high performance computing (HPC) resources available in a cloud-computing environment. To ensure that the new Amazon EC2 service will be able to handle a gamut of demanding HPC applications ranging from electronic design automation to financial services, Amazon Web Services worked closely with researchers at the Lawrence Berkeley National Laboratory (Berkeley Lab).

"Many scientific research areas require high-throughput, low-latency, interconnected systems where applications can quickly communicate with each other. NERSC has extensive experience in setting up and maintaining these types of high-performance computing systems and we were happy to share this expertise in our collaboration with AWS," says Keith Jackson, a computer scientist in the Advanced Computing for Sciences Department of the Berkeley Lab's Computational Research Division (CRD).

The National Energy Research Scientific Computing Center (NERSC) at the Berkeley Lab is the primary high performance computing facility supporting unclassified scientific research sponsored by the U.S. Department of Energy. NERSC serves approximately 3,000 researchers annually in disciplines ranging from cosmology and climate to chemistry and nanoscience. To ensure that NERSC computing systems can successfully handle the wide range of scientific computing applications required by its users, the center's staff runs a series of comprehensive benchmarks on every new machine procured by the facility. Researchers in NERSC's Software and Programming Group also developed the Integrated Performance Monitoring software (IPM) to measure how well scientific applications perform on these HPC systems. To test the HPC performance of Cluster Compute Instances for Amazon EC2 the Berkeley Lab team applied these same tools to the company's new offering.

"When we applied these tests to the new Cluster Compute Instances for Amazon EC2, we found that the new offering performed 8.5 times faster than the previous Amazon instance types," adds Jackson, who led the Berkeley Lab portion of the collaboration.

Magellan, a Department of Energy project funded by the American Recovery and Reinvestment Act, is investigating whether cloud computing could meet the specialized computing needs of science. As part of this project NERSC experts are looking into the characteristics that are required to support a scientific HPC workload in a cloud computing environment, and are making these findings available to the entire community. They are also conducting comparative studies to understand how commercial clouds behave and whether they will be beneficial for science research.

In addition to Jackson, two members of the NERSC's Magellan research team contributed to this collaboration with Amazon Web Services, Lavanya Ramakrishnan and Shane Canon. Other Berkeley Lab collaborators include: NERSC's John Shalf, Harvey Wasserman and Nick Wright; Information Technology Division's Krishna Muriki and Qin Yong; and Greg Bell, who is currently with the Energy Sciences Network.

For more information about our work with Amazon Web Services, please visit:
http://aws.amazon.com/hpc-applications
For more information about Computing Sciences at Berkeley Lab, please visit: www.lbl.gov/cs