The U.S. Department of Energy’s (DOE’s) Leadership Computing Facilities (LCFs) have awarded a combined 4.7 billion supercomputing core hours to 61 science and engineering projects with high potential for accelerating discovery and innovation through its Innovative and Novel Computational Impact on Theory and Experiment (INCITE) program. The allocation of resources at Argonne and Oak Ridge national laboratories, enabled by the deployment of next-generation high-performance computers, will provide researchers in industry, academia, and government access to some of the world’s most powerful supercomputers to address grand challenges from advancing sustainable energy to understanding environmental consequences of energy use.

“Supercomputing is essential to solving our greatest scientific and technological challenges and improving our economic prosperity, energy security, and global competitiveness,” said James Hack, director of the National Center for Computational Sciences (NCCS), which houses the Oak Ridge Leadership Computing Facility (OLCF) in Oak Ridge, Tenn. “Simulations that exploit massive parallelism play a critical role in building our future. Today’s awards will speed insights into the natural world from subatomic particles to earthquakes to supernovae, and the engineered world from cars and concrete to catalysts and computer chips.”

Added Michael Papka, director of the Argonne Leadership Computing Facility (ALCF) just outside Chicago, “The 2013 INCITE awards will accelerate breakthroughs by allowing greater complexity and realism in simulations, from carbon sequestration underground to turbulent combustion in power and propulsion devices.”

The ALCF’s newest leadership computing resource is Mira, a 10-petaflop IBM Blue Gene/Q system with 49,152 compute nodes and a power-efficient architecture. The ALCF also houses Intrepid, a 557-teraflop IBM Blue Gene/P. The OLCF is home to Titan, a new 20-petaflop Cray XK7 hybrid system employing both central processing units and energy-efficient, high-performance graphics processing units in its 18,688 compute nodes. For 2013, the INCITE program, which is jointly managed by DOE’s LCFs, awarded 2.83 billion hours at the ALCF and 1.84 billion core hours at the OLCF on systems capable of carrying out quadrillions of calculations each second (petaflops).

When INCITE made its first awards in 2004, three projects received an aggregate 5 million hours on DOE supercomputers. Today’s collective allocation of 4.7 billion hours represents an almost 1,000-fold growth in resources provided to researchers. To date, INCITE has delivered more than 10 billion computing hours to the scientific community.

This announcement coincides with Oak Ridge National Laboratory's unveiling of Titan, a system capable of churning through more than 20,000 trillion calculations each second—or 20 petaflops—by employing a family of processors, called graphics processing units (GPUs), first created for computer gaming. Titan will be 10 times more powerful than ORNL’s last world-leading system, Jaguar, while overcoming power and space limitations inherent in the previous generation of high-performance computers.

Titan, which is supported by the Department of Energy, will provide unprecedented computing power for research in energy, climate change, efficient engines, materials, and other disciplines and will pave the way for a wide range of achievements in science and technology.

The Cray XK7 system contains 18,688 nodes, with each holding a 16-core AMD Opteron 6274 processor and an NVIDIA Tesla K20 GPU accelerator. Titan also has more than 700 terabytes of memory. The combination of central processing units (CPUs), the traditional foundation of high-performance computers, and more recent GPUs will allow Titan to occupy the same space as its Jaguar predecessor while using only marginally more electricity.

“One challenge in supercomputers today is power consumption,” said Jeff Nichols, associate laboratory director for computing and computational sciences. “Combining GPUs and CPUs in a single system requires less power than CPUs alone and is a responsible move toward lowering our carbon footprint. Titan will provide unprecedented computing power for research in energy, climate change, materials, and other disciplines to enable scientific leadership.”

Because they handle hundreds of calculations simultaneously, GPUs can go through many more than CPUs in a given time. By relying on its 299,008 CPU cores to guide simulations and allowing its new NVIDIA GPUs to do the heavy lifting, Titan will enable researchers to run scientific calculations with greater speed and accuracy.

“Titan will allow scientists to simulate physical systems more realistically and in far greater detail,” said James Hack, director of ORNL’s National Center for Computational Sciences. “The improvements in simulation fidelity will accelerate progress in a wide range of research areas such as alternative energy and energy efficiency, the identification and development of novel and useful materials, and the opportunity for more advanced climate projections.”

Combustion

The S3D application models the underlying turbulent combustion of fuels in an internal combustion engine. This line of research is critical to the American energy economy, given that three-quarters of the fossil fuel used in the United States goes to powering cars and trucks, which produce one-quarter of the country’s greenhouse gases.

Titan will allow researchers to model large-molecule hydrocarbon fuels such as the gasoline surrogate isooctane; commercially important oxygenated alcohols such as ethanol and butanol; and biofuel surrogates that blend methyl butanoate, methyl decanoate, and n-heptane.

Nuclear Energy

Nuclear researchers use the Denovo application to, among other things, model the behavior of neutrons in a nuclear power reactor. America’s aging nuclear power plants provide about a fifth of the country’s electricity, and Denovo will help them extend their operating lives while ensuring safety. Titan will allow Denovo to simulate a fuel rod through one round of use in a reactor core in 13 hours; this job took 60 hours on the Jaguar system.

Climate Change

The Community Atmosphere Model–Spectral Element simulates long-term global climate. Improved atmospheric modeling under Titan will help researchers better understand future air quality as well as the effect of particles suspended in the air.