The paper, "The Dynamics of Plate Tectonics and Mantle Flow: From Local to Global Scales," describes the whole-earth model and its underlying algorithms and was published in the Aug. 27 issue of Science and featured on the cover. The work demonstrates the scientific advances that can occur when multidisciplinary teams work together to capitalize on advanced mathematical algorithms and supercomputers to tackle frontier science problems.

To create the model, computational scientists at Texas's Institute for Computational Engineering and Sciences (ICES) extended the use of a computational technique known as Adaptive Mesh Refinement (AMR). The team included Omar Ghattas, the John A. and Katherine G. Jackson Chair in Computational Geosciences and Professor of Geological Sciences and Mechanical Engineering, and research associates Georg Stadler and Carsten Burstedde. Michael Gurnis, the John E. and Hazel S. Smits Professor of Geophysics and director of the Caltech Seismological Laboratory, represented Caltech.

"Many physical systems in science and engineering, including global mantle flow, exhibit behavior on a wide range of length scales," says Ghattas. "Modeling and simulation of such systems has always presented notorious challenges, particularly on the parallel supercomputers that are necessary to capture their multiscale behavior."

Mantle flow is described bypartial differential equations, which are solved by subdividing the mantle into a computational grid. Ordinarily, the resolution is kept the same throughout the grid. However, mantle flow models must finely resolve plate boundaries while also representing the entire mantle. The only way to capture this wide range of scales is to use AMR methods.

"AMR adaptively creates finer resolution only where it's needed," explains Ghattas. "This leads to enormous reductions in the number of grid points, making possible simulations that were previously out of reach. The complexity of managing adaptivity among many thousands of processors, however, has meant that current AMR algorithms have not scaled well on modern petascale supercomputers."

To overcome this long-standing problem, the group developed new parallel algorithms that, Burstedde says, "allows for adaptivity in a way that has scaled to the hundreds of thousands of processor cores of the largest supercomputers available today."

The group's work on parallel AMR algorithms has been selected as a finalist for the prestigious 2010 ACM/IEEE Gordon Bell Prize in Supercomputing.

The new AMR algorithms, combined with new scalable solvers, allowed the scientists to simulate global mantle flow and its manifestation as plate tectonics, including the motion of individual faults.

According to Stadler, "The AMR algorithms reduced the size of the simulations by a factor of 5,000, permitting them to fit on Ranger," the 62,976 core supercomputer at the National Science Foundation (NSF)-supported Texas Advanced Computing Center. "Additionally, TACC's visualization resources were critical to model building and analysis of the results," says Stadler.

TACC Director Jay Boisseau, said, "Omar and his colleagues are conducting research that is reshaping our understanding of our world--literally. We are proud that our powerful supercomputing and visualization technologies are enabling them in their cutting-edge research."

A key to the model was the incorporation of data on a multitude of scales. For example, at the largest scale--that of the whole Earth--the movement of the surface tectonic plates is a manifestation of a giant heat engine, driven by the convection of the mantle below. The boundaries between the plates, however, are composed of many hundreds to thousands of individual faults, which together constitute active fault zones.

"The individual fault zones play a critical role in how the whole planet works," says Gurnis. "And if you can't simulate the fault zones, you can't simulate plate movement--and, in turn, you can't simulate the dynamics of the whole planet."

In the model, the researchers resolved the largest fault zones, creating a mesh with a resolution of about one kilometer near the plate boundaries. Included in the simulation were seismological data as well as data pertaining to the temperature of the rocks, their density and their viscosity--or how strong or weak the rocks are, which affects how easily they deform. That deformation is nonlinear--with simple changes producing unexpected and complex effects.

More than a hundred simulations were required, each one running overnight and sustaining more than five trillion operations per second. The result was an estimate of the motion of both large tectonic plates and smaller microplates, including their speed and direction, which were remarkably close to observed plate movements.

The investigators also discovered that anomalous rapid motion of microplates emerged from the global simulations.

"In the western Pacific," Gurnis says, "we have some of the most rapid tectonic motions seen anywhere on Earth, in a process called 'trench rollback.' For the first time, we found that these small-scale tectonic motions emerged from the global models, opening a new frontier in geophysics."

Another surprising result from the model relates to the energy released from plates in earthquake zones.

"It had been thought that the majority of energy associated with plate tectonics is released when plates bend, but it turns out that's much less important than previously thought," Gurnis says. "Instead, we found that much of the energy dissipation occurs in the earth's deep interior. We never saw this when we looked at smaller regions."

The paper was authored by Omar Ghattas, Georg Stadler, Carsten Burstedde, and Lucas C. Wilcox of The University of Texas at Austin, and Michael Gurnis and Laura Alisic of Caltech. The work was supported by the NSF, the Department of Energy's Office of Science and--at the Caltech Tectonics Observatory--by the Gordon and Betty Moore Foundation.