The National Institute for Computational Sciences

NSF’s Largest Supercomputer in Full Production Mode

Kraken passes formal acceptance, ready for science

by Gregory Scott Jones

The world’s fastest academic supercomputer is now up and running. Kraken, a Cray XT5 system located at the National Institute for Computational Sciences (NICS), managed by the University of Tennessee (UT) and funded by the National Science Foundation (NSF), promises to push the limits of simulation science.

The formal “acceptance” of the machine follows a series of rigorous tests designed to gauge its abilities and to ensure it can withstand the coming months of data-intensive simulations. Kraken (named after a mythological sea monster) met a number of criteria for numerous software codes, including HOMME (climate), NAMD (biochemistry), and LSMS (physics), demonstrating its ability to facilitate today’s most challenging high-performance computing (HPC) research projects. All of the codes utilized the full machine, an important metric for the scalability of a system the size of Kraken.

“Kraken will be a premier simulation science tool for years to come, greatly enhancing both our knowledge of the world and our ability to translate that knowledge for human benefit,” said Thomas Zacharia, vice-president of science and technology at the University of Tennessee and associate laboratory director for computing and computational sciences at Oak Ridge National Laboratory (ORNL), where NICS is housed.

A recent upgrade brought Kraken from its original 167 teraflops (trillion calculations per second) to a peak performance of more than 607 teraflops, making it the world’s fastest supercomputer managed by academia. The machine, which features 66,048 computational cores and more than 100 terabytes of memory, officially entered full production mode on February 2. Kraken will remain in full production mode until the next scheduled upgrade in late 2009, granting time to more than 100 research teams on one of the world’s premier supercomputing resources and enabling research in a vast range of areas.

For example, George Karniadakis of Brown University is using Kraken to study cranial blood flow in intracranial aneurysms and observe fundamental changes in the blood flow patterns due to anatomical variations in brain vasculature. Karniadakis will eventually use Kraken to simulate intracranial blood flow in different patients with clinically interesting brain geometry to find correlations among aneurysm geometry, mechanical stimuli from hemodynamics, and aneurysm pathology.

Through his work on Kraken, Jeremy Smith, who holds a Governor’s Chair at the University of Tennessee, is studying the natural process of breaking down grasses, husks, and other cellulose-rich plant sources into sugars for the more efficient conversion of cellulose to biofuels (overcoming the resistance of cellulose to enzymatic hydrolysis is a major obstacle in getting effective, affordable biofuels to market). This research will guide those who are engineering proteins to speed up this natural process of producing energy.

Yifeng Cui of the San Diego Supercomputer Center at the University of California–San Diego has used Kraken for the largest and most detailed earthquake simulation to date on the San Andreas Fault. This work, in collaboration with the Southern California Earthquake Center (SCEC), will improve public awareness and readiness for a future catastrophic earthquake in southern California. The mission at SCEC is to develop a comprehensive understanding of earthquakes and communicate useful knowledge for reducing earthquake risk, a mission that can be advanced with powerful new computing resources like Kraken. Results from SCEC directly influence building codes in Southern California, thus improving the safety of the region’s residents.

Following acceptance, four projects were granted early access for one week on Kraken. Three of the four scaled to the system’s total number of processors, stressing the machine to its maximum capacity and verifying its stability while achieving real scientific results for the researchers.

“We were delighted to see that all four projects with early access to the XT5 were able to adapt their codes to run across the machine in fairly short order. This is encouraging for all of our users,” said NICS Project Director Phil Andrews.

As the main computational resource for NICS, the new system is linked to the NSF-supported TeraGrid, a network of supercomputers across the country that is the world’s largest computational platform for open scientific research.

The system and the resulting NICS organization are the result of an NSF Track 2 award of $65 million to UT and its partners to provide for next-generation HPC. The award was won in an open competition among HPC institutions vying to guarantee America’s continued competitiveness through the next generation of supercomputers (systems greater than 10 teraflops and into the petascale).

As a result of the collaborative relationship between UT and ORNL, NICS promises to deliver state-of-the-art research in many fields of computational science. Together with ORNL’s Jaguar supercomputer, currently the world’s fastest for open scientific research, Kraken makes the laboratory the world’s most powerful computing complex with more than two petaflops of computing power under one roof.

For more information, visit the NICS Web site at