The National Institute for Computational Sciences

The Evolution of Kraken

The Inception of NICS

In 2005 the University of Tennessee, Knoxville (UT Knoxville), and the Joint Institute for Computational Sciences (JICS)—a partnership between UT Knoxville and Oak Ridge National Laboratory—responded to a call from the National Science Foundation (NSF) (solicitation NSF 05-626) to deploy and support a world-class high-performance-computing (HPC) environment of unprecedented capacity and capability.

In its response, UT Knoxville/JICS proposed to deploy a 1-petaflop, peak-performance Cray XT-5 system that would enable JICS to fulfill its objective of empowering the U.S. academic research community to pursue leading-edge science and engineering projects using petascale-level computational resources.

Success came in August 2007 when NSF announced that the National Science Board had approved an award of $65 million for five years to fund the deployment and operation of an extremely powerful supercomputer at UT Knoxville/JICS. With that award, UT Knoxville created the National Institute for Computational Sciences (NICS) project within JICS.

Kraken: Workhorse for Innovative Research

The newly established NICS acquired a Cray XT3 system, named it Kraken and made it available to the research community, along with technical support from an expert staff and the computational resources of NSF’s TeraGrid project. With a configuration nearly four times the capacity of the entire TeraGrid infrastructure at the time, Kraken provided a large percentage of compute resources for TeraGrid, which ended in 2011.

Today the usability and scalability of Kraken continue to make it an especially noteworthy workhorse for TeraGrid’s successor, the XSEDE (Extreme Science and Engineering Discovery Environment) project. Indeed, as it has throughout its history, Kraken successfully and reliably enables the pursuit of innovative research in a variety of disciplines. Researchers from more than 550 educational and research institutions have used time on the machine, as allocated by the TeraGrid/XSEDE Resource Allocation Committee.

As of December 2012, Kraken had delivered more than 3 billion compute hours to scientists in more than 2.5 million jobs, with over 95-percent uptime and 81-percent utilization. And, remarkably, the use of Kraken exceeded 91 percent during the period 2009–2012, a rare feat for any supercomputer.

Kraken’s computing power has garnered significant recognition in the broader HPC world as well. Besides having once held the distinction of being the world’s most powerful academic supercomputer, on the November 2009 Top500 list Kraken ranked as the third-fastest computer in the world. As of January 2013, the system was still among the power elite on the Top500 list, standing at number 25.

Kraken’s Evolution

The Kraken system has evolved through a series of deployments—from a Cray XT3 at project inception to a Cray XT4 in mid-2008, and then to a Cray XT5 in 2009. The XT5 deployment progressed in late 2009 through a planned upgrade from 4-core processors to 6-core processors. In early 2011 an unplanned but essential memory upgrade was successfully proposed and added to the 6-core-processor upgrade, and an additional 12 cabinets were added to the original 88. All of these deployments and upgrades were executed on scope, within budget and on time.

The 6-core-processor and memory upgrade of Kraken was completed nearly one month ahead of schedule, even with the addition of an unplanned component. And during the 12-cabinet addition to Kraken, more than half of the existing resource remained available to users even though the upgrade required the re-plumbing of nearly half of the existing system, relocation of existing equipment and the addition of new cooling equipment.

Based on NSF’s approval of two years of extended service, the Kraken computer continued providing its computing power to the research community until it was decommissioned on April 30, 2014.

Read more