The National Institute for Computational Sciences

The Retirement of Kraken

Remembering the Accomplishments of the First Academic Computer to Break the Petaflop Barrier

Decommissioned as of April 30, 2014, the Kraken supercomputer leaves a legacy as the "go-to" flagship resource in the National Science Foundation's (NSF) TeraGrid and later, the eXtreme Science and Engineering Discovery Environment (XSEDE), for empowering and preparing the U.S. academic research community for sustained petascale science and engineering, and breakthroughs that have expanded the boundaries of human understanding.

Ten Especially Notable Facts About Kraken

From April 2008 until its decommissioning, Kraken, managed by the National Institute for Computational Sciences (NICS) and housed at Oak Ridge National Laboratory (ORNL), delivered more than 4 billion core-hours of computing and maintained an average uptime (availability) of 96 percent.

The U.S. was world leader in computer simulations from the expansion of the NSF's open-science capability in the mid-1980s until the early 2000s when Japan came forth with a simulator that was an order of magnitude greater than anything the U.S. had installed at the time. However, NSF's Kraken, in combination with the Department of Energy's former Jaguar supercomputer, helped restore the global preeminence of the U.S. in computer simulations. Kraken, together with Jaguar, then the world's fastest computer for open scientific research, made ORNL the most powerful computing complex on the planet, with more than two petaflops of power under one roof.

For a period of time, Kraken provided more than 60 percent of the allocated compute cycles in the TeraGrid portfolio of approximately a dozen resource providers. Even to retirement, Kraken has remained one of NSF's most highly used systems—from August 2008 through March 2014, it supplied an average of 43 percent of all allocated compute cycles for TeraGrid/XSEDE. Moreover, users have consistently been able to run across the entire machine with high efficiency.

Before the deployment of the Blue Waters supercomputer at the University of Illinois, Kraken fulfilled the role of capability computing resource. Capability users are those who effectively use a system up to its limits.

NICS, Kraken's managing organization, instituted an innovation to enable the optimal performance of its operational mission of providing the maximum number of total compute cycles to the scientific community while enabling full machine runs for capability users. The innovation, called bimodal scheduling, entails a forced "draining" of the system on a weekly basis, followed by consecutive full machine runs. Implementation of bimodal scheduling led to utilization of more than 90 percent—the equivalent of a 300-plus teraflop supercomputer, or several million dollars of compute time a year. Average utilization during the course of Kraken's life has been an exceptional 92 percent. Bimodal scheduling was the brainchild of the late Phil Andrews, the first director of NICS.

Kraken is a series of Cray XT systems that culminated in a 112,896-core XT5 system with a peak performance of 1.17 petaflops (1,174 teraflops), 147 terabytes of memory, and 2.4 petabytes of dedicated formatted disk space. Each evolutionary milestone was delivered on scope, schedule, and budget.

Kraken entered full production mode on Feb. 2, 2009, with a speed of 607 teraflops, or 607 trillion calculations per second. In the latter part of that year, Kraken became only the fourth open supercomputer ever to perform a petaflop, or 1,000 trillion calculations per second.

Kraken held the distinction of being the world's most powerful computer managed by academia and was the third fastest on the Top500 list in November 2009. As of November 2013, it was still ranked number 35.

The Kraken supercomputer could not have become a reality without the existence of the Joint Institute for Computational Sciences (JICS), which was established by the University of Tennessee (UT) and ORNL in 1991 to encourage and facilitate the use of high-performance computing in the state of Tennessee. The synergy of UT and ORNL's Oak Ridge Leadership Computing Facility fostered the winning of the $65 million award from NSF in August 2007 that funded the creation of Kraken and NICS.

The first academic computer to break the petaflop barrier of more than a quadrillion floating-point operations per second, Kraken enabled researchers in myriad scientific and engineering domains—from physics to molecular biology, atmospheric sciences, climate, and many others—to achieve advances that prior academic computing resources lacked the power to support.

Fun Fact: A floating-point operation is equivalent to multiplying, adding, subtracting, or dividing two 15-digit numbers. Assuming that every man, woman, and child could do one of these operations per second as his or her full-time job, it would take the entire population of Tennessee 25 years to do the same amount of work that Kraken can do in one second.

For more information on Kraken's History and Infrastructure, click here.

For a broad, by-the-numbers picture of the Kraken supercomputer's contribution to open scientific research, view the infographic.

Article posting date: 25 April 2014

About JICS and NICS: The Joint Institute for Computational Sciences (JICS) was established by the University of Tennessee and Oak Ridge National Laboratory (ORNL) to advance scientific discovery and state-of-the-art engineering, and to further knowledge of computational modeling and simulation. JICS realizes its vision by taking full advantage of petascale-and-beyond computers housed at ORNL and by educating a new generation of scientists and engineers well versed in the application of computational modeling and simulation for solving the most challenging scientific and engineering problems. JICS runs the National Institute for Computational Sciences (NICS), which had the distinction of deploying and managing the Kraken supercomputer. NICS is a leading academic supercomputing center and a major partner in the National Science Foundation's eXtreme Science and Engineering Discovery Environment, known as XSEDE. In November 2012, JICS sited the Beacon system, which set a record for power efficiency and captured the number one position on the Green500 list of the most energy-efficient computers.