Kraken will be decommissioned on April 30, 2014. For more information see Kraken Decommission FAQs
Kraken will be decommissioned on April 30, 2014. For more information see Kraken Decommission FAQs
The National Institute for Computational Sciences

Galactic Power Packs

Black Holes Impact Galaxies, Serve as Laboratories for Research

By Scott Gibson


Dense and compact collections of matter with gravitational fields so strong that not even light can escape, black holes are compelling scientific curiosities chock full of information. Indeed, the black hole, which NASA likens to having 10 times the mass of the Sun packed into a sphere about the size of New York City, could unveil revelations about the early universe and either confirm or disprove Einstein’s general relativity theory.

Scientists have known for decades that black holes — formed from collapsed stars — exist in the nuclei of most galaxies, and astrophysicists such as Roger Blandford, director of the Kavli Institute for Particle Astrophysics and Cosmology (KIPAC) at Stanford University, believe research will continue to find answers to obstinate questions about black holes.

More Power Than 1,000 Galaxies

One problem presenting a major challenge to astrophysicists is how a black hole, smaller than the solar system in the middle of a galaxy, can produce more power than 1,000 of those galaxies. Besides being so mighty, black holes have a major impact on the formation, growth and evolution of galaxies, which adds to their intrigue.

Researchers must rely on calculations and simulations in their investigations, coupled with observations of the entities that surround black holes — namely accretion disks and relativistic jets. Accretion disks are collections of interstellar material (usually gas); and relativistic jets are streams of high-temperature plasma and electromagnetic field that propagate outside the galaxies and produce double radio sources.

From Pencil and Paper to Supercomputer Simulations

A scientist with a rich perspective on black-hole research, Blandford says he has seen the method of solving black-hole problems evolve from analytic calculations using pencil and paper to today’s application of supercomputers.

While great, inspired feats of applied mathematics — such as the discovery of the Kerr metric, which solves Einstein’s field equations of general relativity — were accomplished using analytic calculations, computations have enabled studies that could never have been completed analytically, Blandford says.

Computer simulations have, in fact, offered insights in controversies. At issue, for example, was whether the energy in a black hole came from gas swirling around it or from the black hole itself. Simulations substantiated that the energy emanates from the black hole. In addition, by combining observations with simulations, researchers affirmed the remarkable robustness of the relativistic jets of black holes. The jets, they discovered, could efficiently extract energy from the spin of the black hole and propagate and maintain their integrity when confronted with difficult circumstances.

Seeing Complicated Flows Clearly

Blandford credits the endeavors of his research colleagues — Jonathan McKinney, an assistant professor of physics at the University of Maryland at College Park; and Alexander Tchekhovskoy, a post-doctoral fellow at the Princeton Center for Theoretical Science — as having made impressive technical advances in simulations possible. Getting there, he explains, required refining computer codes, learning from mistakes, bolstering the stability of the computing runs and improving efficiency.

The work of McKinney and Tchekhovskoy combined with the availability of high-performance computing (HPC) has produced what Blandford describes as a highly efficient graphics engine that reveals in 3D what’s going on in very complicated astrophysical flows and allows the researchers to present the results to a wider audience.

A spinning black hole (at center) produces a powerful jet (white-blue) along its spin axis. While near the hole, the disk rotational axis and jet direction are aligned with the black hole spin axis. Farther away the jet deviates and eventually points along the outer disk’s rotational axis. A black hole movie simulation can be viewed here.

[Courtesy: Simulation — Jonathan McKinney (UMD), Alexander Tchekhovskoy (Princeton), Roger Blandford (KIPAC), Visualization — Ralf Kaehler (Stanford Linear Accelerator Center)]

McKinney and his research team colleagues convey in recent a Science paper how, through the use of simulations, they discovered that the behavior of black holes that have thick accretion disks differs from longstanding assumptions. The belief has been that accretion disks lie flat along the outer edges of black holes while the relativistic jets shoot out perpendicularly to the disks. However, the simulations showed that the configuration becomes more complex at large distances from the black hole spin axis, with the jets becoming parallel to, but offset from, the accretion disk's rotational axis; in the process, the disk warps and the jet bends, influencing what one sees at different viewing angles.

McKinney explained that key in making this discovery was being able to reduce the symmetry of the problem in their numerical code. To do that, the researchers used spherical polar coordinates that employ radius and two different angles to describe the coordinates. As a result of their approach, they were able to capture the black hole’s asymmetrical shape.

The project simulations also dispelled the belief that the accretion disk controlled the relativistic jet — the interaction was shown to be just the opposite.

The magnetic field threading the black hole impacts jet properties. Although other research projects treated the magnetic field as a free parameter in their calculations, this project attributed it with maximum importance and equated its pull on the disk with that of the black hole’s gravity, Tchekhovskoy explains.

XSEDE Resources Play Key Role in Research

McKinney says XSEDE (eXtreme Science and Engineering Discovery Environment) resources provided the highly parallel computing capability and the compute time the research team needed to create several models and run 12 simulations on the Kraken supercomputer, which is managed for the National Science Foundation (NSF) by the University of Tennessee’s National Institute for Computational Sciences (NICS). The researchers also used HPC resources from the Texas Advanced Computing Center (Lonestar and Ranch supercomputers) as well as NASA Advanced Supercomputing (Pleiades supercomputer).

“We were tilting the black hole by a little more each time,” McKinney explains. Because of that he and his colleagues ran several simulations that used roughly 8,000 processor cores. Parallel computing combined with their use of the spherical polar coordinates was key in getting accurate results, he says.

The researchers used the Nautilus supercomputer (also managed for NSF by NICS) to do most of the analyses and visualizations for the project.

McKinney says that throughout the investigation, visualization has been essential because numbers or other elements alone would not have been enough to make discoveries. And he notes that the subject matter’s lack of symmetry posed a particular challenge to achieving proper rendering and visualization.

Next Step: Testing General Relativity

Simulations can be done using the equations of general relativity theory, which describes gravity as a geometric property of space and time. Black holes, because of their strong gravity, represent a useful laboratory for testing general relativity because no one knows if the theory remains true in the presence of high gravity.

Determining whether Einstein was right will involve the use of observations and simulations. A September 2012 paper in Science by Sheperd Doeleman of MIT reported the first images of the jet-launching structure near the supermassive black hole, M87, at the center of a neighboring galaxy.

The images, captured using the Event Horizon Telescope, an array of four telescopes positioned in three geographical locations, will provide McKinney, Tchekhovskoy and Blandford the observational element they need to compare with their simulations. McKinney says they are only in the very early stages of making those comparisons, but may be able to make statements in two to three years as to whether they believe Einstein was correct.

McKinney explains that to test general relativity, simulations of the plasma surrounding black holes need to be emitting light for accurate comparisons with observations, but the project’s simulations currently lack that illumination.

For that reason, he says, “I’m in the process of making my simulations shine, in reality.”


About NICS: The National Institute for Computational Sciences (NICS) operates the University of Tennessee supercomputing center, funded in part by the National Science Foundation. NICS is a major partner in NSF’s Extreme Science and Engineering Discovery Environment, known as XSEDE. The Remote Data Analysis and Visualization Center (RDAV) is a part of NICS.