Researchers Use XSEDE HPC Resources to Take on One of Physics’ Most Important and Enduring Problems
by Scott Gibson
While most people think of turbulence as the source of unsettling bouts of chaotic airflow during a flight on an airplane, physicists have a much deeper concept of its significance in the world around us. In fact, Nobel Laureate and theoretical physicist Richard Feynman once said, “Turbulence is the most important unsolved problem of classical physics.”
Probing the puzzling nature of turbulence is essential because of its impact on the flow physics of liquids and gases, and, by extension, the influence of those fundamental states of matter streaming outside and inside things. For example, turbulence must be considered in designing vehicles and in understanding how particles — such as pollution and volcanic ashes — disperse in the atmosphere. Turbulence also affects the flow inside a jet engine, a combustor or a nuclear reactor.
To convey the essence of the ubiquitous influence of turbulence, researcher Antonino Ferrante of the William E. Boeing Department of Aeronautics and Astronautics of the University of Washington, Seattle, quotes Greek philosopher Heraclitus: “Everything flows and nothing abides; everything gives way and nothing stays fixed.”
“As I sit and look around me, I notice several examples of turbulent flows: the smoke flowing out of a chimney, the wind moving between the leaves and branches of trees, massive clouds moving in the atmosphere, the air surrounding a flying bird and an airplane,” Ferrante says. “No matter how big or small, or how close or far you look, you will see fluids in motion. As the ratio of the inertial forces to the viscous forces in the flow (that is, what’s known as the Reynolds number) increases above a certain threshold, the flow transitions from laminar to turbulent, or from smooth to random. Most flows in nature and engineering applications are turbulent. Thus, understanding turbulent flows is very important for human progress and for a sustainable planet Earth.”
XSEDE HPC Resources: Essential to Project Success
Ferrante is principal investigator for the Computational Fluid Mechanics group, a research team engaged in the study of turbulence modeling and simulations, one of the most challenging areas of fluid dynamics — the natural science of fluids, both gases and liquids, in motion. Turbulence simulations are particularly difficult because of the wide-range scales of motion involved. Resolving all of those scales requires fine computational grids with billions of points that include the tiniest of scales, where viscous dissipation — the heating of the fluid due to resistance — occurs.
The goals of the team’s project are to simulate fluid flow using the largest Reynolds number ever reported, perform the first direct numerical simulation (DNS) of fully resolved droplet-laden isotropic (without a preferential direction) turbulence and share the generated DNS database with the scientific community. Accomplishing their objectives required the team to use the XSEDE-allocated Kraken supercomputer at the National Institute for Computational Sciences (NICS).
The researchers also availed themselves of the XSEDE Extended Collaborative Support Services program, through which they consulted with staff at NICS and the National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana–Champaign.
The assistance of NICS enabled Ferrante’s team to simulate turbulent flows as well as turbulence coupled with other phenomena — including chemically reactive and multiphase (gas and liquid combined) turbulent flows — and get results in a reasonable time.
NCSA provided petascale scaling and deployment, as well as the development of a high-level HDF5 (Hierarchical Data Format Release 5) parallel input and output library layer (called H5DNS). HDF5 is the name of a set of file formats and libraries designed to store and organize large amounts of numerical data. NCSA also contributed custom visualization of simulation results to the project.
DNS modeling, Ferrante explains, is particularly HPC dependent because it requires that several flow variables, such as velocity and pressure, be advanced in time in billions of grid points. The problem the researchers were investigating required a minimum resolution of 10243 data points.
“Using Kraken, we were able to solve the problem involving billions of unknowns accurately,” Ferrante says. “For the first time, we are simulating the effects of droplets on turbulence by performing fully resolved DNS. In practice, we are running a real-life experiment on a supercomputer rather than a wind tunnel.”
Ahead: Larger-scale Simulations and More-Complex Flows
Ferrante says the outcome of the research was an accurate and robust numerical methodology that couples the droplets and the turbulent flow to better understand droplet-laden turbulent flows, and that research in turbulence simulations will continue on larger scales and solve more-complex flows. He presented a talk on the results of the project in May 2013 to the International Conference on Multiphase Flow in Jeju, Korea, and has submitted a paper entitled “A coupled pressure-correction/volume of fluid method for DNS of droplet-laden isotropic turbulence” to the publication Computers and Fluids.
About NICS: The National Institute for Computational Sciences (NICS) operates the University of Tennessee supercomputing center, funded in part by the National Science Foundation. NICS is a major partner in NSF’s Extreme Science and Engineering Discovery Environment, known as XSEDE. The Remote Data Analysis and Visualization Center (RDAV) is a part of NICS.