The National Institute for Computational Sciences

Boosting Environmental Research

NICS and XSEDE Advance an Investigation of the Gulf Coast Ecosystem

Annette Engel (in pink), associate professor of earth and planetary sciences at the University of Tennessee, Knoxville, samples Louisiana marsh sediment with other researchers from the Coastal Waters Consortium. Engel and her team are studying changes in marsh sediment microbiology using the Nautilus supercomputer at the National Institute for Computational Sciences in consultation with NICS high-performance computing experts through the National Science Foundation’s Extreme Science and Engineering Discovery Environment (XSEDE) Extended Collaborative Support Service program. [Image courtesy: Audrey T. Paterson, the University of Tennessee, Knoxville]

On April 20, 2010, the Deepwater Horizon oil rig in the Gulf of Mexico exploded and sank, claiming the lives of 11 people and massively assaulting the ocean and connected ecosystem. After the explosion, oil gushed from the bottom of the sea at the Deepwater Horizon site for 87 days until it was capped on July 15, 2010.

Also referred to as the Macondo oil spill or the BP oil spill, the disaster is considered the largest accidental marine release of oil into the environment in the history of the petroleum industry.

Ever since the cataclysm, Annette Engel, associate professor of earth and planetary sciences at the University of Tennessee, Knoxville, has been visiting the Gulf Coast to study how the oil has changed the coastal ecosystems.

Researcher Annette Engel of the University of Tennessee, Knoxville, on location in a Louisiana marsh to investigate the hydrocarbon degradation potential of microbes in coastal environments following the Deepwater Horizon oil spill. [Image courtesy: Annette Engel]

Project Principal Investigator Engel and her team of graduate and undergraduate students and staff have made new discoveries about bacterial diversity and oil degradation processes in marshes.

The research has uncovered fundamental changes in the types of bacterial communities associated with oil and carbon degradation of importance to oil quality and quantity, as well as to the overall ecosystem of the Gulf Coast area. According to project findings, the changes in microbial diversity have been associated with increased marsh deterioration.

Analyses of DNA sequences from microorganisms in water, soils, and sediments inform Engel and her team about the base of ecosystem functioning in the natural environments. Microbes are at the food and energy base of any ecosystem, and they serve as the interface between the physical and chemical conditions of a habitat and the animals and other life that make up an ecosystem. Studying the microorganisms will enable Engel and her team to understand what communities existed in the marshes before the spill, and how those communities have changed in response to the oil.

"By determining the microbial response and investigating how long it takes for communities to return to pre-spill membership, if they ever will, we may be able to predict how the overall marsh food, comprised of plants, insects, birds, fish, and other animals that require Gulf Coast marsh habitats, may have responded and will continue to respond in the years to come," Engel says.

But while advances in DNA sequencing technology make it possible to obtain hundreds of thousands to millions of sequence reads from a single sample, Engel found that analyses of the large datasets—specifically to determine fluctuations within microbial taxonomic groups across time and space—were limited by the capabilities of her lab computers.

“I was really happy that through a series of circumstances we were able to connect with NICS [the National Institute for Computational Sciences] and Bhanu Rekepalli and learn about all the different opportunities that they offered to non-traditional high-performance computing users,” she says.

Rekepalli introduced Engel to the Extended Collaborative Support Service (ECSS) program of the National Science Foundation’s Extreme Science and Engineering Discovery Environment (XSEDE). ECSS enables researchers to request XSEDE computing, storage, and visualization resources through a peer-reviewed allocations process; or ask to be paired with expert staff members for an extended period.

ECSS staff provide expertise in many areas of advanced cyberinfrastructure and can work with a research team to accelerate and enhance their efforts through that knowledge.

Marshes like this one in Louisiana represent about 40 percent of the wetlands of the continental United States. The Coastal Waters Consortium has been studying marsh ecosystem processes and health since the Deepwater Horizon oil spill in 2010. [Image courtesy: Annette Engel, the University of Tennessee, Knoxville]

As an XSEDE service provider, NICS was able to make HPC consultation services from its computational scientists Rekepalli and Junqi Yin and an allocation on the NICS-managed Nautilus supercomputer available to Engel’s project.

Rekepalli met with Engel and her team to develop a work plan and provide her students with basic HPC training, which included remote login, module operation, and job submission. They also discussed the sequence data pipeline analysis that would be used in the project.

Engel’s Ph.D. student, Chanda Drennen, has handled most of the HPC tasks on the project, working directly with Yin, who has helped her use the advanced computing in analyzing the DNA sequence data.

Drennen’s role in the project corresponds with the mission of the Coastal Waters Consortium (CWC), which through the Gulf of Mexico Research Initiative, is providing the funding for Engel and her team. CWC’s mission is to have students directly involved, or driving, the research, and for Drennen, the ability to perform the data analyses using HPC is essential to the completion of her dissertation.

NICS instrumented and optimized several commands in the open source bioinformatics application called mothur for an operational taxonomic unit analysis pipeline with OpenMP on shared memory architecture, and the result was an acceleration of processing time by an order of magnitude as compared with what was possible on the desktop.

The NICS ECSS team—Rekepalli, Yin, and Pragnesh Patel—collaborated with Drennen and Engel on a proceeding paper about the project for the XSEDE14 conference in Atlanta, July 13–18, 2014.

“We are able to process the data and get out results that are allowing us to intrepret the changes in the microbial communities from these marshes, and we couldn’t do that before,” Engel says. “But it has been a very long process getting this worked out. If it weren’t for XSEDE, we would not even be to this point, I think.”

Engel and her team are now able to perform operational taxonomical analyses on hundreds of thousands of sequencing data and get a clearer and more comprehensive understanding of it.

Researchers will be assessing the impact of the Deepwater Horizon oil spill for a long time to come, and HPC will accelerate and enhance the quality of discoveries that Engel and her team can add to the body of knowledge about the impact of the disaster on coastal environments.

Project details will be provided in papers that are currently in development for submission to journals. Engel and members of her team have given oral and poster presentations on the work at conferences.


Yin, J., Rekepalli, B., Patel, P. Drennen, C., Engel, A.S. (2014) Instrumenting Genomic Sequence Analysis Pipeline Mothur on Shared Memory Architecture. XSEDE ’14 Proceedings of the 2014 Annual Conference on Extreme Science and Engineering Discovery Environment, ACM New York, NY, USA; Article No. 19. DOI:10.1145/2616498.2616505.

Drennen, C., Engel, A.S. (2014) Increased marsh deterioration associated with changes in microbial Diversity following the 2010 Deepwater Horizon oil spill. 2nd Gulf of Mexico Oil Spill and Ecosystem Science Conference, Mobile, AL, Jan. 26–30. (poster presentation)

Drennen, C., Engel, A.S., Turner, R.E. (2013) Response and recovery of microbial communities in Louisiana coastal marshes after the Deepwater Horizon oil spill and possible implications for marsh erosion. 22nd Biennial Conference of the Coastal and Estuarine Research Federation, “Toward Resilient Coasts and Estuaries, Science for Sustainable Solutions,” 3–7 November 2013, San Diego, CA. (poster; received 2nd best poster award)

Information on Project Funding

Original funding for the CWC was from 2011–2014. The new award, for CWC II, is from 2015–2017.

Scott Gibson, science writer, NICS, JICS

Article posting date: 3 February 2015

About JICS and NICS:The Joint Institute for Computational Sciences (JICS) was established by the University of Tennessee and Oak Ridge National Laboratory (ORNL) to advance scientific discovery and leading-edge engineering, and to further knowledge of computational modeling and simulation. JICS realizes its vision by taking full advantage of petascale-and-beyond computers housed at ORNL and by educating a new generation of scientists and engineers well-versed in the application of computational modeling and simulation for solving the most challenging scientific and engineering problems. JICS operates the National Institute for Computational Sciences (NICS), which had the distinction of deploying and managing the Kraken supercomputer. NICS is a leading academic supercomputing center and a major partner in the National Science Foundation's eXtreme Science and Engineering Discovery Environment (XSEDE). In November 2012, JICS sited the Beacon system, which set a record for power efficiency and captured the number one position on the Green500 list of the most energy-efficient computers.