Peregrine Supercomputer flies high and fast

 By Andrew Burger

Located inside the U.S. Dept. of Energy National Renewable Energy Laboratory’s Energy Systems Integration Facility (ESIF) in Golden, Colorado, Peregrine is the largest high-performance, supercomputing system in the world dedicated to advancing renewable energy and energy efficiency technologies…

Designed and built by Hewlett-Packard Enterprise (HPE) to NREL’s specifications, Peregrine is comprised of a combined 58,752 Intel Xeon processors (SandyBridge, IvyBridge and Haswell), the computing capacity of which maxes out at some 1.680 petaflops (quadrillion floating point) calculations per second. In addition, Peregrine employs 576 Intel Phi many-core co-processors with an aggregate computing capacity of about 582 teraflops. That brings Peregrine’s total computing capacity to 2.24 petaflops.
NREL is one of 20 U.S. Dept. of Energy National Energy Laboratories dispersed across the U.S., each of which engages with private and public sector research scientists, engineers and teams at the cutting and leading edges of renewable energy and energy efficiency technology R&D. In addition to NREL’s own staff, any and all of them can make use of Peregrine’s super-computing capabilities to help solve their most vexing, data-intensive problems.

The U.S. Dept. of Energy National Renewable Energy Laboratory's Energy Systems Integration Facility (

NREL’s Peregrine supercomputer

Configured in a Linux cluster, Peregrine uses a fast InfiniBand network to connect computing nodes. The massive amounts of data Peregrine processes require massive amounts of data storage capacity. Hence, the supercomputer is equipped with 2.25 petabytes of disk storage.
In addition, NREL’s Gyrfalcon Mass Storage System provides long-term data retention for the high-performance computing environment. Gyrfalcon is designed so that the most frequently used data is quickly accessible while infrequently accessed data is stored economically. That’s achieved by storing the most frequently used data on high-performance disks while transferring older, less used data to more economical magnetic tape storage.
The data storage and management process is completely automated. Custom-built software algorithms, a large Oracle StorageTek robotic tape library and a set of high-performance T10000C tape drives lie at the core of Peregrine’s data storage architecture.
Fittingly, Peregrine is one of, if not the most, energy efficient supercomputers in the world. Among other notable innovations, NREL’s HPC (High-Performance Computing) data center makes use of a warm-water liquid cooling system and captures and recovers waste heat for reuse. Over time, that translates into significant cost savings, as well as lower environmental impacts and natural resource use.

High-performance computing for renewables, energy efficiency R&D

Scientists and engineers use Peregrine to create, run and analyze computer models and simulations in order to explore processes and technologies that cannot be directly observed in a laboratory, or are too expensive or too time-consuming to be conducted otherwise.
Doing so saves time and money. It also raises the likelihood of making breakthroughs and useful advances considerably. Furthermore, using such technology reduces the risks and uncertainties that often serve as barriers to industry adoption and application.
The centerpiece of NREL’s HPC Data Center, Peregrine resides within the Energy System Integration Facility on NREL’s campus in Golden, Colorado. As NREL explains: “ESIF is a center for research that addresses challenges related to the interconnection of distributed energy systems and the integration of renewable energy technologies onto the electricity grid”.

Energy secretary Ernest Moniz joins Steve Hammond, NREL in unveiling Peregrine (

Unique energy insights and innovation

Adjacent to NREL’s HPC data center is the Insight Center. There, researchers use advanced visualization technology to provide on-site and remote views of experimental data, as well as high-resolution visual imagery and large-scale simulation data.
The Insight Center is divided into two work spaces. In the Collaboration Room, researchers spanning a wide variety of scientific and engineering disciplines create interactive visualizations that make it much easier to view, comprehend and gain insight into highly complex, large-scale data sets, systems and operations.
The Collaboration Room’s main workspace provides an immersive, stereoscopic virtual environment that’s created by six projectors that illuminate two surfaces — a wall and the floor. This virtual, dynamic, 3D virtual work space can be used in conjunction with an optical tracker that results in visualizations that respond in relation to the movement of the user.
As a result, researchers are able to physically explore their data by viewing and interacting with visualizations in real-time. This includes running tests and simulations of renewable energy, energy efficiency and systems and grid integration equipment and technologies.

Real-time, multi-dimensional, interactive energy data visualization and collaboration

The fundamental aim is to support the analysis of a wide range of research within the U.S. Dept. of Energy’s Office of Energy Efficiency and Renewable Energy (EERE). That spans any and all relevant research areas that make use of and/or generates large data sets.
That, in turn, may be a representation of new renewable energy, energy efficiency or grid integration equipment, devices or processes, which is interactive and rendered in three, and possibly four or more, dimensions. An example might be an atomistic simulation and 3D or 4D visualization of the molecular dynamics of new materials to the boundaries of wind turbine wakes or other turbulent processes and structures in wind farm simulations.
Use of so-called measured data, rather than predicted or estimated data, provides another example. Such data extends from exploring the nano-scale structures seen in 3D microscopy data associated with pretreatment of biomass to the complex dynamics of the North American electrical power grid, NREL explains.
Complementing the Collaboration Room is the Visualization Room. A meeting space, the Visualization Room features a large, rear-projected, 14-megapixel image display that creates large-scale, high-resolution visual imagery. Researchers use these interactive visualizations to convey information and illustrate research findings to stakeholders and visitors more clearly and concisely.

The ESIF Insight Center uses advanced visualization technology to provide on-site and remote viewing of experimental data, high-resolution visual imagery and large-scale simulation data (Dennis Schroeder, NREL)

The Visualization Room also stimulates the exchange of ideas among NREL researchers and their collaborating partners. “Using the high-resolution, large-scale display, researchers and others now have the visual ‘real estate’ to lay out a significant amount of data that will enable them to analyze large-scale simulations, ensembles of simulations and highly detailed visual analytics displays,” NREL says.
All told, some 200 NREL researchers and support staff work in NREL’s Energy Systems Integration Facility. The focus is on energy systems integration – the analysis, design and control of the interactions and interdependent nature of renewable and other energy resources.
“By optimizing energy systems across multiple pathways and into the Smart Grid, we can better understand and make use of co-benefits that increase reliability and performance, reduce cost and minimize environmental impacts”, NREL explains.

READ MORE: A Swiss story about supercomputing by Nicholas Newman

about the author
Andrew Burger
Andrew Burger has been reporting on energy, technology, political economy, climate and the environment for a variety of online media properties for over five years.