New Flatiron Institute Supercomputer the Most Power-Efficient Ever Built

Nov 14, 2022

Leveraging the new machine’s speed and parallel processing, researchers will apply machine learning techniques to open questions in science.
Collaboration with Lenovo and NVIDIA pushes the boundaries of sustainable high-performance computing (HPC) to earn #1 on Green500 List

A new supercomputer operated by the Flatiron Institute in New York City tops the latest Green500 List of the most power-efficient supercomputers in the world.

The revised list, which ranks supercomputers based on the number of floating-point operations per second (or ‘FLOPS’) per watt of power, clocks the new supercomputer at 65.091 billion FLOPS per watt. That edges out the previous record holder, which managed 62.684 billion FLOPS per watt. At 2.038 million billion FLOPS, the new Flatiron Institute supercomputer is the world’s most energy efficient supercomputer as measured by the

“This supercomputer opens up opportunities for doing new kinds of science,” says Ian Fisk, co-director of the Flatiron Institute’s Scientific Computing Core. “This is a workhorse machine, and we’re going to let our researchers try new things and drive discoveries.”

Researchers at the Flatiron Institute will tap the new supercomputer’s power to tackle thorny problems in computational astrophysics, biology, mathematics, neuroscience and quantum physics. Leveraging NVIDIA’s accelerated computing platform, the system is well suited for machine learning applications, such as multi-body simulations of the universe’s evolution, predicting how proteins fold and function, and finding correlations in genomic studies. Accelerated computing also excels at linear algebra calculations that simulate how electrons behave at the quantum scale. That’s because GPUs run many more calculations in parallel than traditional CPUs can.

Housed in a data center in New Jersey, the system itself was built by Lenovo and leverages the efficiency capabilities of the company’s ThinkSystem SR670 V2, a server designed to install easily into traditional data centers and be accessible to more researchers. Its efficient design effectively keeps things cool even in a high-heat, multi-GPU environment. The new Lenovo supercomputer includes 80 new NVIDIA H100 Tensor Core GPUs, connected with NVIDIA’s Quantum 200Gb/s InfiniBand network and installed in standard data center racks.

“As a world leader in supercomputing, Lenovo is committed to making the same Exascale-ready technologies available for users of all sizes and across all industries,” said Kirk Skaugen, Executive Vice President, Lenovo Group and President of Infrastructure Solutions Group. “Our collaboration with Flatiron and NVIDIA demonstrates a pivotal milestone in enabling supercomputing technology at nearly any scale and supporting smarter technology for all.”

Researchers can start from as small as a single server, to a partial rack, all the way to exascale-levels while helping to accelerate scientific outcomes and achieving new breakthroughs in sustainable computing.

The transistors on the NVIDIA chipsets measure just 5 nanometers wide, which means that more computational muscle can be packed on the same size chip. That smaller transistor size and many other optimizations helped reduce the new supercomputer’s energy consumption and maximize performance efficiency.

Lowering a supercomputer’s power usage without sacrificing performance comes with many benefits. Electricity is a significant component of a machine’s operating costs and environmental impact. Every watt consumed is converted into heat that must be removed from the system, requiring even more power. Also, more energy-efficient systems can slot into existing data centers without needing costly electrical upgrades.

Fisk says the number one power-efficiency ranking wasn’t the goal. “This computer will allow us to do more science with smarter technology that uses less electricity and contributes to a more sustainable future,” he says. “That’s what’s important to us.”

The new Flatiron Institute supercomputer supplants Frontier Test & Development System (TDS), located at Oak Ridge National Laboratory in Tennessee, on the power efficiency list. Frontier TDS set an extremely high bar on the June 2022 list at 62.684 billion FLOPS per watt, while the main Frontier system — which broke the exaFLOP (1 billion FLOPS) barrier and remains the fastest supercomputer in the world — checks in at 52.227 billion FLOPS per watt. As a result, unseating Frontier was going to be a daunting challenge, Fisk says.

While both are record setters, the new Flatiron Institute supercomputer is a markedly different beast than Frontier, which was laboriously designed and built to reach 1 exaFLOP. Conversely, the new Flatiron Institute system utilizes commonly available systems, NVIDIA accelerated computing and InfiniBand networking, making it “very high performance and very efficient without being particularly exotic,” Fisk says. “It only took a couple of people to load the system in. This kind of efficiency is now accessible to a lot more groups rather than just the largest supercomputing centers.”

Focused on a bold vision to deliver smarter technology for all, we are developing world-changing technologies that power (through devices and infrastructure) and empower (through solutions, services and software) millions of customers every day and together create a more inclusive, trustworthy and sustainable digital society for everyone, everywhere.