Nvidia Just Solved Data Centers' Cooling Problem. Here's How.
Nvidia's latest Vera Rubin technology achieves up to a 100% reduction in water consumption for cooling, allowing data centers to drastically cut their water usage. Let's break it down.
Introduction
Water. Right now, that simple yet crucial resource is the number one problem for data centers and AI companies alike – and Nvidia has a solution. At least, for the cooling part of it.
Let’s dive into the data center water problem, why it is such a pressing issue, and how Nvidia’s latest technology could save millions of gallons of the world’s most precious resource.
Before we dive in, here are some key terms to know. If you’re already experienced in this field, feel free to skip ahead.
A GPU, or graphics processing unit, has thousands of simple cores, allowing it to handle various simple tasks at the same time. This is known as parallel processing, and it is crucial to the development of AI, which needs those cores to make thousands of complex calculations and decisions at the same time.
Data centers house tens of thousands of GPUs and are the home base for AI operations and cloud computing services.
Cloud computing is the on-demand delivery of various online services, including servers, storage, databases, and software, across the internet. Cloud computing services store the data collected from users, ranging from your online shopping cart to your digital documents and email archives, inside massive data centers.
Introduction | The Problem | The Solution | The Caveats
Reading Time: 7 minutes. This won’t be super long, so stay tuned.
The Problem
Data centers use a lot of electricity. A single, mid-sized data center runs continuously with a steady demand of 5-20 megawatts (MW). Larger data centers use up to 100 MW at any given time, while hyperscale data centers, like those utilized by tech giants, can maintain a demand of as high as to 1,000 MW, equivalent to 1 gigawatt.
By comparison, a nuclear reactor consistently produces around the same amount of energy – but there’s a crucial difference. The United States has 94 nuclear reactors, as opposed to the nation’s 750 hyperscale data centers.
Within these data centers, 99.9% of the electricity consumed by a GPU is converted into thermal energy. This means that a single graphics card operating at peak wattage can generate up to 450 watts of heat under load, while an entire, AI-capable server rack can emit as much as 100,000 watts of thermal energy continuously. Multiply those numbers by thousands of GPUs, and you end up with staggering statistics.
GPUs themselves operate at a temperature between 150°F (65°C) and 185°F (85°C), but ensuring that they do not exceed that threshold is crucial. Without proper cooling management, GPUs begin thermal throttling, where they systematically slow down or halt operations to prevent overheating. Even then, continued exposure to heat levels above 200°F (94°C) can damage or even melt many AI-level GPUs, which cost upwards of $50,000.
Furthermore, a study conducted by researchers from various institutions, including Cambridge University, found that the facilities even have their own “data center heat island effect.” The report found that the land area around data centers rose in temperature by around 3.6 degrees Fahrenheit (2 degrees Celsius), while certain locations saw an increase of as much as 16.4 degrees Fahrenheit (9.1 degrees Celsius).
For these reasons, proper cooling infrastructure is crucial. Currently, data centers utilize water for this purpose, but their needs are constant and immense. A large data center uses 1-5 million gallons of water a day, equivalent to the daily water usage of a town with 10,000 to 50,000 people.
This water is cooled using large industrial chillers and then circulated through massive, noisy cooling towers, but the chips’ intense heat emissions cause the liquid to evaporate immediately. This prevents data centers from utilizing a closed-loop system – instead, the water they consume is immediately emitted into the atmosphere, depleting local water supplies. As much of the United States continues to face extreme drought conditions, data centers suck up water that is crucial to communities facing these situations.
However, Nvidia may have just changed the game.
The Solution
Nvidia recently announced the arrival of their latest round of next-generation chips – the Vera Rubin platform. Utilizing the brand-new Vera CPU and Rubin GPU, Nvidia’s latest technology can provide technology companies with an incredible return on investment. The company estimates that for every $100 million invested, corporations could make more than $5 billion in revenue, thanks to the new system’s upgraded attention mechanisms and highly efficient computing power.
The key element in this development, though, is the infrastructure around Vera Rubin. Alongside the release of the new chips, Nvidia will provide companies with the Vera Rubin DSX AI Factory reference design, which serves as a blueprint for data centers that use the technology.
This design dictates how to organize the server racks, run power distribution, and, most importantly, manage cooling infrastructure. And, included in the Vera Rubin DSX, could be the solution to data centers’ cooling problems.
The DSX utilizes a closed-loop system filled with a mixture of 75% water and 25% propylene glycol. This combination, when circulated inside a sealed system, prevents water evaporation and ensures that the same liquid continues to cool the chips without the need to replenish the fluid.
Additionally, this coolant is not cold, contradicting the name. Instead, Nvidia’s coolant mixture enters the server racks at a warm 113°F (45°C), eliminating the need for large chillers to cool down the circulating liquid. The Vera CPU and Rubin GPU are engineered to run safely even when supplied with warmer fluids, allowing the mixture to enter at 113°F (45°C) and exit at 131°F (55°C).
Then, the coolant travels to a set of dry coolers on the outside of the building, which are essentially giant industrial car radiators with massive fans. The mixture, which would be at a temperature of 131°F (55°C), is already much hotter than the air outside, allowing it to rapidly shed heat without requiring massive rows of dry cooling fans.
Then, the fluid is circulated back into the data center to re-cool the servers.
This reduces data centers’ water needs for the purpose of cooling to almost zero, allowing them to minimize water consumption. Furthermore, alongside the financial savings for eliminating water-based cooling, data centers will be able to save millions of dollars in electricity costs – rather than paying for expensive mechanical chillers and noisy evaporation towers, investing in dry coolers drastically reduces infrastructure needs.
The Caveats
Of course, there are always drawbacks.
The first and most significant of these shortcomings is the fact that the new strategy only applies to the water consumption of data centers themselves.
Fossil fuel power plants, which include coal and natural gas-based facilities, use more than 2.7 billion gallons of water per day, according to the US Geological Survey, and they supply more than half of all electricity used in data centers.
In order for AI companies to truly minimize their overall water usage, they must also invest in energy sources that use less water – and this doesn’t just mean renewable energy. Even nuclear power plants maintain extremely high water usage demands. Other sources, like hydropower and solar power, consume almost no water, making them an ideal choice for an industry looking to minimize its overall water consumption.
The production of the semiconductor chips used in data centers is also incredibly water-intensive, using thousands of gallons of water in the manufacturing process of a single chip. AI companies must invest in more efficient strategies to produce these semiconductors to further reduce their environmental impact.
Another major drawback is that the DSX only applies to new data centers that are yet to be constructed. Data centers that have already been built cannot possibly integrate new cooling infrastructure without the destruction of the entire facility, which is nearly impossible and raises costs exponentially.
Additionally, the Vera CPU and Rubin GPU are specifically engineered to withstand warmer overall temperatures, allowing this cooling strategy to work. However, data centers that contain older Nvidia chips or GPUs from other companies will continue to maintain their current water consumption.
Finally, Nvidia has confirmed that their new technology also depends on the geographical location of the data center. Facilities in tropical or desert areas could require traditional, mechanical chillers during the hottest parts of the year, when air temperatures could approach or exceed the 113°F (45°C) temperature of the water-glycol mixture.
There is a lot more to this story than just the initial headline. Yes, semiconductor and electricity production will maintain massive water demands. Yes, the architecture will primarily apply to new developments, and it will not be a perfect solution for every geographical region.
But Nvidia’s breakthrough in liquid cooling is still a vital step forward for AI and society as a whole. It proves that engineers driving the artificial intelligence boom are not entirely ignoring the planet’s environmental needs. If this cooling strategy becomes the new standard for the industry, it could mark the first major shift toward sustainable artificial intelligence.
While data centers are unlikely to ever reach zero water consumption, minimizing usage should be the ultimate goal – and the industry is building the tools to get there.
Enjoyed this deep dive? Hit the like button at the top or bottom of this post so we know to keep breaking down big events like this one!