Common rack energy density has greater than doubled in two years, from 8 kW to 17 kW, and is projected to achieve 30 kW by 2027, in accordance with an October 2024 McKinsey report, with AI coaching racks already nicely forward of that common.
These limits present up in GPU clock pace. H100 GPUs beneath insufficient air cooling can throttle to a fraction of their rated clock pace inside seconds of a sustained coaching run. In distributed jobs throughout 1000’s of GPUs, one throttled chip can stall the whole run. The DOE estimates cooling accounts for as much as 40% of knowledge heart vitality use.
JLL research establishes three density thresholds:
- As much as ~20 kW per rack: air cooling is sufficient
- As much as ~100 kW: rear-door warmth exchangers lengthen viability
- Above ~175 kW: immersion cooling is required
Direct-to-chip cooling fills the center band, dealing with densities between ~100 and ~175 kW the place rear-door exchangers fall quick and immersion shouldn’t be but warranted.
Scorching water modifications the economics
Mechanical chillers are one of many largest vitality attracts in any liquid-cooled knowledge heart, and till just lately there have been an unavoidable price of liquid cooling. Nvidia’s Vera Rubin processor is altering that.
At CES in January 2026, Jensen Huang introduced that Vera Rubin helps liquid cooling at 45 levels Celsius, excessive sufficient for knowledge facilities to reject warmth by way of dry coolers utilizing ambient air slightly than mechanical chillers. Nvidia’s CES press release confirmed Rubin is in full manufacturing, with buyer availability within the second half of 2026. In line with Nvida’s product specifications, the Vera Rubin NVL72 makes use of warm-water, single-phase direct liquid cooling at a forty five°C provide temperature, permitting knowledge facilities to reject warmth by way of dry coolers utilizing ambient air slightly than energy-intensive chiller techniques.
