One factor is abundantly clear: Each single information heart will turn out to be an AI information heart. And the important differentiator can be how shortly they’ll get there.
I’ve had the pleasure of writing for DCN for a while, and I’ve additionally authored the AFCOM State of the Information Middle Report for the previous eight years. Over these years, I’ve seen this business develop and shift to accommodate new tendencies. Nonetheless, nothing has been as thrilling and ground-moving as what we have seen over the previous 12 months. The query is: can rack density maintain tempo?
To contextualize at present’s rack density conundrum, I’d prefer to take us again. Eight years in the past, in our first-ever State of the Information Middle report, respondents indicated that their common density was 6.1 kilowatts per rack. Given the everyday sorts of workloads information facilities supported, this metric is fairly anticipated. Whereas high-density functions have been a factor, most information facilities have been nonetheless operating very conventional functions like e-mail servers, databases, and different business-critical providers.
Over the previous 12 months, every little thing has modified. Within the 2024 report, respondents indicated that the common density of the rack had elevated to 12 kilowatts. Most respondents (60%) are actively working to extend density of their racks (58%), primarily by bettering airflow, adopted by containment (42%) and liquid cooling (40%). I am positive you may guess what’s driving all of this. Per the report, most respondents (53%) consider new AI workloads (generative AI) will “positively” enhance capability necessities for the colocation business.
This is the loopy half: Despite the fact that density has doubled, at the very least per our report, it is nonetheless inadequate to help AI and high-density structure. To place this inperspective, a single node of NVIDIA’s DGX H100 playing cards will eat between 8-11 kilowatts per rack. Based mostly on our findings, a standard information heart might solely help one among these high-end models of their rack regardless of development in density. This fast tempo of evolution has turn out to be the driving pressure of innovation in our business. Essentially the most important distinction is that it is occurring extremely quick.
Are Liquid Cooling and Rear-Door Warmth Exchangers the Reply?
Certainly one of my mentors, Peter Gross, as soon as stated, “The info heart business loves innovation so long as it is ten years previous.” The problem is that we actually do not have ten years. We do not even have ten months anymore.
One thing should change because it pertains to density. We are actually asking information heart operators to maneuver from supporting 6-12 kilowatts per rack to 40, 50, 60, and much more KW per rack. Whereas airflow and containment are wonderful strategies to enhance effectivity and density, we’re shortly reaching the boundaries of the physics of airflow.
So, the subsequent logical step is to show to liquid cooling. In full transparency, I’ve spent the final 12 months working in generative AI and inference, particularly specializing in the info heart and colocation house. We have taken information heart companions on a unprecedented journey to turn out to be extra ready to help generative AI and high-density use circumstances. The excellent news is that rack producers, direct-to-chip liquid cooling applied sciences, and options like rear door warmth exchangers have come a great distance.
In accordance with Vertiv’s put up, “In contrast to air cooling, which continues to work ever more durable, the cooling mechanism of a rear door warmth exchanger or direct chip liquid cooling answer produces higher cooling outcomes with much less work, resulting in much less vitality use and fewer carbon emissions. These applied sciences is also used collectively to drive 100% of the warmth load into the fluid.”
“The proliferation of more and more dense AI workloads will usher within the age of liquid cooling,” stated Brad Wilson, VP Expertise at Vertiv says that the proliferation of more and more dense workloads “will usher within the age of liquid cooling.”
“Whereas direct-to-chip cooling will finally symbolize essentially the most important cooling effectivity enhance because the introduction of the PUE metric, rear-door warmth exchangers are an efficient and energy-efficient answer for medium or high-density functions – together with present air-cooled information facilities which are in search of a liquid cooling technique,” Wilson stated.
I am a giant fan of rear-door warmth exchangers, that are sometimes radiator-like doorways connected to the again of racks that permit for direct warmth alternate or for chilled water or coolant. These are fantastic options that may be built-in into conventional colocation architectures. You don’t have to tear and exchange your whole ecosystem to help high-density structure. In truth, that is exactly what one information heart chief did.
Stepping on the ‘GaaS’ – Constructing a Facility to Assist Over 3000 H100 GPUs
To help the most recent use circumstances round AI, we should transfer past the hype and have a look at precise implementations. So, let’s take a look at someone who has truly executed their imaginative and prescient.
At Information Middle World 2024, Ken Moreano, President and CEO of Scott Information, will present on the creation of an information heart spanning 110,000 sq. ft, outfitted to deal with greater than 3,000 NVIDIA H-100 GPUs. His speak will cowl the transition of a facility from a Division of Protection SCIF (Delicate Compartmented Info Facility) into an information heart that meets the Uptime Institute’s Tier III requirements, emphasizing the incorporation of know-how providers that facilitate GPU as a Service (GaaS) and large-scale HPC colocation.
“Our profitable working historical past, together with our entrepreneurial tradition, positioned Scott Information to leverage early market alerts for intentional investments in our infrastructure,” says Ken Moreano, President and CEO of Scott Information Middle. “This funding demonstrated a strong enterprise case but additionally reaffirm our longer-term core values of being a technical innovator and chief in our market house.”
In talking with Ken, we realized that he was motivated by the market and his enterprise purchasers. Many companies have already got their most important providers with their colocation companions. The following logical step is to help high-density AI workloads. Nonetheless, the largest problem is that these information heart companions weren’t prepared or outfitted to help high-density workloads. And that is why Scott Information Middle went on this journey.
And, after seeing what number of information heart of us attended NVIDIA’s occasion, I will not be stunned if extra amenities quickly increase their high-density computing capabilities.
Wanting Forward and Executing a New Imaginative and prescient
It is vital to notice that what we’re experiencing as an business is rather more than a know-how shift. What we’re seeing is a shift in how humanity interacts with information. For the primary time, we are able to ask a query to information and get a “aware” reply. Authentic content material is generated based mostly on our request. Behind that’s a unprecedented quantity of computing to create giant language fashions and do inference coaching. Our amenities can be on the coronary heart of this revolution. Your process can be to search out inventive, progressive, and sustainable methods to help this new period of digital infrastructure.
Let me offer you an instance. Just lately, producer Vertiv suggested us to “Begin with the rear doorways, then take into account direct-to-chip cooling.” They’re completely proper in that method. Even Nvidia CEO Jensen Huang on the 2024 SIEPR Economic Summit acknowledged that Nvidia’s next-generation DGX servers may have liquid cooling.
