For many years, servers have been the basic constructing block of information facilities. Deploying a workload requires sufficient servers to help that workload. Infrastructure monitoring, energy administration, and so forth additionally largely happen on the server stage.
Within the AI period, server-scale approaches to knowledge facilities have gotten more and more insufficient. A greater technique, arguably, is rack-scale computing – an concept that’s not all that new, however that has maybe lastly come into its second.
What Is Rack-Scale Computing?
Rack-scale computing is the follow of provisioning {hardware} inside knowledge facilities utilizing server racks fairly than particular person servers as the principle unit of IT infrastructure.
In different phrases, while you embrace rack-scale computing, your major objective is to make sure you have sufficient racks – and the optimum mixture of compute, reminiscence, storage, and networking {hardware} inside every rack – to help a given workload.
That is totally different from conventional infrastructure methods, which focus on particular person servers. Most knowledge middle directors are accustomed to pondering when it comes to what number of servers are assigned to workloads, not what number of racks are provisioned for them, which is why it’s widespread to dimension Kubernetes environments primarily based on what number of nodes they embody, for instance, or use complete servers as a proxy for complete knowledge middle capability.
Below a rack-scale method, an information middle’s complete variety of servers ceases to be the main target. As an alternative, the important thing issue for driving infrastructure success turns into the overall variety of racks and the configuration of every rack.
Rack-scale computing finds its second within the AI period, the place 1 MW racks ship the built-in efficiency conventional server-centric approaches can not match.
The Advantages of Going Rack-Scale
At first look, rack-scale computing may appear to be an unconventional method to knowledge middle infrastructure administration. In spite of everything, the overall variety of servers that may match inside a single rack can fluctuate extensively relying on rack dimension. If the computing capability of a rack is so variable, why would you deal with racks as the basic constructing block of information facilities?
A part of the reply is that servers, too, aren’t truly a really constant manner of measuring infrastructure capability as a result of the computing energy of servers can fluctuate extensively.
The extra compelling motive for embracing rack-scale computing is that it permits a extra versatile method to knowledge middle infrastructure administration. Particularly, rack-centric infrastructure permits companies to:
-
Meet the wants of large-scale workloads. Fashionable workloads typically require a number of servers – and therefore profit from having a complete rack assigned to them.
-
Construct extra resilient infrastructures. Particular person servers are susceptible to failure, but it surely’s uncommon for a complete rack to go down. Due to this fact, while you use racks as your constructing block, your workloads are inherently extra dependable.
-
Optimize infrastructure configurations. Specializing in rack design and parts makes it simpler to outfit every rack with {hardware} optimized for a given workload. For instance, if a workload generates an particularly excessive quantity of community visitors, it may be supported with a rack that features a high-end swap – and even a number of switches.
A Resolution Whose Time Has Lastly Come
Apparently, the idea of rack-scale computing has been round for greater than a decade. Microsoft was promoting it in 2013, and distributors like Intel have been seizing upon it years in the past as a part of composable infrastructure methods.
On the time, rack-scale computing by no means actually caught on totally. The info middle trade didn’t shift to a mannequin whereby racks grew to become the basic constructing blocks of infrastructure.
However the necessity to help trendy AI has catalyzed renewed curiosity in rack-scale infrastructure methods. For instance, talking at Knowledge Middle World 2025, analyst Jeremie Eliahou Ontiveros pointed to rack-scale architectures as a part of the answer to provisioning AI workloads with adequate infrastructure.
This method is especially well-suited to AI workloads. Not solely do AI workloads require large quantities of compute, reminiscence, and (in lots of instances) storage sources, in addition they work loads higher when the infrastructure they run on is optimized on the {hardware} stage. Rack-scale computing might help to satisfy each objectives.
For instance, 1 MW racks – which might accommodate a lot larger server capacities than conventional racks – might help be sure that AI workloads have the sources they should function. On the identical time, rack architectures that optimize the motion of information between particular person servers inside a rack, whereas additionally serving to to steadiness warmth dissipation, assist to keep away from processing bottlenecks and optimize workload efficiency.
Particular person server provisioning can not obtain comparable optimization since it could be more difficult to combine these servers optimally.
The Way forward for Rack-Scale Computing
To make certain, rack-scale computing has drawbacks. Chief amongst these is that when racks change into the first models, they could restrict scalability as a result of it will likely be difficult to provision extra servers than a single rack can deal with. Nonetheless, this concern diminishes if knowledge facilities shift towards higher-capacity racks – resembling, once more, these able to accommodating as much as 1 MW value of {hardware}.
Thus, as racks modernize, anticipate knowledge middle architectures to modernize with them as companies pivot towards rack-scale approaches. Particular person servers will stay necessary as a result of not each workload requires its personal devoted rack. However probably the most important – and costly – workloads are more likely to function at rack scale.
