You’ll be able to’t practice and function most forms of AI workloads with out Graphics Processing Models, or GPUs. However you can also’t deploy GPUs with out servers to host them. Because of this conversations about the way forward for GPUs inside information facilities should take into account not simply how companies will purchase, configure, and energy GPUs themselves, but additionally how they’ll construct out the server infrastructure the place the GPUs will stay.
To that finish, let’s study what a GPU server entails, specializing in the way it differs from conventional servers and what meaning for the position of GPUs in information facilities.
What Is a GPU Server?
A GPU server is just a server geared up with a number of GPUs. Usually, GPU servers host workloads that require huge quantities of parallel computing capability. Since GPUs are perfect for parallel processing, they excel at use instances like coaching AI fashions, which work finest when workloads can carry out many operations on the similar time.
GPU servers embrace Central Processing Models, or CPUs. A CPU is critical to carry out conventional computing duties, like operating the working system that controls the server. However a GPU server gives GPU compute assets along with a conventional CPU.
The Rising Position of GPU Servers in Information Facilities
Till the generative AI increase, it was uncommon to seek out GPU servers inside information facilities. As an alternative, most information heart servers have been conventional gadgets whose compute assets have been supplied solely by CPUs, that are nice at powering most forms of conventional workloads, like internet hosting web sites and databases.
However as demand for infrastructure able to supporting AI mannequin coaching and inference has grown, the power to host GPU servers has develop into more and more vital for information facilities.
Excessive-density GPU servers with superior cooling techniques energy AI workloads in trendy information facilities. Picture: Alamy.
GPU Servers vs. CPU Servers
In lots of respects, GPU servers are much like conventional servers that embrace solely CPUs. They’re usually the identical dimension, match into normal server racks and require the identical forms of networking and energy connections.
Nonetheless, GPU servers are distinct in a couple of vital methods:
-
Extra enlargement slots: Conventional servers normally embrace comparatively few enlargement slots, which technicians can use to attach GPUs and different specialised {hardware} gadgets to a server motherboard. It’s because most conventional servers don’t use many add-on playing cards. However since offering a strategy to plug in GPUs is important for a GPU server, GPU servers want enlargement slots – usually, extra of them than you’d discover in a conventional server. Some GPU servers have sufficient slots to accommodate as many as 10 particular person GPUs.
-
Elevated energy wants: GPUs eat a number of electrical energy. That implies that GPU servers have to be able to delivering extra energy than a typical CPU-only server. The way in which that energy flows to GPUs can range. In some instances, it comes by the server motherboard, however high-end enterprise GPUs typically have devoted energy connections. Both manner, nonetheless, a GPU server (and the rack that hosts it) wants to have the ability to present sufficient whole incoming power to maintain GPUs operating.
-
Better cooling capability: Excessive energy consumption goes along with excessive warmth output. Because of this, GPU servers have to be particularly adept at dissipating warmth. They’re more likely to require extra superior cooling options than conventional followers.
Getting ready Information Facilities for GPU Servers
As a result of GPU servers can sometimes be housed in conventional server racks, they don’t pose any particular challenges to information heart operators from a bodily area perspective. However information facilities might must make adjustments in different areas to accommodate GPUs.
The largest, maybe, is within the realm of energy. As famous above, GPU servers sometimes want extra power. For information heart operators, this implies not simply guaranteeing that amenities can provide sufficient whole energy to maintain GPU servers operating. Investing in sustainable power sources may be a precedence for firms that need to keep away from letting power-hungry GPUs undercut sustainability commitments.
The power to maintain GPU servers cool may even be a precedence for information heart operators. Packing dozens of GPU servers right into a single rack won’t be possible with out refined cooling techniques that may take away warmth very effectively.
There may be additionally the difficulty of information heart catastrophe restoration. Defending information heart infrastructure towards failures, and recovering shortly once they do happen, is vital whatever the forms of servers inside a facility. But it surely’s particularly important when the servers embrace GPUs that value tens of 1000’s of {dollars} apiece.
On high of this, recovering from an outage could also be more difficult when you’ve got GPU servers within the combine as a result of workloads that have been operating on these servers are more durable to carry and shift onto different servers. If an ordinary server goes down, you may sometimes exchange it with every other normal server. This isn’t essentially the case for a GPU server that features specialised {hardware} and particular configurations.
The underside line: Making information facilities into perfect houses for GPU servers is definitely doable. But it surely requires some particular planning and funding. A GPU server might match into the identical rack as an ordinary, CPU-only server, however that doesn’t imply it would work flawlessly with out distinctive provisions in place.
