With the growing adoption of artificial intelligence over the past 12 months, data center networking infrastructure is being pushed to its limits, with enterprises and cloud operators increasing capacity and looking at wholesale infrastructure improvements to meet demand.
Each segment of the data center operation is expecting an increase in spending, including the data center switch market, which a recent report by the Dell’Oro Group forecasts will expand by 50% this year.
While most of the growth in the past in the switch market has come from front-end networks, Sameh Boujelbene of the Dell’Oro Group said in a release: “This rapid growth [of AI] necessitates the deployment of thousands or even hundreds of thousands of accelerated nodes. Connecting these accelerated nodes in large clusters requires a data center-scale fabric, known as the AI back-end network, which differs from the traditional front-end network used mostly to connect general-purpose servers.”
This may force data center operators to rewire parts of the data centers to be fully optimized for cloud workloads, potentially increasing the costs of running the data center.
Ethernet or InfiniBand?
A key decision in the data center switch market is Ethernet or InfiniBand, the two most common networking communication standards in data centers. InfiniBand is increasingly being looked at by data center operators due to its very high throughput, low latency, and scalability, all of which make it appealing for high-performance computing (HPC) and AI.
NVIDIA, which manufactures InfiniBand bus adapters and network switches, has been promoting the technical proficiency of the standard for a while now, but some enterprises are still reluctant to adopt it. Ethernet has been the standard for generations, and while InfiniBand may sport better performance in some areas, it is also more costly, complex, and has interoperability problems with infrastructure.
Ethernet is also evolving to meet the demands of AI, with the formation of the Ultra Ethernet Consortium (UEC) hosted by The Linux Foundation with founding members AMD, Broadcom, Cisco, Intel, and others. The UEC aims to improve Ethernet stack architecture for AI and HPC by reducing tail latency through the adoption of packet spraying, in which every communication exchange is simultaneously used. Other upgrades are being investigated by the consortium.
