Simply because it appeared that the tech {hardware} provide chain was recovering from COVID-induced shortages, a brand new bottleneck has sprung up that might doubtlessly affect the provision of information middle GPUs and the enlargement plans of information middle builders.
In Could, South Korean reminiscence producer SK Hynix introduced that its provide of high-bandwidth reminiscence (HBM) chips was bought out for 2024 and most of 2025. One among its opponents within the subject, Micron, had issued an identical assertion in March. Samsung, the final participant within the HBM market, has made no public touch upon product availability.
Amid the surge in AI and digital companies, the HBM scarcity poses a brand new problem for information middle enlargement. Whereas a lot focus has been positioned on GPU bottlenecks, rising demand for high-bandwidth reminiscence may additional affect the business’s development plans.
What Are HBM Chips and Why Are They Wanted in Knowledge Facilities?
Excessive-bandwidth reminiscence is used within the precise GPU package deal itself, with the chips bodily sitting subsequent to the GPU silicon, versus customary DRAM which is mounted on DIMM sticks and sits subsequent to the CPU.
The HBM design offers a lot larger pace and decreased latency and is essential to the efficiency of AI processing.
In the course of the COVID-19 provide chain shortages, carmakers couldn’t acquire chips for his or her automobiles, in order that they merely constructed vehicles with out the chips and mothballed them till they might shore up stock.
GPU distributors like Nvidia and AMD don’t have that choice. No high-bandwidth reminiscence means GPUs can’t be assembled as a result of the HBM needs to be added to the GPU package deal on the manufacturing stage.
It’s clearly a delicate topic. When approached by DCN, information middle {hardware} distributors Hynix, Micron, Samsung, Nvidia, Intel, and AMD all declined to touch upon the difficulty.
Schematic of a graphics card that makes use of high-bandwidth reminiscence (Picture: Shmuel Csaba Otto Traian, Wikimedia Commons)
TrendForce predicts HBM’s share of the general reminiscence market will nearly double in 2024 – from 2% in 2023 to five% this 12 months. Trying additional forward, HBM’s market share is anticipated to surpass 10% of the general reminiscence market by 2025.
By way of market worth, high-bandwidth reminiscence is projected to account for greater than 20% of the overall DRAM market worth beginning in 2024, doubtlessly exceeding 30% by 2025.
Knowledge Middle Development Continues, however HBM Scarcity Might Pose Challenges
HBM reminiscence is costlier to make, harder to make, and takes longer to make than customary DRAM. So, it’s not like reminiscence makers may pivot on a dime and swap to rising their HBM manufacturing. Such fabrication vegetation, like a CPU fab, take time to construct.
A scarcity of information middle merchandise may affect the business’s enlargement and development plans, however the provide chain is at present holding out. Knowledge middle development marches on regardless of some corporations having to attend for GPU {hardware}, notes Alan Howard, principal analyst for colocation and information middle development with Omdia.
Demand actually isn’t slowing. Omdia tasks that for 2024 for the 100 corporations it tracks there are 37.7 million sq.ft and 6 GW of deliberate capability estimated to come back on-line globally.
“A GPU scarcity is just not more likely to have a dramatic affect on information middle development plans within the foreseeable future,” Howard informed DCN. “The one factor which may put a dent in information middle development, with regard to compute or AI {hardware}, can be a dramatic and long-term provide chain nightmare. Not going, however like a pandemic, not not possible.”
And if a big GPU scarcity does occur? “It’ll be deal metropolis,” mentioned Jon Peddie, president of Jon Peddie Analysis, a tech {hardware} analysis agency. “Whichever a kind of corporations is keen to pay a premium to get on the head of the checklist, then they’ll get the primary shipments, after which these sorts of offers are provided on a regular basis.”
Survival of the Greatest?
The actual downside is that this mannequin doesn’t permit for any new entrants into the market within the occasion of an HBM bottleneck, mentioned Anshel Sag, principal analyst with Moor Insights and Technique.
“Nvidia goes to get the lion’s share of the HBM, however AMD and others have possible already put their orders in for some time,” Sag defined. “So, in the event you’re making an attempt to launch one thing that makes use of HBM, and also you haven’t already negotiated your provide, you’re most likely not getting any.”
It additionally impacts smaller gamers like SambaNova, which makes devoted AI processing servers utilizing their very own customized silicon and non-GPU components. Sag factors out that AMD’s Versal line of FPGA processors additionally makes use of HBM reminiscence, and this may occasionally additionally undergo from additional shortages.
Peddie says there’s already a backlog of GPUs on the order books. He expects Nvidia to ship 800,000 GPUs in Q2, however it may most likely promote extra. “They most likely can’t meet the rise in demand, however they are going to meet 80% or extra of the demand,” he mentioned. “It’s just a bit little bit of discomfort. You already know, it’s like, ‘Gee, I don’t get dessert tonight, however I had a terrific dinner’.”
Sag says the one means he may see reminiscence makers increasing capability aggressively can be if a vendor like Nvidia requested a 3rd get together to construct up capability. “Different corporations… have achieved stuff like that with foundries. Corporations like Apple and Qualcomm have offered instruments and capital to foundries to speed up their deployment of sure applied sciences. So, there’s a precedent for chip distributors to offer incentives to foundries to develop capability or enhance their yields,” he mentioned.
Past the HBM Scarcity: Broader Growth Challenges
Peddie predicts information facilities at present beneath development will get constructed out, however going ahead, GPU and reminiscence provide points would be the least of their issues as a result of information middle operators and builders are fighting different issues. This contains acquiring actual property, enough energy and cooling, and all the opposite elements that go into constructing an information middle. So, the GPU provide concern could also be moot.
“The put in base will get to a sure level, it’ll begin to strategy demand after which prospects will shrug it off as a result of they are going to say we do not want any extra boards proper now we’ve bought sufficient,” mentioned Peddie. “And never solely do we now have sufficient, we now have no place to place new ones.”
