That hole will not be static. Promode Nedungadi, Chief Expertise Officer, mentioned the architectural and algorithmic traits driving AI are making the community drawback tougher, not simpler. Methods like mixture-of-experts fashions and the disaggregation of inference into separate prefill and decode levels all require extra knowledge motion.
“Each a type of requires extra knowledge to be moved round,” he mentioned. “The quantity of knowledge being moved per token is rising.” The dimensions problem additionally has multiple dimension. Perkins described three: scale-up, which refers to interconnecting GPUs inside a single coaching area; scale-out, which covers the broader cluster cloth; and what he referred to as scale-across, an rising requirement that requirements our bodies are starting to deal with.
“We predict that scale-across is sort of attention-grabbing as nicely,” Perkins mentioned.
Structure: silicon, packaging, and software program
A key differentiator for Eridu will come from silicon.
“There’s little doubt that we’re growing our personal silicon. We’re growing essentially the most superior silicon within the networking sector, bar none, interval, and that’s completely needed,” Perkins mentioned. “You don’t get to an order-of-magnitude increased scale utilizing off-the-shelf silicon.”
Eridu has a partnership with TSMC for course of expertise and superior system integration. Perkins mentioned TSMC sees the networking bottleneck as tied on to its personal enterprise. The silicon method is more likely to profit from chiplet-based structure and superior packaging.
“We imagine that you must be on a unique expertise arc than what the mainstream expertise is,” Omar Hassen, Chief Product Officer, informed Community World. “When it comes to issues like superior packaging, you’ve obtained to benefit from every little thing you may from chiplet-based structure, clean-sheet design, and superior packaging. We imagine we’re on the correct expertise arc that may take us past what the prevailing incumbents are doing.”
Essentially, Eridu’s method is an try to interrupt by way of the architectural ceiling going through incumbents to get an order-of-magnitude enchancment.
“It’s important to begin from a unique place, in any other case you may hit the ceiling when you simply use the identical structure as all people else,” Hassen mentioned.
Eridu at a look:
- Based: 2023
- Funding: $200 million.
- Traders: Socratic Companions, John Doerr, Hudson River Buying and selling, Capricorn Funding Group, and Matter Enterprise Companions.
- Headquarters: Saratoga, California
- CEO: Drew Perkins
- What they do: AI networking
