Altman-backed Volantis reveals plan to vault the memory wall by baking photonics into AI accelerators

Sam Altman-backed AI chip startup Volantis has a plan to break through the memory wall using light, and this time it's not just an IP play.

Founded in 2022, the San Francisco-based startup is developing an AI accelerator that will use a photonic interposer to dramatically increase memory bandwidth without compromising on capacity.

Volantis believes it can pack upwards of 10 TB of memory at speeds up to 240 TB/s onto a single massive package. This, it claims, should be enough to serve a 20 trillion parameter model at 10,000 tok/s per user.

REG AD

To do this, Volantis is taking a somewhat unusual approach. Rather than developing some novel compute architecture or memory technology, the company is focusing its efforts on the optical interposer. Everything else it plans to license from IP providers.

REG AD

"The differentiation is primarily coming from the optical interposer. So why don't we license absolutely everything we possibly can, even if it's not the highest performance in the world, because we're already biting off a lot of risk with the interposer and the integration," CEO Tapa Ghosh tells El Reg.

The basic idea behind the chip, which Volantis is calling A-1, is that by using optical wave guides rather than electrical interconnects it can dramatically increase the amount of high-bandwidth memory per package.

Because of the speeds at which most AI accelerator memory, like HBM, operate, and the size and number of data pins used, it needs to be as close to the compute as possible to minimize power consumption and maximize signal integrity. As a result the maximum capacity and bandwidth of modern GPUs is limited by the compute die's shoreline. That's the physical perimeter around the chip. For now, that means the cap is right around 432 GB on AMD's MI455X.

Photonic interconnects can extend that much farther. Instead of a few millimeters the memory can be located a few centimeters away, eliminating the shoreline as a bottleneck. While HBM is useful as an example, we'll note Volantis hasn't said what memory tech its chip will use just yet.

Volantis aims to achieve this using optical waveguides integrated into an interposer. Compute and memory designed or supplied by its partners will be packaged on top.

If any of this sounds familiar, Volantis isn't the first to propose using silicon photonic interposers in this manner. As our sister site Next Platform has previously reported, both Lightmatter and Celestial AI have toyed with the idea in the past. In fact, we strongly suspect the idea of HBM-at-a-distance contributed to Marvell's decision to acquire Celestial for $5.5 billion.

However, unlike either Celestial or Lightmatter, Volantis isn't trying to license its tech to other chipmakers. At least not yet. Instead it aims to develop a fully fledged inference chip.

"It is, at least on day one, a way easier problem to demonstrate a solution that ships," Ghosh says.

REG AD

Another differentiator is the technology on which the startup's interposer is based.

Rather than relying on external lasers, Volantis' design uses micro vertical-cavity surface-emitting lasers — micro-VCSELs for short — and means the light source is integrated directly into the interposer.

VCSELs aren't new by any means, but have faced criticism over their reliability. However, Volantis CTO Roy Meade tells us the technology has come a long way in recent years and by giving up some speed, significantly higher reliability can be achieved, a philosophy he describes as wide and efficient.

"People are still constrained to this, what I would call a fat pipe point-to-point mentality, and naturally, if that's the case, you're going to go towards SerDes," he says. "If we were to go back and take a look at the benefits of a wide-and-efficient approach. HBM for example. Taking that same playbook makes a lot of sense."

What Meade is getting at is that interconnects can achieve higher bandwidth in a couple different ways. The first is by increasing the link speed, and the second by using more links. For example an 800 Gbps link can be achieved using four 200 Gbps links or eight 100 Gbps links. In his example of HBM, higher bandwidth is achieved by using an extremely-wide bus but slower data rate per pin relative to other memory technologies. The same concept applies to Volantis' optical interposer-based accelerator.

Power efficiency remains an open question that's dogged many photonics startups, but this may be less of a problem for Volantis because it's not trying to license its first-gen interposer tech. For its first product the company is targeting 1 picojoule per bit per 24 Gbps lane and a total system power of around 20kW.

Volantis still has a ways to go before it tapes out its first production-ready chip, and will need to raise a fair bit more in venture capital to do that — far more than the $88 million it raised this week with the help of Lachy Groom, Abstract Ventures, and others.

Ghosh is well aware of this. He says the funding will enable the company to scale from small-scale optical demonstrators to something more closely resembling the final product. The interposer alone is challenging enough, but then there is the complexity of actually integrating the IP they license.

REG AD

"Aggressively, we should aim for an MVP product in 12 to 18 months as our target," he tells us, noting that's to have a working prototype not volume production. ®