Marvell: Inference Models Drive 10x KV Cache Growth, Pushing Optical Interconnect and Memory Expansion to Forefront
At the Six Five Summit 2026, Marvell executives Will Chu, executive vice president of custom cloud solutions, and Dave Lazovsky, executive vice president of the data center networking group, said that the rise of inference models has expanded KV cache demand roughly 10x in the past nine months. Copper interconnect is nearing its performance limits, making optical interconnect and memory expansion key variables for the next phase of compute scaling.
Lazovsky noted that early foundational models established baseline demands for total memory capacity, while inference-time compute has further intensified memory pressure. HBM capacity within a single rack can no longer support frontier models, requiring scale-up networks to grow from 144 XPUs to pods of roughly 576 XPUs, which requires shifting from copper to optical interconnect. Marvell is advancing both near-package optics (NPO) and co-packaged optics (CPO), having deployed CPO on Ethernet and UAL protocol switches.
Chu added that Marvell offers migration paths from traditional copper to NPO and CPO, with customization spanning compute, I/O, and switch ports. On the memory front, Marvell is investing in high-density SRAM, custom HBM, 3D stacking, and CXL-based expansion. The company also introduced its Photonic Fabric Memory device at FMS, which provides a 32TB external memory pool in a separate rack space, combining DDR capacity and cost traits with HBM bandwidth and multi-pseudochannel parallelism. Marvell's CXL products have seen considerable demand, including from customers looking to attach older memory to new servers or use compression for 2x effective capacity.
Lazovsky also highlighted the efficiency of analog SerDes, saying a scale-up network built on analog SerDes can cut power by roughly 4x compared with traditional 2.4T SerDes optical links, potentially reducing overall data center power consumption by more than 25% after full deployment.
