The High Bandwidth Memory market grew from $7b in 2024 to $34-38b in 2025—a 5x increase in one year. It’s projected to reach $100b by 2030. nVidia’s Vera Rubin platform announced at CES 2026 connects 576 GPUs via NVLink 7.0, achieving a total bandwidth of 1.5 petabytes per second.

1995 paper titled Hitting the Memory Wall: Processor speed was improving 50-80% annually Memory speed was improving only 7-10% annually

The difference between two exponential functions is itself exponential.

Over the past 20 years, compute capability has grown 60,000x, but memory bandwidth has grown only 100x and interconnect bandwidth only 30x.

Total Data Transfer = Speed × Number of Paths × Information per Transfer

1. Go Faster — SerDes To send data faster, we use a circuit called SerDes (Serializer/Deserializer). Multiple slow signals are combined into one fast signal (Serialize), transmitted, and then separated at the receiving end (Deserialize).

Today’s SerDes has reached 224Gbps per lane—4x faster than the 56Gbps of five years ago.

Companies competing in this space: Synopsys, Cadence, Alphawave.

2. Use Multiple Routes — HBM and Parallelization While conventional DDR memory has a 64-bit path (like an 8-lane route), HBM has a 1,024-bit path (a 1,024-lane route). The upcoming HBM4 doubles this again to 2,048 bits.

Foundries like TSMC and Intel are betting their futures on advanced packaging technology.

Recently, this technique has been applied to NAND flash memory, which is slower than DRAM but offers larger storage capacity. Called HBF (High Bandwidth Flash), it applies HBM’s 3D stacking technology to flash, providing 1.6TB/s bandwidth with 512GB capacity. Mass production is expected in 2027.

3. Load More Per Trip — PAM4 The conventional method (NRZ) uses only two voltage levels: 0V and 1V. PAM4 uses four voltage levels: 0V, 0.33V, 0.67V, 1V. Four states can represent 2 bits (00, 01, 10, 11), so you can send twice the data in the same time.

Speed (SerDes): Better processes enable higher speeds. But process miniaturization is largely saturated. Going from 3nm to 2nm, 1.8nm, 1.4nm… is getting increasingly difficult. We’re approaching physical limits.

Paths (HBM, Parallelization): Doubling or tripling pathways has also reached its limits. Transfer Efficiency (PAM): Moving from PAM4 to PAM6 or PAM8 is no easy task.

Alternative 1: Use Light Instead of Copper Light travels through optical fiber at = 67% of the speed of light, while electrical signals in copper travel at 50-64% of light speed.

Bandwidth capacity: Light frequencies (~10¹⁴ Hz) are 100,000 times higher than what copper can practically handle (~10⁹ Hz). Theoretically, fiber can carry 100+ Tbps, while copper maxes out at tens of Gbps. Distance: At 224Gbps, passive copper cable reaches less than 1 meter. Optical fiber can reach 100+ meters. Starting 2026-2027, as CPO (Co-Packaged Optics) technology becomes commercially available, optics will reach right next to chip packages. Market researchers project the CPO market to reach $20 billion by 2036.

Alternative 2: Move Less Data in the First Place Efficient Inference Chips: Cerebras’s Wafer-Scale Engine integrates 44GB of on-chip SRAM in a single massive wafer, so data rarely needs to leave the chip. The result: 21 PB/s memory bandwidth—7,000x that of an H100 GPU.

Alternative 3: Quantum Computers