The boldest memory idea at Hot Chips 2026 came from a company that does not make memory. d-Matrix presented Raptor, which it calls the first 3D DRAM accelerator for generative inference, built by bonding a TSMC 4nm compute die face to face onto a custom DRAM die at a 36 micron pitch, as reported by Tom's Hardware from the session. The result the startup claims: 100 terabytes per second of bandwidth from 32 gigabytes of memory per card.
Picture the usual arrangement inverted. Conventional stacked memory puts DRAM layers above a base die and parks the whole tower beside the processor; Raptor puts the logic die on top and the memory directly underneath, so the cooling plate touches the hot compute silicon and the DRAM below doubles as the interposer, the wiring layer that carries signals down and out. Co-founder and CTO Sudeep Bhoja explained the payoff at the session. "For the same amount of power, you can drive the bandwidth up," Bhoja said, "and so we were able to drive the bandwidth up here to 100 terabytes per second."
Numbers from working silicon give the claim some spine. Bhoja put the vertical interface's energy cost at 0.37 picojoules per bit against roughly 2.4 for moving data into an HBM4 base die, per the report, and an accompanying ISCA 2026 paper written with the University of British Columbia projects about 4.7 times the throughput per card of HBM based designs. A picojoule, for scale, is to a joule what a single second is to about 32,000 years, and in AI hardware those trillionths add up to the electric bill.
Engineering this close together forced clever compromises. The DRAM runs at a junction temperature where data retention collapses from 32 milliseconds to 4, so the memory refreshes eight times more often, a penalty d-Matrix absorbed by shrinking each microbank so a full refresh sweep costs just 1.37 percent of bandwidth, per the report. Around 9 percent of the die's 840 banks per chiplet are spares that can be switched in for failures.
The unanswered question is the interesting one: who fabricates the custom DRAM die. TSMC makes the logic and Alchip handles packaging, but neither runs a DRAM fab, and the report notes d-Matrix has never named its memory maker across the talk, the paper and every announcement since its test silicon arrived. Only three companies produce leading edge DRAM at volume, and their capacity is committed to sold out HBM lines.
Timing cuts both ways. The company's CEO has pointed to a 2027 launch, and every performance figure so far, including a projected 988 tokens per second on a 2.8 trillion parameter model, is a d-Matrix projection from early silicon. Launching a novel memory architecture into the tightest DRAM market in over a decade is either terrible luck or perfect timing, depending on whether the shortage makes buyers desperate enough to try something new.
Leave a Reply