Another Conversation with Val Bercovici Memory Markets

General chatter about memory, agents, and more!

This post originally appeared in Fabricated Knowledge.

“We’re seeing trillion-token consumption by AI startups now as a routine thing, daily sometimes.”

Doug: Welcome back to the Fabricated Knowledge podcast. I have Val from WEKA once again, and we are here to talk about memory. I think from last time to now, it was actually a pretty complete podcast. We were rambling a little bit about KV cache, NAND offloading. And I think from then to now, we’ve really seen quite a different market. NAND prices have exploded. All memory prices have exploded. And I still think it’s the same story. I wanted to update everything, talk to Val, and get some more current thoughts on the whole market. Thanks for being here.

Val: It’s a pleasure. Happy to be a repeat guest.

Doug: Thanks for coming. I think we were talking about this a little bit before — let’s actually talk about the Jensen conversation, because at CES, the big news was — and people took this to the utmost extreme, everyone’s like, “everyone needs 16 terabytes per GPU.” Maybe that’s a little bit of a BlueField advertisement, but let’s just talk about the endorsement of the KV cache market overall.

Val: Absolutely. Like we were saying earlier, it was kind of a confusing topic last summer. The ecosystem wasn’t really ready for KV offloading. Agentic demand hadn’t exploded yet, even though you and I probably anticipated it back then. We’re here now, right? Agents are exploding — even pre-Claude Code. We’re all Claude Code-pilled over the past holidays. We’re seeing trillion-token consumption by AI startups now as a routine thing, daily sometimes. So the demand is there, and it’s cool to see that different vendors had different names for it before Jensen just anointed it as the “context memory” market — and particularly “context memory storage” as we extend storage beyond HBM and DRAM onto NVMe.