The 'Father of HBM' Says AI Is Really About Memory — and Most GPUs Are Wasting 70% of Their Time

There’s a growing tension inside the AI industry, and it doesn’t involve chatbots or regulation. It’s about where the real bottleneck is — and one of the people who helped create today’s solution says we’ve been thinking about it wrong.

Kim Jung-ho, a professor at KAIST’s School of Electrical Engineering and widely known as the father of HBM (High Bandwidth Memory), told the Dong-A Ilbo in a recent interview that AI’s core competitive advantage is shifting away from GPUs and toward memory. His argument is blunt: the AI industry has spent billions scaling up GPU clusters, but most of those chips are sitting idle during inference.

The reasoning is straightforward. Every time an AI model generates an output, data must travel from HBM to the GPU, get processed, and then be written back. Even with a million GPUs deployed, the actual computation time never exceeds 10 to 30 percent of the total cycle. The rest is spent waiting on data movement.

During the training era, that inefficiency was secondary — GPU compute was the binding constraint. But inference is a different game. In production, what matters is how much data the system can handle at once and how fast it can move it. Memory bandwidth and capacity, not raw FLOPS, increasingly dictate real-world AI performance.

HBM was designed for exactly this problem. It stacks DRAM vertically and sits close to the GPU, delivering the bandwidth AI workloads need. But Kim argues that even HBM won’t be enough for what comes next.

As AI moves toward multimodal models and agentic AI, systems will need to hold onto far more cold data — video archives, document collections, persistent context, long-term memory stores. That kind of storage demands NAND flash, not DRAM. Kim predicts that HBF (High Bandwidth Flash), which stacks NAND the same way HBM stacks DRAM, will overtake HBM in market demand within a decade.

Looking further out, Kim envisions HBS (High Bandwidth SRAM) as the next frontier. SRAM is roughly 1,000 times faster than DRAM at reading and writing, though it’s far more expensive and density-limited. His concept involves laying SRAM across an entire 12-inch wafer to reach roughly 1,600 GB of capacity, enabling both high speed and massive scale.

Kim paints the future AI computer as a three-dimensional city. HBM stacks serve as the commercial district — fast, transactional memory close to the processor. HBF layers form the residential zone — dense, persistent, and cheap. HBS sits at the top as high-speed cache. Together, they’d stack roughly 100 layers deep into a single 3D composite architecture, feeding data to the GPU in a coordinated pipeline.

Kim has been working on high-bandwidth memory since the beginning. He began collaborating with SK Hynix on HBM1 in 2010 and participated in long-term roadmap planning for HBM4 through HBM8 last year. His current work at KAIST continues to focus on 2.5D and 3D integrated packaging.

The implication is clear: the next AI hardware race won’t be about who can build the biggest GPU. It will be about who can feed it fastest.