GenStorAIGE's AI90 Turns SSDs Into VRAM, Cuts First-Token Latency by 50x

July 20, 2026

There’s a dirty secret in AI inference that nobody talks about: the GPU runs out of memory long before it runs out of compute. The KV Cache — that growing map of everything the model has seen in a conversation — eats VRAM for breakfast. Once it spills over, everything slows to a crawl.

GenStorAIGE (芯展速), a Chinese storage and AI acceleration company, showed up at WAIC 2026 this week with a practical answer. Their AI90 platform treats a fast PCIe Gen5 SSD as an extension of GPU memory, creating a three-tier hierarchy: HBM, DRAM, and SSD. The KV Cache lives where it fits best, and the OS moves data between tiers automatically.

The numbers are hard to ignore. First-token latency drops from seconds to sub-seconds — a 50x improvement. Throughput jumps 5.1x, and VRAM usage falls by 39%. When you wire up eight NVIDIA GeForce RTX 5090 cards with smart P2P multi-GPU interconnect, the whole cluster runs inference 5.8x faster than stock, handling contexts of 128K tokens or more.

The secret sauce is the PT200Z AI SSD that GenStorAIGE built alongside the platform. This isn’t your desktop NVMe drive. It runs on pSLC flash with a PCIe Gen5 interface, rated for 100 drive writes per day — basically indestructible for cache workloads. Sequential reads hit 14.8 GB/s, random reads reach 3.1 million IOPS, and read latency sits at 54 microseconds. Write latency is even tighter at 10 microseconds.

That endurance rating matters. Offloading KV Cache to an SSD means the drive gets hammered with writes — every new token in a conversation writes to the cache. A standard consumer SSD would degrade fast. The PT200Z is built for this specifically.

The approach is pragmatic. HBM is expensive and supply-constrained. DRAM is faster than flash but still limited on GPU boards. SSDs have capacity to spare. GenStorAIGE’s bet is that smart tiering — putting the hot data in HBM, warm data in DRAM, and cold data on flash — is good enough for production inference, especially for long-context models that would otherwise require a rack of H100s.

Whether the industry adopts this kind of hybrid memory architecture depends on how quickly HBM costs come down and how long the longest AI workloads get. But for anyone running LLMs on a budget — or trying to push past the 128K context wall on consumer GPUs — the AI90 answers a real problem without waiting for the next generation of GPU hardware.