VAST Data is formalizing a broad partnership with AMD, committing its AI Operating System to run on AMD EPYC CPUs and Instinct GPUs across training, inference, and agentic workloads. The headline technical move is adopting 6th-gen EPYC processors for VAST’s storage hardware platforms, gaining PCIe Gen 6 bandwidth that doubles I\/O throughput versus the prior generation. Early testing on an AMD Instinct MI355X showed a 9x speedup in time-to-first-token and 9.7x more token throughput when offloading KV cache to VAST’s storage layer<\/a> over NFS\/RDMA, compared to local host RAM.<\/p>\n If your AI infrastructure is currently Nvidia-only, this announcement is less about AMD winning and more about VAST signaling that it intends to be the data layer for whatever GPU fleet you run. Organizations already evaluating AMD Instinct as a cost alternative to Nvidia H100 or B200 clusters now have a credible, named-customer answer to the software integration question. Crusoe, Vultr, and Core42 are already running AMD-powered services on VAST, which means the reference architecture is not purely theoretical.<\/p>\n The KV cache story is the part worth examining closely. KV cache, the temporary memory store that lets a model recall earlier context in a long conversation or agentic task, has historically lived in GPU HBM or host RAM because latency requirements were too tight for network storage. VAST’s claim is that NFS over RDMA, combined with AMD’s Pensando Pollara 400 NIC, has closed that gap enough to make remote storage viable. If true, it shifts the economics of large-scale inference materially: GPU HBM is expensive and finite, while NVMe-backed storage is cheap and elastic. The 9x TTFT speedup is real but framed relative to a specific baseline, and Blocks and Files, whose editorial model depends on storage vendors remaining strategically relevant, has an interest in the storage-as-inference-tier thesis landing well. That doesn’t make the benchmark wrong, but it does mean your team should validate the comparison conditions before treating 9x as a procurement input.<\/p>\n The automated KV cache lifecycle management feature, which ties cache expiration to data governance policies, is the detail most CTOs will under-weight and most CISOs will eventually demand. Agentic workloads cache sensitive user context by design, and regulatory pressure on AI data retention is arriving faster than most infrastructure teams have planned for. Whether you are on AMD or Nvidia today, the vendor that owns your data layer will own your compliance posture for inference. That is the renewal conversation worth preparing for now.<\/p>\n A KV cache stores the intermediate computations a language model generates while processing a long prompt or a multi-turn conversation, so the model does not have to recompute that context with every new token it generates. Think of it as the model’s working memory for a single conversation. Normally it lives on the GPU itself. Offloading moves it to external storage over a fast network, freeing GPU memory for more concurrent users, at the cost of retrieval latency. The business case is more users per GPU dollar.<\/p>\nWhat this means for your business<\/h2>\n
Concept deep-dive: KV Cache Offloading<\/h2>\n