待翻譯：Route the Work, Not Just the Data: GPUs, CPUs, and the Rise of AI-Native Storage

AI 服務暫時不可用，以下為來源正文，待恢復後補全翻譯。

Paul Woll · Triuna Labs Research · August 18, 2026 🧠 Route the Work, Not Just the Data: GPUs, CPUs, and the Rise of AI-Native Storage When we think about Large Language Models, we tend to picture GPUs. That makes sense. Modern generative AI would not exist at its current scale without them. But GPUs also expose one of AI's increasingly important architectural problems: The fastest place to compute is not the cheapest place to keep data. An NVIDIA B200 GPU has 180 GB of HBM3e and can move data through that memory at up to roughly 8 TB/s. At the other end of the hierarchy, Micron began shipping its 245.76 TB 6600 ION SSD in May 2026. One drive can hold more than a thousand times as much data as a single B200's HBM. But flash operates nowhere near HBM's bandwidth or latency. That enormous gap between hundreds of gigabytes of extraordinarily fast memory and hundreds of terabytes of comparatively inexpensive persistent storage creates a fascinating architectural question: What if storage stopped being merely the place where AI data waits for the GPU? What if some AI workloads were processed near the storage itself, while an intelligent storage tier decided what actually needed to reach expensive GPU memory? That may sound futuristic. It actually follows a research path stretching back almost three decades. And LLMs may provide one of the strongest reasons yet to pursue it. Here is the thesis of this article, stated plainly: An LLM request is not one monolithic computation. It is many kinds of work, and only some of it needs a GPU. As model state outgrows GPU memory, the winning architecture will route each operation to the cheapest tier that can perform it (GPU, CPU, or increasingly intelligent storage), and the cost that decides the route is data movement. The next major optimization is not making the GPU faster. It is reducing how much data has to reach it in the first place. Everything that follows is the evidence: what already ships, what research demonstrates, what I measured on my own hardware, and what remains genuinely speculative. HBM wins on bandwidth. Flash wins on capacity. AI architecture increasingly lives in the gap. ⚙️ First: What Is an LLM Actually Doing? An LLM is not searching a giant database for a sentence matching your prompt. At a simplified level, it repeatedly performs enormous amounts of numerical computation. Your text is divided into tokens. Those tokens are converted into numerical vectors and passed through many layers of a neural-network architecture called the Transformer. The Transformer was introduced in the landmark 2017 paper Attention Is All You Need. Two operations are especially important. Attention Attention helps the model determine which previous tokens matter when interpreting the current token. A simplified version creates three representations: Query: What am I looking for? Key: What information do I represent? Value: What information should be passed forward? Queries are compared against keys. Attention scores are calculated. Those scores determine how strongly different values influence the next representation. Feed-Forward Networks Each Transformer layer also contains large learned matrices that transform the token representations. Across billions of parameters, this creates an enormous amount of multiplication and accumulation. Eventually, the model produces a probability distribution over possible next tokens. It selects a token according to the decoding strategy, appends it to the sequence, and performs the process again. And again. And again. That repeated numerical workload helps explain why GPUs became so important. 🧮 Why GPUs Are Better Than CPUs for LLMs CPUs are extraordinary general-purpose processors. They are designed for workloads such as: Operating systems Application logic Branching Databases Networking Scheduling Serial dependencies Irregular computation Many different instruction types A CPU's strength is flexibility. An LLM workload is different. Huge portions of it repeatedly ask something closer to: Can you multiply these enormous arrays of numbers as quickly and in as much parallelism as possible? GPUs were built for parallelism. Modern AI GPUs contain thousands of execution units plus specialized Tensor Cores designed for matrix operations using formats such as FP16, BF16, FP8 and increasingly lower-precision representations. They also sit beside extraordinarily fast High Bandwidth Memory. NVIDIA lists a B200 at up to roughly 8 TB/s of HBM bandwidth per GPU. A high-performance PCIe Gen5 SSD such as Micron's 9550 reaches roughly 14 GB/s of sequential read bandwidth. Those are completely different performance classes. There is no plausible architecture in which NAND flash simply becomes a drop-in substitute for GPU HBM. But that is not the interesting question. The interesting question is: How much data could we prevent from needing to cross that boundary at all? The goal is not to make SSDs behave like GPUs. The goal is to stop sending the GPU work and data it does not need. 🧠 LLM Inference Has a Memory Problem Inference generally contains two broad phases. Prefill When you initially submit a prompt, many prompt tokens can be processed in parallel. This stage can be highly compute-intensive and maps well to GPUs. Decode Then generation begins. The model produces one token. Then another. Then another. Each new token depends on information derived from what came before. This autoregressive process means inference repeatedly accesses enormous model weights and an expanding amount of context state. Depending on model architecture, batch size, hardware and workload, the bottleneck can therefore shift away from raw arithmetic throughput toward memory bandwidth and data movement. This is one reason FlashAttention became such an important contribution. FlashAttention does not make multiplication fundamentally faster. It reorganizes attention specifically to reduce expensive movement between GPU HBM and faster on-chip SRAM. Its authors explicitly frame attention optimization as an I/O-aware problem. That is an important lesson: Even inside a GPU, moving data can become as important as computing it. Now expand that problem outside the GPU. 🗃️ The KV Cache: LLM Working Memory During attention, Transformers generate key and value tensors representing prior tokens. Without retaining those tensors, the system would repeatedly recompute previous attention state every time another token was generated. Instead, inference engines store them in a Key-Value cache, usually shortened to KV cache. That dramatically reduces redundant computation. But the cache grows with: Context length Number of model layers Model architecture Number of simultaneous requests Persistent agent histories Long-running reasoning Repeated document interactions The scale becomes surprisingly large. NVIDIA has illustrated an example in which a 128K-token context for Llama 3 70B consumes roughly 40 GB of KV-cache memory for a single user at batch size 1. Forty gigabytes is not the model. It is just the cached attention state associated with one long-context request in that example. Multiply long contexts across hundreds or thousands of concurrent requests, persistent agents, document workflows or reasoning processes and the problem becomes obvious: HBM is incredibly fast, but it is scarce. 🪜 The Emerging AI Memory Hierarchy Increasingly, AI infrastructure has to treat memory as a hierarchy: GPU SRAM / cache ↓ GPU HBM ↓ CPU DRAM ↓ Local NVMe SSD ↓ Remote / network storage Each step generally offers more capacity. Each step generally sacrifices latency and bandwidth. Modern inference software is beginning to explicitly manage that hierarchy. NVIDIA Dynamo, for example, supports KV-cache offloading beyond GPU memory. Its architecture can spill KV-cache blocks into CPU memory or local storage, allowing larger contexts and reuse of previously computed prefixes. NVIDIA's FlexKV work extends this concept across tiers including GPU, CPU and SSD-backed storage. So one part of this article is no longer speculative: SSDs are already becoming part of the LLM inference memory hierarchy. The more interesting question is what happens next. 💽 The External-SSD Question Every local-AI forum gets this question weekly: "can I just run the model from an external SSD?" It is the right question asked one layer too early, and the answer exposes exactly what today's architecture is. Ordinarily: the SSD stores the model. The runtime loads the weights into memory, computation runs from memory, and storage is touched again mainly for persistence. Unless the runtime explicitly supports weight streaming or offload, the drive affects load time and nothing else. If the model already fits in RAM, a faster external SSD does not make inference better. The storage is a container, not a participant. The current strategy for models that do not fit is compression. A widely shared writeup this week walks through squeezing Qwen 3.8-27B, a 55.6 GB vision-language model, down to 11.55 GB so it runs entirely inside a 16 GB Mac mini at reading pace. It is impressive work, and its author is honest about the limit: the compressed build still cannot reliably drive a serious coding agent. That admission is the interesting part. Fitting the model is not the same as fitting the workload. An agent needs more than resident weights: persistent context, retrieval indexes, tool state, checkpoints, a KV cache that grows with every step. Compression shrinks the weights and does nothing about everything else that increasingly capable systems drag along with them. Three different levers are in play, and they are worth keeping distinct: Model compression reduces the size of the weights. Memory-hierarchy optimization decides what is resident, and when. AI-native storage, the subject of this article, would decide what should move, what stays cached, what gets transformed near the data, and what never needs to reach the GPU at all. The future is not merely running AI from an SSD. It is giving the storage system enough intelligence to participate in running AI. 🕰️ The Idea Is Old. The Workload Is New. Computing near stored data is not a new idea. Researchers were publishing Active Disk architectures in 1998, proposing drives with embedded processors so that data-intensive work could happen where the data already lived, and the motivation they wrote down reads like it was drafted yesterday: why continuously move enormous datasets to a central processor when some of the work can happen where the data resides? Three things have happened since. Flash replaced spinning disks, and a modern SSD is already a small computer: controllers, firmware, parallel NAND channels, error correction, address translation. Adding an FPGA made it a programmable one, and Samsung shipped exactly that, twice, as the SmartSSD, marketed for compression, filtering, search and transformation. And the Storage Networking Industry Association (SNIA) gave the field an architectural vocabulary: computational storage, with defined APIs and interoperability work. (A thorough tour is Past, Present and Future of Computational Storage: A Survey.) So the natural question is: if the idea is twenty-five years old and the hardware shipped, why is it not everywhere? Because for twenty-five years the dominant workloads did not reward it enough. General-purpose queries touch data unpredictably. The win from pushing a filter into a drive was real but modest, and the software cost of programming storage was not. What changed is the workload. Two numbers from later in this article make the point as a pair: a measured retrieval query needed 0.8% of a 102.4 GB corpus, and an independent out-of-core implementation of Kimi K3 activates under 4% of its 2.78 trillion parameters per token. LLM state is en [truncated for AI cost control]