翻訳待ち：PagedAttention vs. RadixAttention: Optimizing LLM KV Cache Management

AI サービスが一時的に利用できないため、復旧後に翻訳を補完します。

--> KV Cache Management: PagedAttention & RadixAttention India's Most Futuristic AI Conference Is Back – Bigger, Sharper, Bolder d : h : m : s Career GenAI Prompt Engg ChatGPT LLM Langchain RAG AI Agents Machine Learning Deep Learning GenAI Tools LLMOps Python NLP SQL AIML Projects Reading list How to Become a Data Analyst in 2025: A Complete RoadMap A Comprehensive Learning Path to Tableau in 2025 A Comprehensive NLP Learning Path 2025 Learning Path to Become a Data Scientist in 2025 Step-by-Step Roadmap to Become a Data Engineer in 2025 A Comprehensive MLOps Learning Path: 2025 Edition Roadmap to Become an AI Engineer in 2025 A Comprehensive Learning Path to Master Computer Vision in 2025 Best Roadmap to Learn Generative AI in 2025 GenAI Roadmap for Enterprises Large Language Models Demystified: A Beginner’s Roadmap Learning Path to Become a Prompt Engineering Specialist PagedAttention vs. RadixAttention: Optimizing LLM KV Cache Management Janvi Kumari Last Updated : 21 Aug, 2026 12 min read Modern LLMs rely on quantization, pruning, distillation, and faster attention kernels, but production performance often depends most on KV cache management. As context windows grow, the cache consumes significant GPU memory, limiting concurrency, throughput, and latency. Two breakthroughs transformed this challenge: PagedAttention improves memory allocation, while RadixAttention enables efficient prefix reuse. Together, these techniques make LLM serving faster and more memory-efficient. In this article, we examine how PagedAttention and RadixAttention work, why they matter, and how they enable high-performance LLM serving. Table of contents Why the KV Cache Is the Real Bottleneck PagedAttention: Solving the Memory Allocation Problem How PagedAttention Works RadixAttention: Solving the Prefix Reuse Problem How RadixAttention Works PagedAttention vs. RadixAttention: What’s the Difference? Conclusion Frequently Asked Questions Why the KV Cache Is the Real Bottleneck Every transformer generates text one token at a time. For each new token, the model must attend to all previously generated tokens by using their key (K) and value (V) vectors. Recomputing these vectors at every step would make generation prohibitively expensive, so serving engines store them in memory as the KV cache. This cache eliminates redundant computation and makes autoregressive decoding practical, but it introduces a new challenge: memory consumption grows linearly with sequence length. For long-context models, the KV cache often becomes the largest dynamic consumer of GPU memory, determining how many requests can run simultaneously. Why memory becomes the limiting factor The size of the KV cache depends on the model architecture and the number of tokens stored. The per-token memory requirement is: Where: SymbolMeaning LNumber of transformer layers HkvNumber of KV heads DHead dimension BBytes per value (2 for FP16) For a Llama-3 8B class model with 32 layers, 8 KV heads, 128-dimensional heads, and FP16 precision, each token occupies approximately 128 KiB of KV cache. A 100,000-token context therefore requires nearly 12.8 GiB of memory before considering batching or additional requests. The two fundamental problems As GPU memory fills with KV tensors, serving systems encounter two distinct bottlenecks: Memory fragmentation: This occurs when the system allocates KV memory inefficiently, leaving large portions of GPU memory unusable and reducing the number of concurrent requests. Redundant computation: Identical prompt prefixes are repeatedly prefetched and encoded, even though their KV states have already been computed. These problems are independent, and each inspired a different solution. PagedAttention addresses efficient memory allocation, while RadixAttention focuses on reusing previously computed KV cache across requests. Together, they define the foundation of modern LLM serving. PagedAttention: Solving the Memory Allocation Problem By 2023, the industry identified the biggest inefficiency in LLM serving as the storage method of the KV cache rather than attention itself. The system allocated one large contiguous block of GPU memory to hold the entire KV cache for every request. Since the serving engine could not predict how long a response would be, it typically reserved space close to the model’s maximum context length. Most of that memory remained unused throughout the request, drastically reducing the number of sequences that could be served simultaneously. The problem with contiguous allocation Traditional allocation creates two forms of fragmentation: Internal fragmentation: A request reserves thousands of token slots but generates only a small response, leaving most of the allocated memory idle. External fragmentation: As requests of different lengths finish, scattered gaps appear across GPU memory. Although the total free memory may be sufficient, it is no longer available as one contiguous block for new requests. The result is poor GPU utilization and lower throughput, even when plenty of memory technically remains available. How PagedAttention Works The core idea behind PagedAttention is simple: allocate KV memory only when it is needed. Instead, the system divides the KV cache into fixed-size blocks (typically 16 or 32 tokens) rather than reserving one large contiguous buffer for an entire sequence. As generation progresses, new blocks are allocated only after the previous one becomes full, allowing memory to grow incrementally rather than being over-provisioned from the start. Step 1: Divide the KV cache into blocks The system splits each sequence into equal-sized logical blocks, while the system can store the actual blocks anywhere in GPU memory. Step 2: Use a block table for address translation Every request maintains a block table that maps logical block IDs to their physical locations in GPU memory. During attention, the kernel consults this table to gather the required keys and values, making the sequence appear continuous even though its data is physically scattered. Logical blockPhysical GPU block Block 0Memory Block 18 Block 1Memory Block 42 Block 2Memory Block 07 Block 3Memory Block 31 In fact, this indirection draws inspiration from page tables in operating systems: the model operates on a logical sequence, while the serving engine manages physical placement. Step 3: Grow memory on demand Instead of allocating space for thousands of future tokens, PagedAttention expands the KV cache one block at a time. A request generating 60 tokens occupies only the blocks required for those 60 tokens. No memory is reserved for tokens that may never be produced, which dramatically reduces internal fragmentation. Step 4: Share blocks with copy-on-write One of the most powerful features of PagedAttention is block sharing. If multiple requests begin with the same prompt, they reference the same physical KV blocks instead of storing duplicate tensors. When two requests eventually diverge, the system copies the shared block only at the point of modification, a mechanism known as copy-on-write. This makes prefix sharing highly memory-efficient for beam search, parallel sampling, and concurrent requests with identical system prompts. Why this changed LLM serving PagedAttention does not change the attention algorithm or the model’s outputs. Its innovation is purely architectural: it replaces inefficient contiguous allocation with a paged memory layout. The result is dramatically lower memory waste, higher GPU utilization, and the ability to serve many more concurrent requests on the same hardware. RadixAttention: Solving the Prefix Reuse Problem PagedAttention made GPU memory efficient, but it left another major inefficiency untouched: the system still recomputed identical prefixes for every new request. In real production workloads, requests are rarely independent. Thousands of users share the same system prompt, chat conversations repeatedly include their entire history, and agent workflows continuously append to an existing context. Consequently, the system spends much of the expensive prefill phase generating KV tensors that already exist. The authors introduced RadixAttention to eliminate this redundant computation by turning the KV cache into a searchable, reusable index rather than a temporary memory buffer. The key idea: Store prefixes in a radix tree Instead of discarding KV tensors when a request finishes, RadixAttention retains them inside a radix tree a compressed trie where each edge represents a sequence of tokens. The system stores every unique prompt prefix once, while different requests branch only where their tokens begin to differ. For example, three requests may begin with the same system prompt: System: You are a helpful assistant. User: What is AI? System: You are a helpful assistant. User: What is Machine Learning? System: You are a helpful assistant. User: What is Deep Learning? Rather than storing three identical copies of the shared prefix, the radix tree keeps it once and creates separate branches only for the final user query. How prefix matching works When a new request arrives, RadixAttention performs three operations: Match: Find the longest token prefix already present in the radix tree. Reuse: Load the existing KV tensors for that matched prefix instead of recomputing them. Insert: Compute only the unmatched suffix and append it back into the tree for future requests. The longer the shared prefix, the less work the model performs during prefill. This directly reduces Time to First Token (TTFT), especially for long conversations and agentic applications. Why it matters Unlike PagedAttention, which improves memory utilization, RadixAttention improves computational efficiency. It transforms repeated prompts into cache hits, allowing serving engines to skip thousands of identical transformer computations. The benefit is largest in workloads with stable system prompts, multi-turn chat, RAG pipelines, coding assistants, and agent loops where contexts evolve incrementally instead of being rewritten from scratch. How RadixAttention Works Unlike PagedAttention, which organizes memory, RadixAttention organizes knowledge. Its goal answers one question efficiently: How much of this prompt has the system already computed? To do that, it maintains a global radix tree that indexes token sequences and their corresponding KV cache entries. Every new request either reuses an existing prefix or adds only the missing suffix. Step 1: Find the longest matching prefix When a request arrives, the serving engine traverses the radix tree token by token to find the longest prefix that already exists. Instead of comparing entire prompts, it simply follows the matching path through the tree. If 1,900 tokens of a 2,000-token prompt already exist, the model immediately reuses those KV tensors and computes only the remaining 100 tokens. Step 2: Compute only the unmatched suffix Next, once the system identifies the shared prefix, prefill begins exactly where the match ends. The system loads the reusable KV states from cache, while only the new tokens pass through the transformer. This is why RadixAttention primarily improves Time to First Token (TTFT) rather than memory efficiency it eliminates redundant transformer computation. Step 3: Insert the new path into the tree Finally, after prefill (and later during generation), the system inserts the newly computed KV tensors back into the radix tree. Future requests can now reuse this longer prefix, allowing the cache to grow organically as real traffic arrives. Rather than treating completed requests as disposable, RadixAttention turns them into reusable cache entries for subsequent requests. Step 4: Evict unused prefixes intelligently Because GPU memory is finite, the system cannot retain every cached prefix [truncated for AI cost control]