AI 服務暫時不可用，以下為來源正文，待恢復後補全翻譯。
Proudly brought to you by SerpAPI: Today’s release covers a LOT of ground, but you can scrape all this into markdown for your agents! 50% token cost of JSON. We last covered the Inference Inflection in April, and historically most of the advances have been closed source decacorns from Fireworks to Baseten to Modal. Well, time for the open source response. It is day 1 of PyTorch Conf, and DeepSeek timed an intercontinental ballistic missile into the discussion by open sourcing DeepInfer, their production stack, capable of up to 72 hour prompt caching to the 12 hours OpenAI just announced: This is their customization of vLLM with credit to SGLang and TRRTLLM, but there is a lot added… minus their crown jewels - their ultra optimized KV cache (compact + >8hour lifetime), PD/disggregation, and load balancing (their 3FS was open sourced a while ago). We have translated their blogpost and published it here together with a complete research guide and explainers and codemap and subsystem guide: As we point out in their additional disclosure, their claim to serve 155T tokens/day gets them $38 per SM100/A950 card per hour at off peak prices (currently $0.15/$0.60 per million input/output tokens: 155000000 tokens * $0.14 blend cost / 24 hours is ~$1m per hour). Taking their production data (which we translated) of 1.5 million cache-inclusive input tokens/second per prefill GPU, 96% input cache hit, and 14,000 output tokens/second per decode GPU (over ~2400 GPUs): with all-in costs $5-8 per card per hour that is an >80% inference margin! Doing some cluster math, that is roughly $166B per GW, with 81 day payback period on inference margin (DeepSeek famously plans for 10 month payback). Imagine what the Whale could do with more GWs. Sponsored by SerpAPI! Give Your AI Agents Live Search Data with Fewer Tokens Receive results from any of SerpApi’s 100+ APIs as clean, structured Markdown, using roughly 50% fewer tokens than JSON on average. Simply add output=md or use the /search.md endpoint. It requires zero configuration and is included with every plan at no additional cost. AI News for 10/19/2026-10/20/2026. We checked 12 subreddits, 544 Twitters and no further Discords. AINews’ website lets you search all past issues. As a reminder, AINews is now a section of Latent Space. You can opt in/out of email frequencies! AI Twitter Recap Top Story: DeepSeek DeepInfer—Production Inference, Open-Source Boundaries, and the Economics of Cached Tokens What happened DeepSeek released DeepInfer, the inference engine behind V4.1 Flash, alongside production-serving details that make the engineering report as important as the code. The release covers the inference engine and API-processing layer. DeepInfer supports distributed inference, speculative decoding, SWA Bounded Replay, and structured output on NVIDIA and Huawei Ascend platforms. The accompanying deepseek-recipe component implements production token encoding/decoding and official-API-compatible processing. DeepSeek says future model inference implementations and optimizations will also appear in this repository. @deepseek_ai, @deepseek_ai, announcement translation The reported production figures are substantial—but describe a heavily cached service. DeepSeek’s engineering report, as summarized and quoted in the supplied posts, reports 155 trillion tokens/day from one cluster, 96% input-cache hits, 400 ms average time to first token, and 230 tokens/s generation, reaching 300 tokens/s for coding. It also reports $38 revenue per accelerator-hour at off-peak prices. These are first-party production claims, not independently audited measurements. @ZhihuFrontier, @swyx “Fully open source” does not mean the entire production service is reproducible from this release. Although DeepSeek describes DeepInfer as fully open source, the technical account says the KV-cache management and load-balancing components are not yet fully released, because they remain coupled to internal infrastructure. The engine is open; important service-level orchestration remains a deployment responsibility. @deepseek_ai, @ZhihuFrontier vLLM supplied the most concrete comparative response. After acknowledging that DeepInfer was initially built primarily on vLLM and released under Apache 2.0, the vLLM team reported decode throughput within 6% of DeepInfer’s published results on GB200 under a matched configuration. It also published AgentX results and reproduction recipes, materially qualifying claims that DeepSeek’s efficiency is inaccessible to existing open-source engines. @vllm_project, @vllm_project, reproduction resources The release triggered two distinct debates: whether DeepSeek has a uniquely strong serving stack, and whether its disclosures demonstrate high inference profitability more generally. The first now has comparative benchmark evidence; the second remains sensitive to workload mix, cache accounting, hardware cost, and which expenses are included. @Tim_Dettmers, @zephyr_z9, @Yuchenj_UW Architecture: separate stages, schedule compute rather than tokens DeepInfer’s disclosed architecture separates vision encoding, prefill, and decode, allowing each stage to scale against its own bottlenecks. The recommended deployment shapes are ordinary eight-accelerator machines for prefill and 64–128-accelerator supernodes for decode. Decode uses wide expert parallelism to distribute MoE experts while maintaining large effective batches. These are deployment recommendations, not sufficient evidence of the exact hardware composition of DeepSeek’s production cluster. @ZhihuFrontier The scheduling details explain why a conventional “tokens per batch” comparison can miss substantial differences: Prefill scheduling estimates attention-compute cost. Work is allocated using estimated compute rather than raw request length alone. Chunk sizes vary to fit a time budget, instead of enforcing one fixed chunk size across heterogeneous requests. @ZhihuFrontier Prefill routing does not require cache affinity. Because cached prefixes are pooled in shared storage, a request can go to an available worker rather than the worker that previously served its prefix. In a detailed Chinese-language analysis, inference engineer Su Jin Yan highlights this as a particularly useful production design: prefill data-parallel groups balance on “compute tokens,” while shared storage supplies reusable state. The same analysis highlights request bucketing and FIFO scheduling with a runtime estimator. @SuJinYan123 Decode routing balances both running-request count and KV-cache length. Request count approximates scheduling pressure; accumulated KV length captures memory/state pressure that request count alone misses. The engineering commentary describes prefill/decode scaling signals as aggregate compute load for prefill and running requests for decode. @ZhihuFrontier, @SuJinYan123 Expert-parallel load balancing replicates popular experts. Replicas are periodically redistributed to reduce inter-device waiting caused by routing skew. The trade-off is additional HBM consumption against higher throughput, with diminishing returns as replication increases; the useful replication factor is workload-specific. @ZhihuFrontier, @SuJinYan123 CPU scheduling overlaps accelerator execution. Asynchronous scheduling removes idle gaps between decode steps rather than treating host-side orchestration as free. @ZhihuFrontier One technically notable observation is the apparent absence of tensor parallelism in the disclosed Blackwell configuration. A commentator contrasts this with default vLLM/SGLang recipes and traces the approach back to DeepSeek V3’s H800-era interconnect constraints. That is an interpretation of the configuration—not evidence that tensor parallelism is generally inferior. @nrehiew_ Scale also matters. Su Jin Yan argues that wide expert-parallel prefill needs enough traffic and workers to pay off, and favors pipeline parallelism for small-to-medium deployments with only a few prefill workers. The author also notes that layer heterogeneity can complicate pipeline partitioning. This is practitioner judgment, not a benchmark establishing a universal crossover. @SuJinYan123 Cache hierarchy: persistent prefixes and cheap reconstruction of transient state The cache design is the central connection between the throughput figures, agent workloads, and reported revenue per accelerator. Persistent block KV lives in a separate 3FS storage cluster with roughly 100 PiB of SSD capacity. Reported retention is three days for agent traffic and seven days for chatbot traffic. Any prefill worker can retrieve a cached prefix, decoupling cache locality from compute placement and allowing storage and compute to scale independently. @ZhihuFrontier That retention is useful for agents that repeatedly resend a growing history of instructions, repository content, tool outputs, and previous responses. It also means 155T served tokens/day must not be read as 155T newly generated tokens/day: cache hits are part of the service’s token accounting, while newly computed prefill and autoregressive output impose very different costs. The reported 96% input-cache hit rate is therefore essential context for the headline throughput. @ZhihuFrontier, economic workload assumptions The implementation addresses two different state lifecycles: Persistent block cache is loaded asynchronously. Host-to-device transfers overlap layer execution. Su Jin Yan highlights a lower-level implementation detail: fine-grained KV block copies perform poorly through the CUDA copy engine, so the described path uses GDR/RDMA-based transfers, aiming to saturate PCIe bandwidth without occupying SMs. This is a report-derived technical observation, not a separately measured result in the tweet. @SuJinYan123 Sliding-window attention state stays in host memory rather than persistent SSD storage. It is useful mainly within active sessions and has a shorter lifetime than reusable block prefixes. @ZhihuFrontier SWA Bounded Replay reconstructs missing short-lived state from only the latest 128 tokens. Replay restricts attention to that segment while reusing the persistent block cache, producing an approximation rather than rebuilding state across the full preceding history. DeepSeek reports that this nearly halves total prefill computation. The approximation and its model-specific validity matter when considering portability to other architectures. @ZhihuFrontier Su Jin Yan additionally reports that the long retention tail buys only approximately 0.5–1 percentage point of extra cache hits, illustrating a storage-capacity-versus-recomputation trade-off. The same commentary suggests decode-led cache management allows generated output to become reusable prefix state on subsequent turns. @SuJinYan123 The timing invites comparison with OpenAI’s separately announced 12-hour API prompt caching, versus its 30-minute default, including mixed retention durations within a request. These are not identical products or architectures, but both target persistent agents that reuse context across long gaps. Claims elsewhere in the feed about indefinite logged-in caching are commentary, not the terms of the API announcement. @OpenAIDevs, retention controls, comparison commentary Speculation, API compatibility, and model-specific optimization DSpark makes speculative verification adaptive. It predicts draft-token acceptance probabilities and changes verification length to maximize expected throughput. DeepSeek reports a 6% production throughput improvement over always verifying five draft tokens. This is an incremental gain over a specific speculative-decoding baseline, not a 6% gain over non-speculative decoding. @ZhihuFrontier deepseek-recipe addresses the serving interface, not just tokenizer speed. It supports Chat Completions, Responses, and Messages APIs, parallelizes tokenization, and distinguishes real control tokens from [truncated for AI cost control]