待翻译：The Web-Search Latency Your Agent Pays

AI 服务暂时不可用，以下为来源正文，待恢复后补全翻译。

← All posts 2026-08-23 · TelemAI Research TL;DR Cache: Every query cache we could observe expires within an hour, most within 15 minutes. p50 vs p95: A provider that looks fast at the median can be slow at the tail. Serpapi's p95 is 13.6 s (3.5× its median) while ceramic's is 1.05× its median. When your agent fans out and waits for every result, the slowest response sets your latency. Snippet generation: "Snippet" can secretly mean "run an LLM". Exa's LLM-written summary field costs 6.8 s and $0.017 per call versus 1.3 s and $0.007 for its extractive highlights, a 5× latency and 2.4× price cliff for text your downstream model would happily read raw. Ask an agent a genuinely hard question then it doesn't run a search: it runs a deep search campaign. The question gets decomposed, sub-agents fan out, and every sub-task searches on its own. One deep research can run on our own infrastructure issued 148 web-search calls before writing a word of its answer. OpenAI tells Deep Research users up front that a single answer may take 5–30 minutes[1]; Anthropic's research agent parallelizes its sub-agents precisely because doing so cuts research time by up to 90%[2]. Search is the inner loop of all of it. Web-search latency is also one of the worst-understood numbers: the latency a benchmark measures, and the latency an agent pays, are usually two different things. Here are two measurements of the same search API, same request shape, same region, hours apart: 105 ms query seen 90s ago or 3,937 ms query never seen before Both numbers are real. Both are median values over 150 requests, not cherry-picked outliers. The only difference between them is whether the provider had seen that query string in roughly the last half hour. That's a 37× envelope for a single provider. As a result, published search-API latency numbers are so hard to reconcile with what you see in production. Almost every published comparison we could find reports a single latency figure without saying whether its queries were fresh. Within a 37× envelope, a number without that disclosure is not interpretable. This post is about the right-hand number: As we'll show, if you're building agents, the cache essentially never helps you. The cold price list To compare nine providers fairly, we forced them onto one output contract (10 results, each with a text snippet). We then found the cheapest request that achieves it on each API, and issued 150 brand-new, never-before-seen queries[5] per provider, strictly serially, with per-phase timing (the numbers below are time-to-first-byte with connection setup subtracted; one network round-trip still rides inside TTFB, which is why providers closer together than the geography envelope are reported as ties rather than ranked). Same contract, all cold: median TTFB (whiskers = bootstrap 95% CI) p95linear axis Three things: The spread is 12×: 320ms to 3.9s for the same deliverable. This is not a "premium vs budget" spread either; the fastest provider here is not the most expensive. The tail is a second ranking hiding inside the first: serpapi's p95 is 13.6 seconds (3.5× its own median). ceramic's p95 is only 1.05× its median, which is slow but metronomic. If your agent fans out to several providers and waits for all of them, the tail is your latency. With a 9-way fan-out, the p50 you experience is the maximum of nine draws. We measured it at 4.3s, with p95 at 13.6s. Some rankings are not decidable from one vantage point. We measured our network distance to each provider (a TLS 1.3 handshake costs one network round-trip[6]; spread 3.7–89ms). Pairs whose medians differ by less than that envelope, such as you.com vs brave (16ms apart), or ceramic vs tavily (73ms), are tied as far as a single measurement location can say, which is why we have not ranked them. Q: Do these rankings survive real agent queries? Our 150 test queries were synthetic[5]. The Information Retrieval literature is actually encouraging here: LLM-built test collections have been shown to reproduce system rankings faithfully[7] even though synthetic queries differ measurably from human ones in absolute terms[8]. We also replayed the 77 queries the real agent had generated (after a three-hour cool-down) through the same instrument, paired across all nine providers, with 693 calls and zero failures. We also included a batch of agent and synthetic queries fired within the same minutes, so time-of-day can't tip the scale. Rank agreement with the synthetic set: Kendall τ = 0.71[9] by our preregistered rule, 0.90 after also excluding the provider whose latency appears server-state-driven rather than query-driven. And side by side, the two kinds of query are simply indistinguishable: synthetic queries real agent queriessame minutes, alternating Every interval contains, or in brave's case grazes, 1.00. The widest belong to the two highest-variance providers (serpapi's point estimate of 0.79 comes with a 0.60–1.33 interval: noise, not a discount). Your agent always pays the cold-start latency, and here are some more interesting findings: Web-search caches are startlingly short-lived. We warmed each provider's cache with fresh queries, then revisited each query once after a controlled delay. We tested each delay with its own independent query set, because revisiting resets the clock. The full cycle, cold → warm → cold again: cold (first-ever request / after expiry) warm revisit log y-axis · 6 independent queries per point One caveat before the table: each point above plots the median of six revisits. That representation is fair for providers whose cache answers consistently, and misleading for one that doesn't, as tavily's row explains. providercache lifetime (measured) serpapi30–60 min you.com≈ 30 min exa5–15 min brave5–15 min tavilynot measurable from this chart, as its warm hits turned out to be intermittent (only 1–2 of 6 revisits fast, at every delay from 1 to 60 minutes), so a median-of-six always lands on the slow mode and the curve above shows "cold" even though some hits are warm. The mechanism appears different in kind, not in degree: measurements and our reading in the cache-anatomy section. Every cache we could observe dies within an hour. Most die within 15 minutes. Some of the providers include interesting Cache: We took 150 queries from MS MARCO[3], which consists of real Bing user queries that have been public, studied, and re-issued by researchers for years. These long-studied queries seemed to be the most likely to be pre-cached of any queries on the internet. First-touch cache hit rates: serpapi: 0.0%. exa: 1.3%. you.com: 4.7%. Brave showed meaningful residual warmth (29%). It could be that some labs were experimenting or benchmarking MSMARCO there. We can now see why the cold column represents something closer to a real-world operating point, rather than a pessimistic estimate. Caches forget within the hour, fame doesn't keep a query warm, and agents never ask the same thing twice. The price list above, not the number on any pricing page, is what your agent will actually feel. What the wait buys you Latency is only half the invoice. The same "10 results with snippets" contract delivers very different amounts of material. Fast is not automatically good value, and slow is not automatically bad: response size per call (decoded) download time shown where it exceeds 1ms — transfer, not thinking tavily's 1.2s buys ~8KB; seltz's 0.32s buys 56KB; brave's 0.64s buys 52KB. Per kilobyte of material, the spread across providers is wider than the latency spread itself. Three providers (linkup, parallel, tavily) also ignore Accept-Encoding: gzip entirely and ship uncompressed responses. The others compress 2.6–4.5×. When "snippet" secretly means "run an LLM" Exa is the only API where snippets are opt-in, and it offers several kinds. We observed significant variance across them: extractive / retrieval-only generative (LLM in the request path) Extractive highlights arrive in 1.3s at $0.007 per call, which is statistically indistinguishable from requesting no text at all. Asking for the LLM-written summary field costs 6.8s and $0.017: a 5× latency, 2.4× price cliff for running a language model inside your search call, which is material your downstream model would happily read raw. Check what your "snippet" actually is. The cost–latency plane Latency and price turn out to be independent axes, so here they are as one picture — every provider under the same contract, cold median TTFB against price per 1,000 calls. Five of the nine sell at the same $5/1k list price, and at that identical price they differ 7.5× in speed. The efficient frontier is two providers: ceramic (two orders of magnitude cheaper than the field, at mid-pack speed) and seltz (fastest, at the standard price) — everything else is dominated, and the most expensive API is also the slowest. The LLM-summary route from the chart above lands exactly where you would fear: the expensive-and-slow corner. How each price was established — published price pages, live reads of the vendors' billing APIs, or billing the API reports in-band — is itemized under the chart. cold median TTFB · n=145–150/provider p95 whisker — length = tail ratio exa LLM-summary route efficient frontier plan-price range (serpapi: $6–25 by plan) log scale · 2× per gridline Where the prices come from Every price is normalized to per 1,000 calls under this post's 10-results contract, August 2026, no volume discounts. Behind that normalization sit three kinds of number: published price pages, live reads of the vendors' own billing APIs, and billing the API reports inside each response. provider$/1khow we know brave · you.com · seltz$5one shared published rate — all three list $5/1k pay-as-you-go. (you.com adds $1 per crawled page; this post's contract crawls none) linkup$5published rate for standard-depth, results-shaped output — the shape this post uses; answer-shaped output is $6/1k and deep search $50–55/1k parallel$5published rate for its advanced mode — the API's default, and what we measured; turbo mode is $1/1k, and excerpts past 10 results bill extra (not triggered at this post's 10) ceramic$0.05published pay-as-you-go rate: $0.00005 per request — two orders of magnitude below the field exa$7measured, not list: exa reports the cost of every call in the response itself — $0.007/call on the extractive route, $0.017/call ($17/1k) when the LLM-written summary is requested tavily$8the pay-as-you-go rate we measured on ($0.008/credit, one credit per basic search); a $500/mo volume plan brings it to $5/1k serpapi$15monthly plans only — serpapi sells no pay-as-you-go at any price. $15/1k on the $75/5,000 plan we held (verified against its live account API); the entry plan is $25/1k, and sub-$6/1k exists only from $1,475/mo up Cache anatomy: it's not a yes/no question "Does this API cache?" turns out to be four separate questions, and providers differ on every axis independently: Admission: how many hits before it stores? Five providers served a warm response from the second sighting onward. Four providers — ceramic, linkup, parallel, seltz — showed no cache benefit even after ten consecutive hits on the same query. Hit consistency: tavily admitted every test query by the second or third hit (like the others), but its warm hits were intermittent: after a single prior hit, only 1–2 of 6 revisits came back fast, at every delay we tested. Depth (how much faster is a hit?): From 2.4× (brave) to 37× (serpapi, whose cached responses arrive in ~12–105ms. The fastest search responses we measured all experiment long came from the slowest cold provider). Lifetime: 5–60 minutes, as above. Below, we see the same experiment side by side. Six fresh queries per provider, each hammered ten times; first request vs the settled repeat requests: first-ever request (cold) same query, 6th–10th request provideradmit [truncated for AI cost control]