AI サービスが一時的に利用できないため、復旧後に翻訳を補完します。
What’s Actually Inside 24,723 Tokens of a Search Result? We Broke It Down, Field by Field - MachineLearningMastery.com What’s Actually Inside 24,723 Tokens of a Search Result? We Broke It Down, Field by Field - MachineLearningMastery.com Sponsored Content It’s no secret that AI agents burn massive amounts of tokens on search results and file retrievals. They pull in dozens of full-length files, logs, and comment blocks, and just reading through those matches can consume tens of thousands of tokens, not to mention the recursive loops that lock in when the agent reruns a query. Just a simple search like “look up coffee shops” can balloon into a payload of nested objects, tracking links and metadata. Much of this data never even gets used by the model. You still wind up paying for each token. However, there are ways to minimize token usage. SerpApi now offers Markdown output that supports a radical trimming of token sizes. Using conventional JavaScript Object Notation (JSON), a search for “coffee” resulted in a response that required 24,723 tokens. Using Markdown, the same search only used 6,435 tokens. That’s a 74 percent reduction and there is more to be gained, as further filtering can require just 1,298 tokens for a more restricted view. That reduction can make the AI model more efficient, reduce costs, or allow other information to stay in the context window. JSON vs. Markdown But how does it work? SerpApi’s Markdown output retains the informational content but strips or reshapes the parts that inflate token count without helping a model reason. Based on SerpApi’s documentation and examples, here’s what typically gets removed or transformed: Tracking links and internal SerpApi URLs: JSON responses often include serpapi_link, serpapi_amazon_link, and similar fields used for refining queries or performing follow-up searches inside SerpApi’s own system. Markdown output removes this internal routing noise and keeps only the user-facing links. Icons and thumbnails: Many result objects carry icon, thumbnail, or image URLs that are useful for user interfaces, but add tokens without aiding an LLM’s understanding. Markdown eliminates these by default. Nested metadata and duplicate fields: JSON preserves the data’s full structure, including pagination objects, search metadata, per-result additional arrays, and repeated fields across sections. Markdown flattens this into tables, YAML frontmatter, and inline links. Title/link separation: In JSON, the title and link are separate keys, often accompanied by displayed_link, snippet, and other variants. Markdown merges these into a single, human-readable line with a native link, reducing key names and structural overhead. SerpApi has not published an exact token cost per field; the before-and-after totals (from 24,723 to 6,435) are authoritative numbers. The pattern, however, is clear: anything that exists for code to parse or a UI to render, but not for a model to read, is prime token waste. Where Markdown Isn’t the Right Fit Markdown output is optimized for LLMs and AI agents, but not for code that needs exact data types. If your pipeline depends on precise numeric prices, coordinates, ratings as floats, or structured arrays for downstream processing, JSON remains the better choice. It relies on the same data, and the same credits. It just provides a different shape depending on who’s reading it. For example, a Naver Shopping result in JSON exposes price, old_price, coupon_price as integers and rating as a float, along with reviews and stores fields. That’s ideal for a pricing engine or analytics job. For an agent that only needs to summarize top products, those typed fields are overkill; Markdown’s table rows and inline links carry the signal with far fewer tokens. How SerpApi Solves It SerpApi addresses this directly with a single switch, available through a query parameter, route extension, or header. You can request Markdown output by: Adding output=md to your existing search call, or Calling the /search.md endpoint instead of /search, or Setting the Accept: text/markdown request header. This is available across all 100+ SerpApi APIs at no extra cost, and it returns YAML frontmatter, structured tables, and native inline links while automatically stripping internal tracking noise and duplicate fields. If you also need tighter control, SerpApi’s json_restrictor (or --fields in the CLI) lets you select only specific keys server-side, reducing payload size before it crosses the network. Check It on Your Own Data You don’t have to take benchmark numbers on faith, by the way. Just try one query you already run, add output=md, and count the tokens before and after on your own response. Then try a restricted field set via json_restrictor and measure again. The delta is your real-world savings, specific to your engine and workflow. Is It Time for Markdown? Markdown isn’t a panacea. But it is true that the shape of your data has a token cost most people never look at, and it’s worth checking wherever data feeds into a model, and not just with SerpApi. In agentic systems, small per-call bloat compounds fast and measuring and reshaping your payloads is one of the better moves you can make. If you are interested in seeing what Markdown can do and comparing it to JSON, have a look at SerpApi’s new feature here. Partners