AI Tokenomics: What AI tokens cost and where they're wasted

Uh oh!

There was an error while loading. Please reload this page.

Notifications You must be signed in to change notification settings

Fork 4

Star 45

BranchesTags

Open more actions menu

Folders and files

NameName

Last commit message

Last commit date

Latest commit

History

19 Commits

19 Commits

.github

.github

concepts

concepts

practices

practices

research

research

scripts

scripts

setups

setups

.gitignore

.gitignore

CONTRIBUTING.md

CONTRIBUTING.md

LICENSE

LICENSE

LICENSE-CODE

LICENSE-CODE

README.md

README.md

claims.md

claims.md

Repository files navigation

A map of what AI tokens actually cost, and where they're wasted vs. well spent.

"I feel nervous when I have subscription left over. That just means I haven't maximized my token throughput."

Andrej Karpathy, No Priors (2026)

As of 2026-07: 197 entries across five areas, 8 practices, 10 concepts, 8 claims, and 7 copy-paste setups.

Topics: Caching · Compression · Context engineering · Memory · Routing · Multi-agent systems · Gateways · Observability · Benchmarks · Cache accounting · Budgets · Pricing models · Energy

A hybrid by design: a curated awesome list at the core, plus a field-guide layer on top (practices, concepts, claims) that carries an editorial point of view.

Contents

Where to start

Legend

Monitor

Optimize

Govern

Understand

Measure

Practices

Concepts

Claims

Setups and skills

Where to start

Just want the numbers: the five area sections below hold every entry. Want the method: read the practices first, then the concepts behind them. Building something: setups and skills holds runnable configurations.

Legend

Each entry ends with a kind badge: (blue, with the license when known), or a gray badge for , , , for companies, and . Plain entries are articles. GitHub-hosted tools also carry live star and last-commit badges.

Monitor

Dashboards

ccusage - An open-source CLI that reads local agent logs to report token usage and cost across 15 coding-agent sources, with caching-aware pricing.

Claude Code Usage Monitor - A live terminal dashboard for Claude Code usage, with burn-rate analytics, P90 limit detection, and session-expiry forecasts.

claude-usage - A local dashboard for Claude Code token usage, costs, and session history; Pro and Max subscribers get a quota progress bar.

ClaudeBar - A macOS menu-bar app that monitors AI coding quotas across 11 providers; the README declares MIT but ships no license file, so the OSS grant is unconfirmed.

CodeBurn - An open-source tracker for 36 coding tools whose optimize command flags named harness-waste patterns with dollar estimates it later checks against actuals.

Codex Usage Tracker - A local-first dashboard, CLI, and MCP tools indexing Codex CLI logs into SQLite to show where tokens, credits, and cost go, including cache ratios.

CodexBar - A free, open-source macOS menu-bar app that shows limits and reset timers at a glance across dozens of AI providers, plus credit balances and spending when…

CodeZeno Usage Monitor - A Windows-taskbar widget showing real-time Claude Code quota and usage at a glance, without opening a terminal.

Datadog LLM Observability - Cost - Datadog's LLM Observability estimates per-request cost across 800+ models from token counts and public pricing; invoice reconciliation is a separate product.

gh-aw (GitHub Agentic Workflows) - GitHub's agentic-workflows runtime with first-party per-run token and cost metering, plus budget caps that stop a workflow mid-run.

Grafana Cloud GenAI Observability - Grafana Cloud's GenAI Observability ships a prebuilt dashboard for LLM cost, token usage, and latency, built on top of the OpenLIT SDK.

OpenLIT - An open-source (Apache-2.0), OpenTelemetry-native platform with a self-hosted dashboard for LLM cost, token, and latency observability.

OpenUsage - A native Swift macOS menu-bar meter for 10 AI coding subscriptions, showing session and weekly limits, credits, and estimated spend from local credentials.

TokenTracker - A local-first token and cost dashboard for 27 coding tools, with a desktop pet, native widgets, and achievements as a distinct gamified take on usage metering.

Ebpf Kernel Capture

AgentSight - Uses eBPF to watch an AI agent from the kernel boundary, correlating what it said it would do against what it did, at under 3% overhead.

OpenTelemetry eBPF Instrumentation (OBI) - GenAI / MCP - OBI is OpenTelemetry's zero-code eBPF instrumentation (formerly Grafana Beyla) that captures GenAI and MCP traces at the kernel layer with no SDK.

Observability

Langfuse - An open-source platform for tracing, evaluating, and analyzing LLM and agent transcripts, with a prompt-management layer on top.

Otel For Llms

OpenLLMetry - An open-source set of OpenTelemetry-based SDKs and instrumentations, built by Traceloop, for LLM apps.

OpenTelemetry GenAI Semantic Conventions - OpenTelemetry's GenAI Semantic Conventions define the vendor-neutral token, cost, and cache attribute names that OpenLLMetry and Phoenix both converge onto.

Tracing

Arize Phoenix - A source-available (Elastic License 2.0) LLM tracing platform recording per-span token counts and USD cost via OpenTelemetry.

claude-tap - A local trace viewer intercepting API traffic from 14+ coding agents, showing per-request token breakdowns: input, output, cache read, cache creation.

LangSmith - Cost Tracking - LangSmith is LangChain's commercial LLM/agent observability SaaS.

Opik - Comet's open-source (Apache-2.0) LLM observability platform, with per-span USD cost estimated from token usage.

Optimize

Caching

GPTCache - An open-source semantic cache returning a stored LLM response for a paraphrased repeat query via vector search, skipping the paid call.

khazad - A transport-layer semantic cache for LLM APIs on Redis 8 Vector Sets: it intercepts HTTP traffic with zero application code changes and replays cached responses…

LMCache - KV-cache reuse layer with token-level cache-hit accounting (self-host) - LMCache is a self-hosted KV-cache layer beneath vLLM, giving token-level cache-hit observability for teams who own their GPUs, not a hosted bill.

prompt-cache - A Go LLM proxy that adds a three-tier semantic cache: high similarity hits directly, low skips, and a gray zone runs a cheap verification model to guard against…

Redis LangCache - Redis's fully-managed semantic cache: a REST API that returns a stored response when a new query is similar to a past one, so paraphrases skip the paid LLM…

Cheap Local Models

llama.cpp - The foundational open-source (MIT) local LLM inference engine most of the local ecosystem runs on, with an OpenAI-compatible server built in.

Ollama - A runtime for running open-weight models like Qwen, DeepSeek, and GLM-5.1 locally, shifting inference onto hardware you already own.

Compression

Context Mode - MCP server that sandboxes tool output out of the context window - This MCP server sandboxes tool calls and returns only the distilled result, claiming a 98% cut: 315 KB of output down to 5.4 KB.

headroom - context compression for LLM and agent pipelines (quality story dissected) - An Apache-2.0 context-compression tool for LLM/agent pipelines, the category's largest repo, confirmed organic by star-forensics.

LLMLingua - Microsoft's prompt-compression library that uses a small model to drop low-information tokens before a prompt reaches the target LLM.

llmtrim - A local proxy that compresses a coding agent's prompt, tool schemas, and history before forwarding, and can reroute Claude calls to Grok.

Minification of state-in-context agents - the clean waste-vs-capability datapoint - This ICPC 2026 study found that minifying code in a coding agent's context cuts input tokens by 42% but costs 12 percentage points of accuracy.

rtk - CLI proxy that compresses dev-command output (reduction-only headline) - rtk is a single-binary Rust CLI proxy that intercepts and compresses the output of common dev commands before it reaches an LLM coding agent's context window.

TOON (Token-Oriented Object Notation) - TOON is a compact, human-readable, lossless serialization of the JSON data model, designed for LLM input.

Context Engineering

AgentDiet - trajectory reduction ("Reducing Cost of LLM Agents with Trajectory Reduction") - AgentDiet is an inference-time module that strips useless, redundant, and expired information from an agent's trajectory, without hurting performance.

Anthropic vendor-native context management (context editing + memory tool + server-side compaction) - Anthropic's context editing, memory tool, and server-side compaction cut token consumption by a vendor-reported 84% in a 100-turn web-search evaluation.

Claude Code compaction engine - the three-tier mechanism and its cache/correctness failure modes - Claude Code's harness - not the API - decides how to trim a filling context window, and it does this through a three-tier compaction engine.

Codex CLI compaction cost - over-eager compaction as a token-amplification loop - Upgrading Codex CLI from v0.116 to v0.118 made context compaction fire twice as often, doubling or tripling token consumption for identical tasks.

Context Rot - LLM performance degrades as input length grows - This is Chroma's controlled study of how LLM output quality changes as input length grows, holding task difficulty fixed.

ContextBudget - context management as a budget-constrained sequential decision - ContextBudget's BACM method has an agent decide when and how much to compress its history based on remaining context budget, not a fixed rule.

Cursor vendor-native context management - dynamic context discovery + Composer self-summarization - Cursor's dynamic context discovery loads tool schemas and large outputs on demand instead of eagerly, a change the vendor reports cut context usage by 46.9%.

Repomix - Packs an entire repository into a single AI-friendly file, reporting token counts and using Tree-sitter to compress code to signatures only.

RULER - the "real context size" long-context benchmark - NVIDIA's RULER benchmark found that of models claiming 32K+ token context windows, only half actually maintain quality once you fill them to 32K.

Self-Compacting Language Model Agents - This paper introduces SELFCOMPACT: instead of fixed-interval summarization, the model itself decides when and how to compress a growing agent trace.

Serena - An open-source (MIT) MCP toolkit that gives a coding agent IDE-grade semantic code retrieval and editing: 'the IDE for your coding agent'.

Cost Controls

Claude Code spend-governance bundle (v2.1.216-219) - caps tightened, fan-out default loosened - Claude Code's v2.1.216-219 bundle (2026-07-20 to 07-24) hardens spend controls - a concurrent-subagent cap and enforced --max-budget-usd - while simultaneously…

Harness-side runaway-loop cost guardrails (Claude Code + Codex, July 2026) - In mid-July 2026 Claude Code and Codex both shipped first-party guardrails against runaway agent loops within days of each other.

Gateways And Proxies

Bifrost (Maxim AI) - Bifrost is a Go-based AI gateway fronting 1000+ models that measured just 11 microseconds of added latency per request at 5,000 requests per second.

Cloudflare AI Gateway (Spend Limits) - Cloudflare AI Gateway is an edge-native LLM proxy that added dollar-denominated spend limits in June 2026, blocking or rerouting requests once a budget is hit.

Helicone - An open-source (Apache-2.0) LLM proxy that logs every request's cost, latency, and tokens in one line of code; Mintlify acquired it in March 2026.

Kong AI Gateway - The AI layer of Kong's API-gateway platform: a proxy that meters LLM/agent/MCP traffic for billing, showback, and chargeback.

LiteLLM - An open-source gateway fronting 100+ LLM APIs that computes real per-request dollar cost from a live pricing map, with spend limits.

OpenRouter - A unified API gateway fronting 40

[truncated for AI cost control]