AI サービスが一時的に利用できないため、復旧後に翻訳を補完します。
Introducing Baseten Router: a Pareto-optimal model router for coding tasks Please rotate your device To continue exploring this website, please rotate your device to a vertical orientation Product Introducing Baseten Router: a Pareto-optimal model router for coding tasks Baseten Router picks the right model per coding task, within 0.8 pts of Opus 5.5’s pass rate at 72% lower cost. Now in public preview on Baseten Grid. Authors Date November 4, 2026 TLDR Baseten Router sends each coding task to the right model, open or closed, so you pay frontier prices only when the task needs it. In benchmarks, it came within 0.8 percentage points of Opus 5.5 at 72% lower cost. It’s the only router we tested that beat random routing by a statistically significant margin. Baseten Router is part of Baseten Grid, now in public preview. Today, we’re sharing our initial results from training an intelligent model router for Baseten Grid. Not every coding task needs the most capable model. As AI coding spend continues to grow rapidly [1] and cheaper open-weight models close the capability gap [2], choosing the right model for each task becomes increasingly valuable. Baseten Router chooses between models based on their expected performance and cost for each coding task. The goal is frontier performance while paying frontier prices only when the task calls for it. Baseten Router cuts costs by 72% compared to using Opus 5.5 for every task while remaining within 0.8 percentage points of its pass rate, measured on a test set spanning DeepSWE, SWE Atlas QnA, and Terminal-Bench. Adjusting the routing threshold lets users choose various trade-offs along the cost-performance Pareto frontier. Among the commercial routers available - including those from Fireworks [3], Together AI [4], and Applied Compute [5] - Baseten Router is the only router we evaluated that beats random routing on these benchmarks at the same cost by a statistically significant margin. Every Baseten point sits above the band where 90% of random routers fall and ahead of the other commercial routers (OpenRouter, FireRouter, Together, Applied Compute), so you get close to Opus-level accuracy at about a quarter of the cost. We’re working closely with design partners to specialize Baseten Router for their production coding workloads and prove out performance in the real world. Early feedback is promising, and we will share those results soon. Almost everyone at Opendoor uses AI, and most people don’t want an opinion on which model to pick. Auto routing sends them to the right model automatically, so they stay fully empowered and we stop paying frontier prices for work that doesn’t need it. — Chintan Parikh, Head of Engineering, Opendoor This post walks through how we built competitive performance into Baseten Router and how we continue to refine our approach. How we built Baseten Router Formulating the problem From the input prompt, Baseten Router decides whether the coding task should be completed by a frontier closed-weight model or a cheaper open-weight model. In these experiments, we route between Opus 5.5 and GLM 5.3 Flash. We chose GLM 5.3 Flash because it is a strong coding model that is approximately 80% cheaper than Opus 5.5 across relevant workloads. Our approach can be extended to any model combination in any harness supported by Baseten Grid. Our aim is to build a model router that decides between two models and gives a better trade-off between success rate and cost than either model alone. We formulated routing as choosing the model with the highest score, where the score is the predicted success rate minus a penalty proportional to the predicted cost: πλ(x)=argmaxm∈{Opus,GLM} p^m(x)−λ c^m(x)\pi_\lambda(x)=\arg\max_{m\in\{\text{Opus,GLM}\}}\ \hat p_m(x)-\lambda\,\hat c_m(x)πλ(x)=argm∈{Opus,GLM}max p^m(x)−λc^m(x) For a given coding task xxx, p^m\hat p_mp^m and c^m\hat c_mc^m are learned predictions of model m’s success rate and cost in USD on the task, respectively, and λ\lambdaλ is a configurable weight that controls the trade-off. Intuitively, this favors Opus when its higher predicted success justifies its additional cost. Illustrative example of Baseten Router, which currently routes between Opus 5.5 and GLM 5.3 Flash. Why start with coding benchmarks There are two distinct ways to evaluate a router: on coding benchmarks and in production. The router’s performance in production matters most to us and to customers: how much model routing reduces a team’s AI coding spend while preserving the quality of their engineering work. However, it is challenging to compare results from online evaluations across routers. These tests involve different users and tasks, different definitions of task success, and different methods for calculating cost savings. Coding benchmarks provide a controlled and transparent framework, enabling us to compare different routers on shared tasks with clear success criteria and cost calculations. Additionally, we believe that the expertise needed to develop a good router on benchmarks is a subset of the expertise needed to perform well in production. Consequently, we view strong performance on coding benchmarks as an important milestone toward deploying in production. By being methodical about data quality, quantifying noise, and comparing against meaningful baselines, we have built confidence in Baseten Router’s capabilities as we bring it to our customers’ production workloads. Building the training and test sets To train Baseten Router, we generated traces on verifiable coding tasks. We built the training set from public coding datasets spanning bug fixes, feature implementation, codebase question answering, and terminal tasks. For each task, we ran both Opus 5.5 and GLM 5.3 Flash at high reasoning effort and recorded the verifier output and model cost. Baseten Router’s training data mix, broken down by category and benchmark. To evaluate generalization beyond the training set, we used three benchmarks spanning long-horizon coding tasks (DeepSWE 1.1), codebase Q&A (SWE Atlas QnA), and terminal tasks (Terminal-Bench 2.1) as our test set. We selected these benchmarks because they cover categories highly relevant to production coding work and were included in Artificial Analysis's Coding Agent Index (v1.4) [6] and OpenRouter's Router Benchmark (initial release) [7]. An underrated aspect of this process was cleaning the datasets. We corrected grader errors and broken task environments, added safeguards to prevent cheating, and removed training tasks with repositories present in the test set. Additionally, due to the non-deterministic nature of agentic coding, we quantified the uncertainty between repeats of the same task and repeated evaluation runs to improve the robustness of our results. We discuss this further below. Controlling for noise Benchmark results are noisy, meaning a router may perform well due to chance rather than genuine capability. To more rigorously control for this, we designed our evaluation to account for two sources of noise: Label noise: The same model can pass the same task during one run and fail it during another. Selection noise: A random router can route tasks, by chance, to GLM where it succeeds, making it seem more effective than it actually is. Label noise Models in production are non-deterministic, meaning the same model can pass a task on one run and fail it on the next. Consequently, a single run is an uncertain measure of how hard a task is for that model. To quantify this uncertainty, we calculate a signal-to-noise ratio (SNR). The signal, τ2\tau^2τ2, measures how much the true gap between Opus’s and GLM’s pass rates varies from task to task. The noise, σO2\sigma^2_O σO2 and σG2\sigma^2_GσG2, measures how much each model’s results vary between attempts at the same task. Averaging kOk_OkO Opus runs and kGk_GkG GLM runs per task shrinks the noise according to the formula: SNR=τ2σO2/kO+σG2/kG.\text{SNR} = \frac{\tau^2}{\sigma^2_O/k_O + \sigma^2_G/k_G}.SNR=σO2/kO+σG2/kGτ2. The higher the SNR, the more precisely we can measure a router’s performance. With one run per model, the pooled test SNR was 0.27, meaning the test set had almost four times as much noise as signal. By running each test task four times with Opus and eight times with GLM, we raised the SNR to 1.51. This higher SNR makes it easier to distinguish a router that makes better decisions from one that is lucky. Every test set pass rate mentioned in this blog post is an average over those runs. Selection noise To demonstrate that a router has actually learned useful signals, we must show that it beats random routing by more than chance. A random router is defined as sending a random subset of tasks to GLM and the rest to Opus. Sending more tasks to GLM lowers both expected cost and expected pass rate, so on average every random router lands somewhere between always using Opus and always using GLM. The figure below plots pass rate against average cost per task. Always using Opus 5.5 (top right) and always using GLM 5.3 Flash (bottom left) are the two ends of random routing. The dotted line between them is the average random router at each cost. For each fraction of tasks sent to GLM, we simulate 10,000 random routers, each choosing a different random subset. The gray points are a sample of these routers. The gray band spans the middle 90% of their pass rates, from the 5th to the 95th percentile. We call a router’s performance statistically significant if it lies above the band: its pass rate beats 95% of random routers at the same cost. Illustrative example of the random routing gray band. Router X lies above the band, so it beats random routing at the same cost by a statistically significant margin. Router Y lies inside the band, so its performance is indistinguishable from random routing. Experimenting with architectures Baseten Router’s problem formulation requires it to find signals in a task that predict the value of choosing one model over the other. This section describes the architectures we tested and how early results shaped our final approach. We compared experiments on the training set using leave-one-benchmark-out cross-validation. Exploring embeddings We began by embedding the task instructions and fitting separate heads to predict each model’s probability of success and expected cost. Pre-trained embedding models, such as Qwen3-Embedding [8] and EmbeddingGemma 2 [9], are a natural starting point: they capture semantic features of the task without needing to train a language model. To understand which predictions mattered, we compared three routing setups: predicting success advantage alone, predicting expected cost alone, and predicting success advantage relative to expected cost. Routing on predicted cost alone captured most, but not all, of the initial router gains over random routing. One hypothesis is that embeddings capture task scope, which is a clearer signal of cost than of which model will succeed. Subsequent analysis supported this: requirement count and estimated code diff size are strongly correlated with cost, but not with the gap in success rate between models. Combining predictions for both success advantage and expected cost led to the best cross-validation results, so we kept that setup for later experiments. Comparison of router pass-rate gain by predicting: only cost, only success rate, both cost and success rate. Looking beyond semantic similarity Having established that the embedding router improved on random routing, we next explored which features were responsible for driving those gains. Some, such as the task category, are legitimate routing signals. For example, implementing a new feature and answering a question about a codebase have different success and cost profiles. However, embeddings can also capture cues specific to the benchmark a task belongs to. Rel [truncated for AI cost control]