Baseten Blog

News Securing the open frontier with NVIDIA OpenShell and Blaxel sandboxes Baseten is a launch partner for the NVIDIA Agent Safety Platform and supports OpenShell in the latest generation of Blaxel Sandboxes. Authors Ni…

News Sheila Vashee joins Baseten as CMO Welcome Sheila Vashee! Authors Tuhin Srivastava Last updated September 25, 2026 Share The next era of AI will be defined by what companies can build with intelligence that is open…

News Fine-tune on your LangSmith traces with Baseten Loops LangSmith Fine-Tuning is in public beta today, and its open-source CLI, smithtune, trains on Baseten Loops. Authors Mudith Jayasekara Aaron Ellis-Bloor Last upd…

AI models NVIDIA Nemotron 3 Diarization: real-time speaker labels at a cent per audio hour NVIDIA Nemotron 3 Diarization answers who spoke when, 300ms–1s behind live, alongside any ASR. Authors Ansel Erol Last updated S…

Product Introducing Baseten Hosted Tools Hosted tool execution on the inference backend augments model capabilities and lowers latency. Authors Sai Maddali Marius Killinger Marylise Tauzia Last updated September 16, 202…

News LangChain trains custom models for LangSmith Engine with Baseten Loops Loops provides managed infrastructure for fine-tuning models through an API, with support for SFT, RL, and long-context workloads. Authors Aaro…

AI models DeepSeek-V4.1-Flash: more efficient prefill for coding agents Discover DeepSeek-V4.1-Flash, a 552B multimodal MoE model featuring a Causal Encoder-Decoder architecture and efficient KV caching. Authors Albert…

News Blaxel is joining Baseten to build the future of agentic infrastructure Baseten has acquired Blaxel Authors Amir Haghighat Tuhin Srivastava Paul Sinai Last updated September 10, 2026 Share Today, Blaxel is joining…

Model performance How Baseten makes pyannote’s diarization models 9.6x faster Using quality-aware quantization, index-based clustering, and scheduling optimizations Authors Matte Lim Ansel Erol Last updated September 9,…

Baseten launched a remote MCP server and skill that let coding agents like Claude Code, Codex, Cursor, Gemini CLI, VS Code, and Windsurf operate Baseten more efficiently. Benchmarks show a 7.5% average reduction in task time and cost, with roughly 50% gains on backend-heavy tasks.

AI models Best open-source models for post-training Compare top open-source models for post-training. Learn what drives cost and which model fits use case. Authors Chloe Florit Last updated September 2, 2026 Share TL;DR…

Model performance The efficient frontier of LLM inference Inference techniques either move a deployment along the latency–throughput frontier or push the entire frontier out, creating more efficiency to allocate. Author…

Model performance Agentic kernels in production Baseten's agentic kernel optimization framework cuts latency by 42.3% on Qwen-Image and by 15.2% on FLUX.2. Authors Brian Li Faraz Shahsavan Pankaj Gupta Last updated Augu…

AI models GLM 5.3: Scaling with post-training, intuitively explained GLM 5.3's gains came entirely from post-training. An intuitive look at the environment design, RL algorithms, and infrastructure behind the leap. Auth…

AI engineering The two AI gateway patterns in production inference Access gateways and serving gateways explained, and how they handle identity, tenancy, limits, and metering for teams serving AI models. Authors Amit Ga…

AI engineering How to run any open model inside DeepSeek Harness Learn how to power DeepSeek Harness with Baseten Model APIs and run open models like Kimi K3, GLM 5.2, and DeepSeek V4 Pro in under 5 minutes. Authors Ale…

Infrastructure How leading platforms ensure observability for LLM inference Learn how metrics, logs, and traces work together to catch slow responses, errors, and failed deployments before users do. Authors Chloe Florit…

Model performance Inference engineering for DeepSeek V4 Pro 0813 DeepSeek V4 Pro 0813 is a 1.7T-parameter open frontier model under the MIT license and is available for inference today on Baseten model APIs. Authors Mod…

Community Baseten delivers open-source inference for You.com Baseten is proud to power the inference behind You.com's search and answer stack. Authors Marylise Tauzia Last updated August 13, 2026 Share TL;DR You.com bui…

AI models Qwen3.8-Max: Alibaba's new frontier reasoning model Qwen3.8-Max is Alibaba's new open-weight frontier model, featuring a 1M token context window and multimodal input. Authors Albert Lee Last updated August 12,…

AI models Introducing NVIDIA Nemotron 3.5 Lightning NVIDIA’s Nemotron 3.5 Lightning, now on Baseten, delivers high-throughput, efficient reasoning for faster and more accurate agentic workflows. Authors Marylise Tauzia…

AI models Introducing NVIDIA Nemotron 3.5 ASR Streaming Deploy NVIDIA Nemotron 3.5 ASR for low-latency, production-ready speech recognition with 6x higher throughput and multilingual support. Authors Ansel Erol Ian Carr…

AI models Laguna S 2.1 goes Greek: a repository-scale game transformation We put Poolside’s new Laguna S 2.1 model to the test, tasking it with a repository-scale transformation of the open-source game Hypersomnia. Auth…

A guide to fine-tuning Qwen3-TTS for high-quality voice cloning, comparing ICL, speaker-embedding-only, and fine-tuning approaches with a full training recipe. Using about 1.5 hours of LJ Speech data, the fine-tuned model trains in about 1 hour on a single H100 and achieves a TTFA of roughly 130 ms, about 16% faster than ICL, while eliminating reference audio processing at inference.

This worklog traces the architectural developments from GPT-2 (124M parameters) to Kimi K3 (2.8T parameters), a 22,580x scale increase in seven years. It explains key innovations including KV cache, linear attention, and DeltaNet, highlighting how the underlying mechanisms evolved to handle massive scale.

Baseten Switch is a local Mac app that lets you seamlessly mix open and closed models in popular AI harnesses like Claude Code and Codex, with a toggle to switch between models like Kimi K3 and GLM 5.2, while tracking cost, performance, and usage.

Baseten launches a new platform for closed-weight model labs to distribute and monetize models, featuring production-grade inference infrastructure, model library distribution, IP protection, and go-to-market support. The platform already has 15 lab partners including Cartesia, Gradium, Inception, NVIDIA, and more.

Baseten introduces a new tokenizer (Basetenkenizer) that speeds up Kimi K3 tokenization by up to 18x for million-token agentic workloads. It combines Rust-based optimizations like specialized pre-tokenization, BPE merge, multi-core chunking, and zero-copy NumPy transfer, while maintaining exact token ID parity. The improvement significantly lowers time-to-first-token for long input sequences, especially when prefix cache hits are high.

Baseten provides day-0 support for Kimi K3, a 2.8T-parameter open frontier model. This article details the technical work required to run the model at scale for launch, including hardware provisioning, weight loading, inference engine integration, quality validation via Kimi Vendor Verifier, performance optimization with custom tokenizer, and deployment across GB300 NVL72 systems.

We are introducing a new Model API tier, GLM-5.2 Fast, optimized for real-time agentic workloads with the same weights as standard GLM-5.2 but on infrastructure tuned for per-user throughput, offering lower latency and stable performance.