AI models Introducing NVIDIA Nemotron 3.5 ASR Streaming Deploy NVIDIA Nemotron 3.5 ASR for low-latency, production-ready speech recognition with 6x higher throughput and multilingual support. Authors Ansel Erol Ian Carr…
AI generation is temporarily unavailable; this entry was preserved with deterministic fallback metadata.
AI models Introducing NVIDIA Nemotron 3.5 ASR Streaming Deploy NVIDIA Nemotron 3.5 ASR for low-latency, production-ready speech recognition with 6x higher throughput and multiling…
AI models Laguna S 2.1 goes Greek: a repository-scale game transformation We put Poolside’s new Laguna S 2.1 model to the test, tasking it with a repository-scale transformation of the open-source game Hypersomnia. Auth…
AI generation is temporarily unavailable; this entry was preserved with deterministic fallback metadata.
AI models Laguna S 2.1 goes Greek: a repository-scale game transformation We put Poolside’s new Laguna S 2.1 model to the test, tasking it with a repository-scale transformation o…
A guide to fine-tuning Qwen3-TTS for high-quality voice cloning, comparing ICL, speaker-embedding-only, and fine-tuning approaches with a full training recipe. Using about 1.5 hours of LJ Speech data, the fine-tuned model trains in about 1 hour on a single H100 and achieves a TTFA of roughly 130 ms, about 16% faster than ICL, while eliminating reference audio processing at inference.
Qwen3-TTS supports ICL and speaker-embedding-only zero-shot cloning, but fine-tuning unlocks richer voice cloning quality.
Fine-tuning uses SFT on utterance-level text-audio pairs and bakes a centroid speaker embedding into the model weights, removing reference audio at inference.
This worklog traces the architectural developments from GPT-2 (124M parameters) to Kimi K3 (2.8T parameters), a 22,580x scale increase in seven years. It explains key innovations including KV cache, linear attention, and DeltaNet, highlighting how the underlying mechanisms evolved to handle massive scale.
GPT-2 had 124M parameters; Kimi K3 has 2.8T parameters, a 22,580x increase.
KV cache avoids recomputation but grows linearly with sequence length.
Baseten Switch is a local Mac app that lets you seamlessly mix open and closed models in popular AI harnesses like Claude Code and Codex, with a toggle to switch between models like Kimi K3 and GLM 5.2, while tracking cost, performance, and usage.
Baseten Switch is a local Mac app (Go) that routes requests between different models.
Enables live switching between open and closed models in harnesses like Claude Code and Codex.
Baseten launches a new platform for closed-weight model labs to distribute and monetize models, featuring production-grade inference infrastructure, model library distribution, IP protection, and go-to-market support. The platform already has 15 lab partners including Cartesia, Gradium, Inception, NVIDIA, and more.
Baseten for Model Labs provides closed-weight model labs with a turnkey solution for model distribution and monetization.
Includes production inference infrastructure, model library visibility, IP protection, and joint GTM support.
Baseten introduces a new tokenizer (Basetenkenizer) that speeds up Kimi K3 tokenization by up to 18x for million-token agentic workloads. It combines Rust-based optimizations like specialized pre-tokenization, BPE merge, multi-core chunking, and zero-copy NumPy transfer, while maintaining exact token ID parity. The improvement significantly lowers time-to-first-token for long input sequences, especially when prefix cache hits are high.
Baseten Tokenizer accelerates Kimi K3 tokenization by up to 18x for million-token sequences.
Optimizations include specialized pre-tokenization, stack-resident BPE merge, multi-core parallelization, and zero-copy NumPy ownership.
Baseten provides day-0 support for Kimi K3, a 2.8T-parameter open frontier model. This article details the technical work required to run the model at scale for launch, including hardware provisioning, weight loading, inference engine integration, quality validation via Kimi Vendor Verifier, performance optimization with custom tokenizer, and deployment across GB300 NVL72 systems.
Kimi K3 is a 2.8T-parameter open model with novel architectures like Kimi Delta Attention and Stable LatentMoE.
Baseten collaborated with Inferact and RadixArk to support Kimi K3 on vLLM and SGLang.
We are introducing a new Model API tier, GLM-5.2 Fast, optimized for real-time agentic workloads with the same weights as standard GLM-5.2 but on infrastructure tuned for per-user throughput, offering lower latency and stable performance.
GLM-5.2 Fast is a separate Model API tier using the same GLM-5.2 weights but with infrastructure optimized for real-time applications.
Designed for agentic workflows as the main agent to coordinate sub-tasks, reducing latency compounding.
This article compares H100 and H200 GPUs for AI inference. H100 is cost-effective for small-to-medium models and low/sporadic traffic. H200, with larger HBM3e memory and higher bandwidth, is better for very large models, long-context inference, and workloads needing more KV cache headroom. Techniques like MIG and async programming are also covered.
H100 is cost-effective for small-to-mid models and low traffic.
H200 has larger memory and higher bandwidth, suitable for large models and long contexts.
Lessons from Notion and Gamma on picking AI models: swap for cost and reliability, match model to task, lean into open-weight models, and fine-tune to grow the market.
Avoid a single model-agnostic harness; modify each model's native harness for your use case.
Model switching is worth the engineering cost for reliability, cost flexibility, and speed of adoption.
Researchers from Baseten successfully added vision capabilities to the open-source language model GLM 5.2 by training only a small 50M-parameter MLP projector, achieving performance comparable to Claude 4.5 Haiku on MMMU-Pro (55%). The process involved SFT and RL with grokking observed, and demonstrated generalization to unseen images.
Post-trained vision onto GLM 5.2 using a tiny 2-layer MLP with 50M parameters.
Achieved vision performance equivalent to Claude 4.5 Haiku on MMMU-Pro without degrading text capabilities.
Baseten's optimized runtime for Wan 2.2 achieves 2.75 seconds per video clip, a 53.6x improvement, using timestep distillation, custom kernels, and NVFP4 quantization.
53.6x speedup over baseline Wan 2.2 implementation
2.75 seconds per video clip with real-time performance
NVIDIA Nemotron 3 Embed models are now available in Baseten. The larger 8B model delivers leading retrieval accuracy, while the smaller, faster 1B model retains 95% of that accuracy. Both are now available on Baseten for AI agents, enterprise search, and code retrieval.
NVIDIA Nemotron 3 Embed 8B and 1B models are now available on Baseten, optimized for retrieval quality and indexing speed respectively.
The 8B model tops the RTEB leaderboard; the 1B model retains 95% accuracy with reduced latency and cost, and NVFP4 version offers 2x throughput on Blackwell GPUs.
Thinking Machines Lab has released Inkling, a 975-billion-parameter open-weight multimodal model that reasons over text, images, and audio. With a mixture-of-experts architecture and 41 billion active parameters, it balances performance and cost. Baseten provides day-zero support for deployment and scaling.
Inkling is a 975B-parameter MoE model with 41B active parameters and a 1M token context window.
It natively processes text, images, and audio inputs.
StepFun's Step 3.7 Flash is now available in the Baseten Model Library in a hardware-efficient configuration. This 198-billion-parameter sparse MoE vision-language model supports multimodal input, a 256k context window, and flexible reasoning for agentic workflows. FP8 quantization enables deployment on 4×H100 GPUs, significantly reducing cost.
198B parameter MoE architecture with 11B activated parameters per token
Native image and video input with 256k context window
AI training teaches models to learn from data. Inference is what runs in production. This article explains the key differences in hardware, cost, and optimization, covering the model lifecycle from pretraining to serving, the four metrics for inference performance, and a comparison between the two phases.
Training is the process of a model learning from large datasets by adjusting its weights, requiring significant compute resources.
Inference occurs every time a trained model generates output for a user request, with no learning involved.
GLM-5.2 is this year's DeepSeek moment, matching closed-source quality while being 4.5x faster and 5x cheaper. This article provides step-by-step instructions to set up GLM-5.2 in Claude Code, Codex, and Deep Agents CLI in under 5 minutes.
GLM-5.2 is a high-performance open-source model that can replace closed models like Opus 4.8
Configure Claude Code by editing environment variables to use GLM-5.2
NVIDIA BioNeMo Agent Toolkit helps transform general-purpose AI agents into scientific agents capable of performing real biological and drug discovery tasks. The toolkit combines BioNeMo Skills, open models, NVIDIA NIM microservices, and agent infrastructure to enable workflows such as protein structure prediction, protein design, virtual screening, genomics analysis, and target discovery. All BioNeMo NIM microservices are available today in the Baseten Model Library, making it easy for developers to deploy and scale scientific AI applications.
NVIDIA BioNeMo Agent Toolkit transforms general AI agents into scientific agents for biology and drug discovery.
It integrates BioNeMo Skills, open models, NVIDIA NIM microservices, and agent orchestration infrastructure.
We compare 8 top open-source LLMs in production: DeepSeek V4 Pro, Gemma 4, GLM 5.1, GPT OSS 120B, Kimi K2.6, MiniMax M3, Nemotron 3 Ultra, and Qwen 3.6. Find the best model for agentic coding, long-context reasoning, cost, and speed.
Kimi K2.6 is the most well-rounded; Qwen 3.6 and GLM 5.1 lead for agentic coding; DeepSeek and Nemotron dominate long-context and enterprise workloads; GPT OSS 120B performs well on cost and speed.
DeepSeek V4 Pro offers a 1M-token context window with CSA and HCA reducing KV cache memory to ~2% of standard models.
Baseten introduces rolling deployments, enabling teams to update models incrementally without downtime or doubled GPU costs. Replicas are replaced one at a time with gradual traffic shifting, plus pause, resume, and rollback controls. Customers report 50-60% more frequent deployments, eliminating off-peak manual babysitting.
Rolling deployments replace replicas step-by-step, avoiding blue-green's doubled GPU cost and hard cutover's all-or-nothing risk.
Two modes: max_surge (scale up new first) and max_unavailable (scale down old first), for latency or cost sensitivity.
Inception's Mercury 2, a diffusion LLM, is now available on Baseten. It generates over 1,000 tokens per second, 5-10x faster than leading speed-optimized models, at half the cost with comparable quality. It enables real-time speed on standard NVIDIA GPUs without custom chips. Augment Code cut costs by 90% and latency by 82% using Mercury 2.
Mercury 2 is the fastest reasoning LLM, using diffusion to generate full output in parallel passes.
It runs over 1,000 tokens per second on standard NVIDIA GPUs, reducing costs and latency.
Nemotron 3 Ultra is a hybrid Mamba-transformer model designed for long-running agents, delivering up to 5x faster inference and 30% lower cost by replacing most attention with Mamba layers. Fully open, it enables agents to complete lengthy tasks efficiently without slowdown.
Nemotron 3 Ultra uses a hybrid architecture with mostly Mamba layers to maintain constant inference speed as context grows.
Achieves up to 5x faster inference and 30% lower cost for long-running agent workflows compared to open frontier models.
Baseten and Microsoft AI announce that MAI-Thinking-1, a new flagship reasoning model, will be available on Baseten. It offers a unique balance between open-source flexibility and closed-model convenience, with clean data lineage, commercial-grade quality, and customization options.
MAI-Thinking-1 is Microsoft AI's new reasoning model that bridges open-source and proprietary models.
Trained on curated data without distillation, ensuring clean and auditable data lineage.
NVIDIA's Cosmos 3 is a foundation model for physical AI, designed to help developers build robots and autonomous systems by understanding and simulating the physical world. It supports six modes and can act as a direct controller or a data factory to generate training data, addressing the data bottleneck in robotics.
Cosmos 3 is a world foundation model for physical AI, not just video generation.
It supports six modes: text2image, text2video, image2video, forward_dynamics, inverse_dynamics, policy.
Baseten and Trajectory have built a production-grade inference pipeline for continual learning, where models are continuously updated from production traces. The pipeline compresses the time from training to deployment to roughly one hour, enabling models that improve through usage.
Continual learning allows models to improve continuously from production usage rather than static releases.
Baseten and Trajectory developed a pipeline that merges LoRA adapters, validates, and deploys them with A/B routing and provenance tracking.
An introductory guide to open-source AI models covering what they are, how they work, when to use them, and their advantages over closed-source models. Includes discussion of model weights, fine-tuning, cost savings, and strategic considerations.
Open-source models typically refer to open-weight models, allowing fine-tuning and self-hosting.
They offer 87% lower cost on average compared to closed-source models.