Kimi K3 is Moonshot AI's 2.8T-parameter open-weight model—the first in the 3-trillion-parameter class. Here's how it's architected, what it costs on Together AI, and how to call it with code examples for reasoning effort, streaming, vision, structured output, tools, and dynamic tool loading.
Kimi K3 is a 2.8T-parameter open-weights model, the largest open-weight model ever released, with 1M context support.
New architecture: Kimi Delta Attention, Attention Residuals, and Stable LatentMoE activate only 16 of 896 experts per token.
ThunderAgent is a program-aware scheduler for agentic inference. By treating each agent workflow as a schedulable program, it eliminates KV cache thrashing to deliver more than 2x single-node throughput and near-linear multi-node scaling.
Introduces a program abstraction for agentic LLM request scheduling, eliminating KV cache thrashing
Achieves up to 2.5x single-node throughput and 2.4x speedup on an 8-node cluster
Together AI and Moonshot AI form a strategic partnership, making Together AI the launch platform for Moonshot's open models, starting with Kimi K3, a 2.8-trillion-parameter sparse MoE model.
Together AI becomes the launch platform for Moonshot AI models, offering day-zero access and US-hosted infrastructure.
Kimi K3 is a 2.8T parameter sparse MoE model with native vision and 1M context, featuring Kimi Delta Attention and Attention Residuals.
We ran 904 DeepSWE rollouts on Kimi K3 and GPT-5.6 Sol. Sol leads pass@1; Kimi K3 wins pass@4 at 2.8x the solves per dollar, and routing between them reaches ~85.6%.
We ran 452 DeepSWE rollouts on Kimi K3 and Claude Fable 5. Fable leads pass@1 by 1.4 points; Kimi K3 wins pass@4 and delivers 2.8x the solves per dollar.
Kimi K3 achieves 68.5% pass@1 vs Fable's 69.9%, but wins at pass@2 (82.0% vs 80.2%) and pass@4 (89.4% vs 88.5%).
Kimi K3 costs $4.65 per rollout vs $13.41 for Fable, yielding 2.8x more solved tasks per dollar.
Together AI releases a major update to its inference platform, giving users control over performance, cost, and quality. New features include canary deployments, A/B testing, autoscaling, and a closed beta for custom training with reinforcement learning and fine-tuning.
Deploy open-weight models in minutes with production-grade controls
Supports canary, blue-green, and rolling updates with auto rollback
Together AI and Y Combinator have partnered to provide YC startups with a dedicated GPU cluster, addressing the compute bottleneck. Startups can commit for weeks instead of two-year contracts, and manage GPUs directly through a self-service portal without YC involvement. The cluster is fully utilized and supports needs from single-node to large-scale expansion.
Together AI and YC launch first dedicated GPU cluster for YC startups, providing direct compute access.
Commitments are for weeks instead of two-year contracts, freeing capital for other business needs.
This article breaks down the real meaning behind reliability numbers like 99%, 99.9%, and 99.99% uptime for AI inference services, explaining the failure domains each tier must survive and the architectural requirements. The authors from Together AI share their experience building reliable inference infrastructure and provide key questions to ask any provider before committing.
Each reliability tier maps to a specific failure domain: 99% for node failures, 99.9% for data center failures, 99.99% for regional outages.
Achieving high uptime requires proactive health checks, multi-facility deployment, and reserved capacity.
See how Together AI is improving production GPU clusters with passive health checks, node repair, stronger Slurm reliability, OIDC, and startup scripts.
Together AI introduces passive health checks and auto node repair for faster failure detection and recovery.
Slurm-on-K8s 2.0 provides self-healing daemons, durable job accounting, and reliable process cleanup.
Thinking Machines Lab released Inkling, a multimodal mixture-of-experts model for token-efficient reasoning, native multimodal understanding, and broad task versatility. Together AI makes it available on its inference platform with support for controllable reasoning effort, text/image/audio inputs, and a 1M context window.
Inkling is a multimodal MoE model with 975B total parameters, 40B active per token, and a 1M context window.
It accepts text, image, and audio inputs and supports adjustable reasoning effort for cost-latency trade-offs.
Provisioned Throughput offers reserved inference capacity for frontier open models with token-based pricing and a 99% uptime SLA, reducing costs by up to 90% compared to proprietary APIs.
Reserved inference capacity with token-based pricing and 99% uptime SLA.
Priced at $0.05 per PTU per minute; no GPU-hour math or infrastructure management.
Together AI raised $800M in Series C funding to accelerate the shift to open-source AI. The company argues that closed models' economics don't scale, and open models combined with full-stack optimization can achieve 6-20x cost reductions. Together AI has launched innovations like FlashAttention-4 and Together Megakernel, becoming one of the world's largest AI token producers.
Together AI raised $800M in Series C funding to accelerate open-source AI
Closed models' cost structure is unsustainable at scale
Eight papers from Together AI accepted to ICML 2026, covering the full AI stack from agents to GPU kernels. These research works are integrated into the Together platform and already benefit production workloads.
Eight papers accepted to ICML 2026, spanning the full AI stack from agents to kernels.
DSGym provides a unified framework for data science agent evaluation and training with over 1,000 tasks across 10+ domains.
ParallelKernelBench tests whether LLMs can write fast multi-GPU CUDA kernels across 87 real workloads. The best model solves under a third, but a few generated kernels beat any public implementation.
ParallelKernelBench (PKB) includes 87 multi-GPU kernel generation problems from real codebases.
The best frontier model (GPT-5.5) solves under a third of problems in zero-shot setting, with only 22 faster than baseline.
We generated 12 landing pages with Kimi K2.7 Code and Claude Fable 5. Kimi cost 94% less and scored within a few points on every page. Open-source models are not only cheaper but genuinely competitive on quality, and the gap is closing fast.
Kimi K2.7 Code costs about 94% less than Claude Fable 5 for generating landing pages.
Quality scores show a small gap between Kimi and Fable, especially after using a design inspiration MCP.
Together AI has achieved ISO 27001:2022 certification from A-LIGN, validating its information security management system for enterprise-grade AI workloads, complementing existing SOC 2 controls.
ISO 27001:2022 certification awarded by A-LIGN Compliance and Security
Scope covers global platform, corporate HQ, and third-party data centers
Together AI optimizes MiniMax M3 serving with KV-block-major sparse attention, paged MSA decode, optimized index scoring, and a Rust-based multimodal gateway, achieving 81–125% throughput improvements across concurrency levels.
MiniMax M3 combines coding, agentic workflows, and multimodal reasoning with a 1M-token context window.
Together AI's kernel team developed KV-block-major sparse attention and integrated MSA with paged attention.
Together AI built the fastest speech-to-text stack on Artificial Analysis by treating ASR as a full-path systems problem, not just a GPU inference problem. This article details optimizations including TensorRT multi-profile encoders, conditional CUDA graphs, shared memory, evented I/O, and gc.freeze() to eliminate tail latency.
Together AI achieved fastest STT by optimizing the entire system path, not just GPU inference.
Key techniques: TensorRT multi-profile encoders, conditional CUDA graphs, zero-copy shared memory, and evented I/O.
On a production coding agent workload, Together Inference Engine delivers 31% more TPS than the next fastest OSS engine on the same hardware, and maintains 2× better TTFT at saturation. The gains come from full-stack optimization: ThunderMLA, custom kernel rewrites, and end-to-end profiling on real traffic.
Real-world inference benchmarks for coding agents: 31% more TPS than TensorRT-LLM, 2× better TTFT at saturation, and 76% lower cost than Claude Opus 4.6.
Full-stack optimization includes ThunderMLA fused kernel, custom kernel rewrites, and end-to-end profiling.
Together AI partners with Pearl Research Labs to launch a discounted Pearl-powered inference endpoint for Gemma-4-31B-it-pearl, using Proof of Useful Work to turn AI workloads into crypto emissions.
Together AI partners with Pearl Research Labs to offer a discounted inference endpoint.
The endpoint uses Proof of Useful Work to mine cryptocurrency while performing AI inference.
Violin is an open-source AI video translation tool combining speech recognition, LLM translation, and text-to-speech to make video content accessible across languages. It offers a web app, CLI, and agent skills, featuring a video-aware chat assistant and personalized voice selection. Built with Together API using models like Whisper, DeepSeek, and Cartesia, it's released under the MIT license.
Violin integrates ASR, LLM translation, and TTS for open-source video translation.
Supports web app, CLI, and agent skills for diverse users.
Voice finder helps developers search, match, filter, and audition 600+ voices across Together AI TTS models using natural-language prompts or uploaded audio samples.
Search, filter, and audition over 600 voices across leading TTS models
Find voices via natural language descriptions or audio sample uploads
Learn how to deploy any HuggingFace model in one session using Goose and Together's Dedicated Container Inference. Skip the setup complexity — one prompt gets your model running in a production-grade GPU environment on release day.
Use Goose and Together's Dedicated Container Inference to deploy models with zero lag on release day.
Author deployed Netflix's void-model with a single session and prompt.
DeepSeek-V4's hybrid attention design (CSA, HCA, SWA) compresses KV cache, turning million-token context from a model challenge into a serving-systems problem. Together AI's early bring-up on NVIDIA HGX B200 reveals how cache policy, prefix caching, and endpoint profiles impact long-context workloads.
DeepSeek-V4's compressed sparse attention (CSA) and heavily compressed attention (HCA) reduce KV cache size, but the inference engine must manage multiple cache layouts.
Sliding window attention (SWA) becomes a bottleneck at long context, requiring careful storage strategy.
As AI moves from research to production, the challenge for AI-native teams shifts from building models to running them — efficiently, reliably, and at scale. Inference costs account for 80-90% of total lifetime cost of a production AI system. Together AI's research (FlashAttention-4, ATLAS) and full-stack optimization enable efficient inference, improving unit economics for customers.
Inference costs dominate AI system economics, comprising 80-90% of total lifetime cost.
Together AI introduces FlashAttention-4 (up to 1.3× faster than cuDNN) and ATLAS (adaptive speculative decoding for 4× faster inference).
Together AI and Adaption partner to bring Together Fine-Tuning natively into Adaptive Data, helping teams optimize datasets, run fine-tuning, evaluate results, and deploy stronger open models.
Together AI partners with Adaption to integrate fine-tuning into Adaptive Data.
The partnership aims to simplify the workflow from data optimization to model deployment.
Together AI details their rapid response to the Linux kernel vulnerability Copy Fail (CVE-2026-31431), which allowed local unprivileged users a precise 4-byte write primitive via the AF_ALG interface, leading to privilege escalation. The team mitigated by unloading the vulnerable kernel module, rolling out patches, and enhancing detection, ensuring AI infrastructure security.
Copy Fail (CVE-2026-31431) is a logic bug in the Linux kernel's crypto subsystem allowing precise 4-byte writes to any readable file's page cache.
Together AI unloaded the algif_aead module and removed its file within hours, blocking exploits without reboot.
DeepSeek-V4 Pro, a 1.6T-parameter MoE reasoning model, is now available on Together AI with a 512K context window, controllable reasoning modes, and cached-input pricing for long-context workloads like code agents, document intelligence, and research synthesis.
1.6T-parameter MoE with 49B activated parameters, 512K context on Together AI (model supports 1M)
Three reasoning modes: Non-Think, Think High, Think Max to match effort to task