[AINews] New AI Infra decacorns: Fireworks, Baseten (with OpenRouter on the way)

Take the 2026 AI Engineering Survey and get >$2k in credits and AIE WF tickets!

Readers like when we report no news, but our second favorite to that is when we can simply reinforce a trend you should be aware of. In April we highlighted the Inference Inflection, and If today’s headline reminds you of last week’s headline, it is exactly the point we are making.

$2k in credits and AIE WF tickets!","cta":null,"showBylines":true,"size":"sm","isEditorNode":true,"title":"[AINews] New AI Infra unicorns: Exa, Modal, TurboPuffer","publishedBylines":[{"id":89230629,"name":"Latent.Space","bio":"Writer, curator, latent space explorer. Main blog: https://swyx.io Devrel/Dev community: https://dx.tips/ Twitter: https://twitter.com/swyx","photo_url":"https://substack-post-media.s3.amazonaws.com/public/images/db0f8d45-1eb8-4c02-a120-650d377ee52d_640x640.jpeg","is_guest":false,"bestseller_tier":1000}],"post_date":"2026-05-22T05:50:58.325Z","cover_image":"https://substack-post-media.s3.amazonaws.com/public/images/ab2507aa-9755-4e9d-9cbf-4c7f755a8527_1086x280.png","cover_image_alt":null,"canonical_url":"https://www.latent.space/p/ainews-new-ai-infra-unicorns-exa","section_name":"AINews: Weekday Roundups","video_upload_id":null,"id":198804002,"type":"newsletter","reaction_count":43,"comment_count":0,"publication_id":1084089,"publication_name":"Latent.Space","publication_logo_url":"https://substackcdn.com/image/fetch/$s_!DbYa!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F73b0838a-bd14-46a1-801c-b6a2046e5c1e_1130x1130.png","belowTheFold":false,"youtube_url":null,"show_links":null,"feed_url":null}">

With the pace of AI fundraising these days, our general policy is to only cover startups when they cross decacorn status (>$10B) - but only when confirmed, and today’s news of Fireworks’ $15B round (“in talks”, 3.75x in 7 months, our podcast here) and Baseten’s $11B round (“is raising”, 2.2x in 3 months) is a bit premature, but the pace of the pickup in Inference land and unicorn to decacorn progression is too juicy not to serve as headline story today, with the $113M OpenRouter Series C (5x volume in 6 months) as the cherry on top: if you are gonna do multimodel inference, you are gonna need a router.

AI News for 5/23/2026-5/26/2026. We checked 12 subreddits, 544 Twitters and no further Discords. AINews’ website lets you search all past issues. As a reminder, AINews is now a section of Latent Space. You can opt in/out of email frequencies!

AI Twitter Recap

Agent Harnesses, Coding Benchmarks, and the Shift Beyond “Just the Model”

Harness engineering is becoming the main differentiator for coding agents: Several posts converged on the same thesis: the winning stack is now model + harness + eval loop, not just a stronger base model. A long Zhihu summary argued that DeepSeek is explicitly building a harness team to close the loop between model outputs, runtime feedback, validation, and correction, with a claimed cached-input cost advantage that would support tighter interaction/verification loops. In parallel, Google’s Gemini Managed Agents guide framed agent infra as a single API call to a managed harness with sandboxing, persistence, and mounts, while LangChain’s updated create_agent docs and dair.ai’s “harness” paper summary formalized the same stack: context governance, trustworthy memory, dynamic skill routing.

Benchmarks are getting closer to real developer experience: DeepSWE, introduced as a new benchmark for agentic coding, got strong endorsement from practitioners; @theo called it “the first code bench that actually aligns with how it feels to use these models coding.” It also created more separation at the top end than public SWE leaderboards often show. Related benchmark signals: Qwen3.7 Max debuted at #4 on Code Arena: Frontend, roughly on par with Claude Opus 4.6 on agentic webdev tasks, and Alibaba amplified the result. Across the tooling stack, Anthropic shipped a security-guidance plugin for Claude Code and reported a 30–40% reduction in security-related PR comments in internal use, while OpenAI highlighted GPT-5.5 in Codex at Databricks for more reliable document parsing.

Research Agents, Long-Horizon Reasoning, and “Sleep” for Context Compression

Math/science agents showed more evidence of capability overhang—conditional on the right harness: The strongest cluster of tweets was around models tackling old open problems. A mathematician reported Claude Mythos solving Erdős problem #90, with follow-up detail that the model often converged to a different, cleaner proof path than OpenAI’s earlier route. This was echoed by @_sholtodouglas, @kimmonismus, and then sharpened by Sébastien Bubeck: with an appropriate harness, both Mythos and GPT-5.5 can reproduce what an internal model had done one-shot, implying a large amount of latent capability not exposed by vanilla chat UX.

Long-horizon memory is resurfacing as a core bottleneck: The paper “Language Models Need Sleep” got notable attention. The mechanism is a sleep-like consolidation phase where recent context is converted into persistent fast weights before clearing the KV cache, moving compute into an offline pass while preserving wake-time latency. dair.ai’s summary emphasized the systems angle: this is an alternative to ever-growing KV caches for agents with long trajectories. This theme connected neatly with ongoing discussion about memory systems in agents, including Omar’s pointer to Anthropic’s memory talk and Dream feature.

Open deep-research agents and science forecasting also advanced: QUEST, a family of open 2B–35B models for long-horizon fact-seeking, citation grounding, and report synthesis, was released as a general-purpose deep research agent. On the science-evals side, Sakana/Stanford/Oxford/AI2’s CUSP benchmark found current models can often identify promising research directions but struggle much more with whether and when breakthroughs materialize.

Model, Optimizer, and Architecture Updates

Optimizer work remains lively, especially around Muon variants and schedule-free training: AMUSE proposes Anytime MUon with Stable gradient Evaluation, combining Muon with schedule-free-style gradient evaluation for stable anytime training without LR decay, reporting gains at 124M / 720M / 1B scale and on ViT/ImageNet fine-tuning. Related implementation discussion came from ClashLuke’s SFMuon snippet and kellerjordan’s Modded-NanoGPT result on Newton-Muon.

Sparse attention design space continues to diversify: MiniMax teased M3 as open source, and follow-on technical commentary suggested a new block-sparse two-stage attention path. @kimmonismus summarized the reported speedups: 9.7× prefilling and 15.6× decoding at 1M tokens versus M2. @eliebakouch added that M3 appears to move back to GQA-based sparse attention with block selection on real KV, distinct from DeepSeek’s compressed-attention variants.

Vision/open model releases and ranking updates: PrismML released Bonsai Image 4B, including 1-bit and ternary variants intended to run locally on laptops and phones; a follow-up noted browser-local execution was possible at ~3GB footprint. On the closed side, Microsoft’s MAI-Image-2.5 debuted at #3 on the Image Arena, breaking a top-5 club previously dominated by OpenAI and Google, with Arena reporting a 1,254 score. Meanwhile, Artificial Analysis measured Gemini 3.5 Flash at up to ~280 output tok/s with materially stronger agentic performance, but at ~5× the cost of Gemini 3 Flash.

Infra, Systems, and the Semiconductor Stack

Huawei’s “τ scaling” paper was read mostly as an engineering roadmap, not a new law: A very detailed thread argued Huawei’s “A Time Scaling Theory for Multi-Layer Electronic Systems” should be interpreted as a strategic manifesto / white paper. The core proposal is to treat time constant τ, not process node, as the unifying metric across device, chip, and datacenter scales. The most concrete claims concerned LogicFolding on a future Kirin design, including +55% density, +41% energy efficiency, and +13% frequency at fixed node, plus packaging/network ideas like a Unified Bus and Hi-ONE optical I/O. The same thread was careful to note missing validation artifacts—die photos, SEMs, workload details, yield curves—and to interpret the most eye-catching numbers as promising but unverified. Follow-up reactions also stressed that Huawei’s path may rely more on packaging and architecture than lithographic catch-up, e.g. @josiah_leee citing Jensen’s point that most of Hopper→Blackwell’s gains came from non-node optimizations.

Datacenter power and inference supply constraints are becoming first-order concerns: SemiAnalysis published on the 800VDC transition, and John Carmack recommended it, highlighting crossovers from EV power electronics into datacenter design, including high-voltage SiC parts. Separately, Epoch AI estimated a possible inference compute crunch: demand appears to be growing faster than serving capacity, especially for long-context workloads. Their rough model suggested that while current global Blackwell supply could serve today’s demand under favorable assumptions, throughput degrades sharply with longer contexts and demand growth may already be outrunning supply.

Production Tooling and Developer Infrastructure

Serving/inference stacks got meaningful performance and observability updates: vLLM merged a Rust frontend as a drop-in alternative to the Python API server, with early numbers showing ~837 req/s vs ~162 req/s on a preprocess-heavy workload in a single process. W&B launched an MCP server to let coding agents inspect experiments and training runs, with a schema-first redesign aimed at avoiding context-window blowups. Unsloth added support for running GPT, Claude, and other APIs inside its local UI, including prompt caching and code execution.

Cloudflare, OpenRouter, and vector/retrieval vendors pushed the “productionization” layer: OpenRouter announced a $113M Series B and said weekly volume had grown from 5T to 25T tokens over six months. Cloudflare relaunched its startups program with up to $350k in credits, while separate posts around Think and agent ergonomics emphasized durable turns, reconnects, stale-state handling, and recovery as key practical differentiators. On retrieval infra, Booking.com discussed scaling to 100M+ embeddings, including filtered vector search, reads-during-writes, concurrency, and human-in-the-loop evals for partner messaging agents.

Top tweets (by engagement)

Codex / agentic coding in practice: The highest-signal product-use tweet was @bunkaich showing Codex help reverse-engineer and patch firmware on a cheap MP3 player, with the workflow spanning chip inspection, OS extraction, binary analysis, and flashing a modified image.

DeepSWE benchmark launch: @serenaa_ge’s DeepSWE announcement became the main reference point for “does this match real coding experience?” discussion.

Claude Code security plugin: @ClaudeDevs’ release stood out because it paired a concrete product launch with an internal metric: 30–40% fewer security-related PR comments.

OpenRouter financing + production token growth: @OpenRouter’s $113M Series B is one of the clearer market signals that routing and multi-model infra are now seen as durable platform layers.

vLLM Rust frontend: @vllm_project’s merge announcement mattered for anyone hitting CPU/API-server bottlenecks in high-throughput serving.

AI Reddit Recap

/r/LocalLlama + /r/localLLM Recap

1. Qwen 3.7 Launch and Qwen 3.6 Local Performance

Waiting for Qwen 3.7 open weight... The new King has arrived... (Activity: 1217): The image is a benchmark/marketing comparison from the Qwen3.7 blog positioning Qwen3.7-Max as a leading frontier model across agentic coding, software engineering, MCP/tool-use, reasoning, and knowledge evaluations versus Qwen3.6-Plus, DS-V4-Pro Max, GLM-5.1, Kimi K2.6, and Claude Opus-4.6 Max. The techni

[truncated for AI cost control]