According to OpenRouter data, agentic AI workloads consume 15x more tokens than a simple chat request. Why? Consider what happens when an AI agent researches a company for an investment decision. The agent queries financial databases, searches news and filings, invokes a sub-agent to run peer comparisons and model valuations, then synthesizes everything into a […]
AI generation is temporarily unavailable; this entry was preserved with deterministic fallback metadata.
According to OpenRouter data, agentic AI workloads consume 15x more tokens than a simple chat request. Why? Consider what happens when an AI agent researches a company for an inve…
arXiv:2608.18090v1 Announce Type: new Abstract: Inside a modern language model sits a single internal direction that tracks how positive or negative a sentence feels. We show how to find this valence axis (V-axis) from just 9 emotion category names plus 50 short narrative paragraphs per emotion -- about 1,500 fewer labels than the usual supervised approach -- and that the same direction appears in vision, audio, and human-brain encoders never jointly trained. The recipe: embed nine emotion-anchored story sets in a frozen encoder, take the top principal direction of the nine averaged embeddings. Projecting new inputs onto it captures 93% of supervised performance on SST-2 (Llama-3-8B-Instruct, AUC 0.772 vs. 0.828), correlates with human valence ratings on 11,811 EmoSet images at r=0.636, reaches AUC 0.906 on ESC-50 audio (p12). A 2-parameter classifier trained on text labels transfers to images (AUC 0.961), audio (0.764), and brain recordings (0.828) without target-modality labels; a generic 16-D subspace stays at chance (0.525). The recipe is bounded to continuous attributes -- seven tests on categorical concepts return near-chance -- and steering is family-specific (Llama/Mistral yes, Qwen/Gemma no).
AI generation is temporarily unavailable; this entry was preserved with deterministic fallback metadata.
arXiv:2608.18090v1 Announce Type: new Abstract: Inside a modern language model sits a single internal direction that tracks how positive or negative a sentence feels. We show how…
arXiv:2608.18089v1 Announce Type: new Abstract: Instruction-tuned models often refuse harmful requests in English but comply with the same requests in Yoruba, Igbo, Igala, and Hausa. This suggests that the refusal mechanism is present in the residual stream but fails to activate for low-resource inputs. Recovering it normally requires labelled target-language data and retraining, neither of which is available at scale for most African languages. We introduce Latent Space Refusal Anchoring (LSR-Anchoring), a training-free method that extracts the refusal direction from English prompts and clamps it onto the residual stream at inference time. The primary variant, Mean-Activation Steering (MAS), operates across the four architectures we tested: Llama-3-8B, Llama-3.1-70B, Mistral-7B-Instruct, and Qwen2.5-7B. On Mistral and Qwen it recovers safety with benign degradation below 0.08. On Llama-3-8B it overcorrects, with Degraded Performance on Legitimate prompts (DPL) reaching 1.00. We address this with SAE-Derived Steering (SDS), which replaces the dense mean-difference direction with a single Sparse Autoencoder (SAE) feature and reduces Kullback-Leibler (KL) divergence by 3.5-7x without benign collapse. Four languages transfer positively, but Arabic fails on every architecture and at every steering magnitude, indicating a geometric mismatch rather than a baseline effect. Massive Multitask Language Understanding (MMLU) accuracy drops remain below 0.35 percentage points at every effective steering magnitude.
AI generation is temporarily unavailable; this entry was preserved with deterministic fallback metadata.
arXiv:2608.18089v1 Announce Type: new Abstract: Instruction-tuned models often refuse harmful requests in English but comply with the same requests in Yoruba, Igbo, Igala, and Hau…
An end-to-end tutorial for supervised fine-tuning of tool-calling LLMs, covering trajectory parsing, structured tool-call extraction, Qwen-compatible ChatML rendering, and LoRA fine-tuning of Qwen3-0.6B on the XYZ-Aquila-SFT dataset.
End-to-end SFT pipeline for tool-calling LLMs using XYZ-Aquila-SFT and Qwen3-0.6B.
Parses multi-turn trajectories, extracts structured tool calls, and preserves reasoning and observation patterns.
Mistral AI wants to turn European AI sovereignty from a talking point into a product — one with a service-level agreement attached. The French artificial intelligence company announced Tuesday a three-part expansion of…
AI generation is temporarily unavailable; this entry was preserved with deterministic fallback metadata.
Mistral AI wants to turn European AI sovereignty from a talking point into a product — one with a service-level agreement attached. The French artificial intelligence company anno…
arXiv:2608.11210v1 Announce Type: new Abstract: Bayesian calibration of process-based models requires a prior distribution for each model parameter. Despite decades of methodological work, researchers almost always fall back on uniform priors. The main reason is that building informative priors from scientific literature is slow and needs both domain and statistical expertise. We present \textbf{Distribird}, an agentic web application that automates this process. Given a parameter name, physical description, and domain context, Distribird deploys a multi-agent pipeline that searches the literature, extracts and weights reported values by domain relevance, and fits a probability distribution via AIC model selection. When no literature is available, the system falls back to sensible uninformative alternatives, and clearly reports both the evidence behind and the confidence level of every prior it produces. It is designed for the problems where the models have physically interpretable parameters, where domain knowledge exists in the published literature. We evaluate the tool on 24~parameters across 10 scientific domains comparing three open-weight models (Qwen3.6 27B, Gemma 4 31B, Mistral Small 4 119B) with a single-prompt LLM baseline. On prior quality the full pipeline \emph{matches} this baseline. Every prior is traced to the specific papers and values from which it was constructed; a built-in validity layer declines to produce priors for out-of-scope requests, whereas the single-prompt baseline returns confident but unfounded priors for them in 11 of 30~model--parameter cases; and every language-model call runs locally, so no parameter description or unpublished modelling detail is transmitted to a third-party LLM provider (only generated search terms reach the public literature databases). For scientific use, we argue these properties matter more than a marginal improvement in point-estimate accuracy.
AI generation is temporarily unavailable; this entry was preserved with deterministic fallback metadata.
arXiv:2608.11210v1 Announce Type: new Abstract: Bayesian calibration of process-based models requires a prior distribution for each model parameter. Despite decades of methodologi…
That’s a lot of people chatting with their AI friends all day. | Image: Google For the 14th time, a Google product has hit 1 billion users. Google CEO Sundar Pichai posted on X that a billion people are using Gemini every month, and that Gemini is Google's fastest-growing product ever. A billion users is a huge milestone, but Google isn't the first AI app to hit it. OpenAI's ChatGPT hit the mark a few weeks ago, though the company buried the announcement that "more than 1 billion people are putting ChatGPT to work" in an otherwise anodyne blog post about how people use AI. External data suggested ChatGPT crossed 1 billion users as early as this June, but OpenAI hadn't announced anything until that post on August 6th. … Read the full story at The Verge.
AI generation is temporarily unavailable; this entry was preserved with deterministic fallback metadata.
That’s a lot of people chatting with their AI friends all day. | Image: Google For the 14th time, a Google product has hit 1 billion users. Google CEO Sundar Pichai posted on X th…
Mistral AI has released Shieldstral 1.0 3B, an open-weights, policy-adaptive multimodal safety classifier that frames content moderation as a single yes/no question instead of a fixed harm taxonomy. Operators supply the policy as a plain-language query at inference time and get back a calibrated safety score from one forward pass — no retraining required to re-target the model. Built on Ministral-3-3B-Base-2512 with a Pixtral vision encoder and trained on roughly 54.1M samples, it reports 84.9% average F1 on text safety (matching GPT-OSS-Safeguard-20B), 83.8% on multimodal safety, and 91.3% on Mistral's adaptability benchmark — while fitting in 16GB of VRAM under an Apache 2.0 license. The post Mistral AI Releases Shieldstral 1.0 3B: An Open-Weights Policy-Adaptive Multimodal Safety Classifier Matching Models 7× Its Size appeared first on MarkTechPost.
AI generation is temporarily unavailable; this entry was preserved with deterministic fallback metadata.
Mistral AI has released Shieldstral 1.0 3B, an open-weights, policy-adaptive multimodal safety classifier that frames content moderation as a single yes/no question instead of a f…
arXiv:2608.05162v1 Announce Type: new Abstract: Pooling is a consequential but under-examined design choice in decoder-only concept representation work: practitioners must collapse token-level hidden states into a passage-level vector, yet no shared protocol exists for comparing this choice across concepts, models, and tasks. Reported gains are confounded by simultaneous changes in dataset, layer, construction method, and pooling rule, making principled decisions impossible. We introduce PoolBench, a benchmark that isolates pooling as the experimental variable under a fixed evaluation protocol. PoolBench covers 17 concepts, 19 pooling strategies, and 3 open-weight decoder-only models (Llama-3.1-8B, Gemma-2-9B, Mistral-7B), evaluated on a single audited corpus of 37,693 real-text passages. The primary axis is linear separability (D1/AUROC); steered concept prevalence (D2/SCP) and output-level disentanglement (D3) serve as diagnostic axes. The primary finding is decisive: W4_hierarchical reaches a cross-model mean AUROC of 0.7799, while the widely adopted P1_last_token baseline reaches only 0.7640 and is statistically significantly worse (Friedman+Nemenyi, p = 2.0e-36; 77 significant pairs among 18 effective strategies). Rankings are stable across layers (rho = 0.961--0.990). A key negative result: strong detection does not imply strong steering -- D2 and D3 are substantially weaker than D1 for most concepts, indicating a fundamental representational limit rather than a pooling failure. On mid-difficulty concepts, W4_hierarchical outperforms P1_last_token by 0.042--0.113 AUROC; construction method choice (DiffMean vs. REPE) has a larger effect (delta AUROC 0.15) than pooling (delta AUROC 0.016), establishing the correct practical hierarchy. We release the corpus, pre-extracted activations, scorer models, steering vectors, and evaluation code as a reusable protocol for pooling research.
AI generation is temporarily unavailable; this entry was preserved with deterministic fallback metadata.
arXiv:2608.05162v1 Announce Type: new Abstract: Pooling is a consequential but under-examined design choice in decoder-only concept representation work: practitioners must collaps…
arXiv:2608.04130v1 Announce Type: new Abstract: Vision-language models for autonomous driving primarily rely on cameras and LiDAR, leaving 4D radar largely unexplored as a standalone perceptual modality despite its robustness to adverse visibility and direct measurement of radial velocity. We introduce Radar4D-VLM, a radar-only temporal vision-language model that reasons from ten consecutive 4D-radar point-cloud sweeps without camera or LiDAR input. Radar4D-VLM extracts geometrically grounded object proposals and organizes radar evidence into a compact hierarchy of object, scene, and kinematic tokens. A parameter-efficient projector maps these tokens into frozen language backbones, while auditable prediction heads jointly model object count, spatial distribution, motion state, collision risk, semantic category, and radial velocity. Radar4D-VLM combines proposal-grounded temporal object tokenization, global scene context, and explicit kinematic tokens within a unified frozen-backbone interface. On sequence-isolated K-Radar development validation, its Top-64 proposal recall reaches 98.13% at 4 m, exceeding fixed-lattice and uniform-random controls by 6.40 and 22.83 percentage points, respectively. We further evaluate 24 matched runs spanning eight frozen Qwen, Phi, Mistral, Llama, and Gemma backbones under an identical adaptation budget. The radar-token interface remains compatible across all five language-model families, while matched aligned, permuted, and no-language controls show sensor dependence but no stable direct-head gain from aligned language supervision. These results establish a reproducible foundation for radar-only multimodal scene and motion reasoning while separating interface compatibility from the benefit of language supervision.
AI generation is temporarily unavailable; this entry was preserved with deterministic fallback metadata.
arXiv:2608.04130v1 Announce Type: new Abstract: Vision-language models for autonomous driving primarily rely on cameras and LiDAR, leaving 4D radar largely unexplored as a standal…
<p>I released <a href="https://llm.datasette.io/en/stable/changelog.html#v0-32">LLM 0.32</a> this morning, the most significant new version of LLM since the initial launch of the project. The new version includes support for visible reasoning traces, server-side provider tools, redesigned content-addressable SQLite logs, new models, and new features enabled by the OpenAI Responses API. I also released new versions of the <code>llm-anthropic</code>, <code>llm-gemini</code>, and <code>llm-openrouter</code> plugins, each with substantial updates of their own.</p> <h4 id="headline-features-for-llm-cli-users">Headline features for LLM CLI users</h4> <p>Running LLM against reasoning models now <strong>displays their reasoning traces</strong> to standard error, so you can see what they are "thinking" without that information being included in the standard output that you might pipe to another tool. Add <code>-R/--hide-reasoning</code> to turn this off.</p> <p><img src="https://static.simonwillison.net/static/2026/best-pelicans.gif" alt="Running llm "think about the best thing about pelicans" in the macOS terminal window - grey text outputs saying Exploring pelican qualities, then after a paragraph of that a white paragraph of text comes out saying: The best thing about pelicans is their wonderfully oversized, practical design: that enormous bill and pouch look comical, but they make pelicans remarkably skilled fishers. Even better, many species cooperate—working together to herd fish before scooping them up. They’re a great mix of goofy, graceful, and surprisingly clever." style="max-width: 100%;" /></p> <p>LLM includes support out-of-the-box for the <strong>GPT-5.6 model family</strong>, and the new default model used with <code>llm "prompt"</code> is now the inexpensive but capable <strong>GPT-5.6 Luna</strong>.</p> <p>LLM calls can now use <strong>server-side tools</strong> from various providers. OpenAI provide <a href="https://llm.datasette.io/en/stable/openai-models.html#code-interpreter">a code execution environment</a> as a server-side tool; LLM can now run prompts that benefit from that like so:</p> <div class="highlight highlight-source-shell"><pre>llm --tool CodeInterpreter <span class="pl-s"><span class="pl-pds">'</span>Show current python and SQLite versions<span class="pl-pds">'</span></span></pre></div> <p>OpenAI also gets a <a href="https://llm.datasette.io/en/stable/openai-models.html#web-search">WebSearch</a> tool.</p> <p>The <a href="https://github.com/simonw/llm-anthropic">llm-anthropic</a> plugin adds <a href="https://github.com/simonw/llm-anthropic/blob/0.26/README.md#web-search">WebSearch</a>, <a href="https://github.com/simonw/llm-anthropic/blob/0.26/README.md#web-fetch">WebFetch</a>, <a href="https://github.com/simonw/llm-anthropic/blob/0.26/README.md#code-execution">CodeExecution</a>, and <a href="https://github.com/simonw/llm-anthropic/blob/0.26/README.md#mcp-connector">AnthropicMCP</a>, which looks like this:</p> <div class="highlight highlight-source-shell"><pre>llm -m claude-sonnet-5 -T <span class="pl-s"><span class="pl-pds">'</span>AnthropicMCP("https://datasette.simonwillison.net/-/mcp")<span class="pl-pds">'</span></span> \ <span class="pl-s"><span class="pl-pds">'</span>how many rows in the blog_blogmark table?<span class="pl-pds">'</span></span></pre></div> <p>That causes Anthropic to execute MCP calls against my new <a href="https://simonwillison.net/2026/Jul/31/stateless-mcp/#datasette-mcp">datasette-mcp</a> plugin as part of a single request/response interaction with their API.</p> <p>The new <strong>llm openai endpoint</strong> command provides a tool for <a href="https://llm.datasette.io/en/stable/other-models.html#run-against-an-endpoint-without-configuring-it">executing prompts against <em>any</em> OpenAI compatible endpoint</a> as a one-liner. These aren't logged, which makes this a handy tool for running one-off prompts against anything that speaks the lingua franca of the LLM API world.</p> <p>Here's how I use that to run prompts against Gemma 4 12B running in my localhost <a href="https://lmstudio.ai">LM Studio</a> API, via <code>uvx</code> (no LLM installation required) and mixing in the <a href="https://github.com/simonw/llm-tools-quickjs">llm-tools-quickjs</a> tool plugin for good measure:</p> <div class="highlight highlight-source-shell"><pre>uvx --with llm-tools-quickjs \ llm openai endpoint http://localhost:1234/v1 -m google/gemma-4-12b \ -T QuickJS <span class="pl-s"><span class="pl-pds">'</span>Use QuickJS to multiply 3434 * 2434<span class="pl-pds">'</span></span> --td</pre></div> <p><img src="https://static.simonwillison.net/static/2026/openai-endpoint-gemma.webp" alt="Output reads Tool call: QuickJS_execute_javascript({'javascript': '3434 * 2434'}) 8358356 The result of 3434 * 2434 is 8,358,356." style="max-width: 100%;" /></p> <h4 id="new-features-in-the-python-api">New features in the Python API</h4> <p>LLM's Python API previously required you to create a conversation and then send messages to it one at a time. This was an abstraction over the true nature of LLMs, where each request carries a complete history of the messages that came before it. That abstraction started to get in the way for some more advanced cases, so the new release introduces a <code>model.prompt(messages=[])</code> parameter that can be used like this:</p> <pre><span class="pl-k">import</span> <span class="pl-s1">llm</span> <span class="pl-k">from</span> <span class="pl-s1">llm</span> <span class="pl-k">import</span> <span class="pl-s1">user</span>, <span class="pl-s1">assistant</span>, <span class="pl-s1">system</span> <span class="pl-s1">model</span> <span class="pl-c1">=</span> <span class="pl-s1">llm</span>.<span class="pl-c1">get_model</span>(<span class="pl-s">"gpt-5.6-luna"</span>) <span class="pl-s1">response</span> <span class="pl-c1">=</span> <span class="pl-s1">model</span>.<span class="pl-c1">prompt</span>(<span class="pl-s1">messages</span><span class="pl-c1">=</span>[ <span class="pl-en">system</span>(<span class="pl-s">"You are a helpful pirate."</span>), <span class="pl-en">user</span>(<span class="pl-s">"What is the capital of France?"</span>), <span class="pl-en">assistant</span>(<span class="pl-s">"Paris, matey."</span>), <span class="pl-en">user</span>(<span class="pl-s">"And Germany?"</span>), ]) <span class="pl-en">print</span>(<span class="pl-s1">response</span>.<span class="pl-c1">text</span>())</pre> <p>LLM previously returned an iterable sequence of strings from each prompt. This worked great when models returned a string response, but failed to predict the weird shape that models would evolve towards. Today many models return a mix of reasoning text, output strings, tool calls, and even image attachments. With LLM 0.32 you can <a href="https://llm.datasette.io/en/stable/python-api.html#structured-messages-and-streaming-events">do this instead</a>:</p> <pre><span class="pl-k">for</span> <span class="pl-s1">event</span> <span class="pl-c1">in</span> <span class="pl-s1">model</span>.<span class="pl-c1">prompt</span>(<span class="pl-s">"Explain cats"</span>).<span class="pl-c1">stream_events</span>(): <span class="pl-k">if</span> <span class="pl-s1">event</span>.<span class="pl-c1">type</span> <span class="pl-c1">==</span> <span class="pl-s">"reasoning"</span>: <span class="pl-en">print</span>(<span class="pl-s">f"[thinking] <span class="pl-s1"><span class="pl-kos">{</span><span class="pl-s1">event</span>.<span class="pl-c1">chunk</span><span class="pl-kos">}</span></span>"</span>, <span class="pl-s1">end</span><span class="pl-c1">=</span><span class="pl-s">""</span>, <span class="pl-s1">flush</span><span class="pl-c1">=</span><span class="pl-c1">True</span>) <span class="pl-k">elif</span> <span class="pl-s1">event</span>.<span class="pl-c1">type</span> <span class="pl-c1">==</span> <span class="pl-s">"text"</span>: <span class="pl-en">print</span>(<span class="pl-s1">event</span>.<span class="pl-c1">chunk</span>, <span class="pl-s1">end</span><span class="pl-c1">=</span><span class="pl-s">""</span>, <span class="pl-s1">flush</span><span class="pl-c1">=</span><span class="pl-c1">True</span>) <span class="pl-k">else</span>: <span class="pl-en">print</span>(<span class="pl-s">f"Other event: <span class="pl-s1"><span class="pl-kos">{</span><span class="pl-s1">event</span><span class="pl-kos">}</span></span>"</span>)</pre> <p>Combine these features and we can <em>finally</em> provide a robust implementation of the semi-standard OpenAI chat completions API, which I've now released as the <a href="https://github.com/simonw/llm-chat-completions-server">llm-chat-completions-server</a> plugin:</p> <div class="highlight highlight-source-shell"><pre>llm install llm-chat-completions-server llm chat-completions-server --port 9000 <span class="pl-c"><span class="pl-c">#</span> Server is now running on http://127.0.0.1:9000/v1</span></pre></div> <p>Now you can run prompts against LLM via that server, using the new <code>llm openai endpoint</code> command!</p> <div class="highlight highlight-source-shell"><pre>llm openai endpoint http://127.0.0.1:9000/v1 <span class="pl-s"><span class="pl-pds">'</span>hello<span class="pl-pds">'</span></span> -m gpt-5.4-mini</pre></div> <p>The bigger challenge with that kind of API concerns logging. If we're going to support the pattern where the message sequence is appended to on every request, ideally we can avoid logging all of that duplicate JSON for every turn.</p> <p>The solution is the new <a href="https://llm.datasette.io/en/stable/logging.html#the-message-store">content-addressable message store</a>, modeled after Git. You can see the new schema for that <a href="https://llm.datasette.io/en/stable/logging.html#sql-schema">in the documentation</a>, but the <code>llm logs</code> and <code>llm logs --json</code> commands have both been upgraded to convert that format back into something that's easy to consume.</p> <h4 id="and-the-rest">And the rest</h4> <p>There is a whole lot more in this release. The <a href="https://llm.datasette.io/en/stable/changelog.html#v0-32">0.32 release notes</a> are pretty comprehensive, and the notes for <a href="https://llm.datasette.io/en/stable/changelog.html#rc2-2026-07-30">0.32rc2</a>, <a href="https://llm.datasette.io/en/stable/changelog.html#rc1-2026-07-30">0.32rc</a>, <a href="https://llm.datasette.io/en/stable/changelog.html#a3-2026-06-09">0.32a3</a>, <a href="https://llm.datasette.io/en/stable/changelog.html#a2-2026-05-12">0.32a2</a>, and <a href="https://llm.datasette.io/en/stable/changelog.html#a0-2026-04-28">0.32a0</a> should fill in any gaps.</p> <p>Existing LLM plugins should all continue to work, but plugins that provide extra models will need to be upgraded to 0.32 in order to participate fully in the new streaming events system. There's a guide to implementing plugins with <a href="https://llm.datasette.io/en/stable/plugins/advanced-model-plugins.html#structured-messages-and-streaming-events">Structured messages and streaming events</a> in the documentation.</p> <p>I've updated some of my own plugins:</p> <ul> <li> <a href="https://github.com/simonw/llm-anthropic/releases/tag/0.26">llm-anthropic 0.26</a> adds support for the Claude 5 family of models, plus <code>WebSearch</code>, <code>WebFetch</code>, <code>CodeExecution</code>, and <code>AnthropicMCP</code> server-side tools.</li> <li> <a href="https://github.com/simonw/llm-gemini">llm-gemini</a> and <a href="https://github.com/simonw/llm-openrouter">llm-openrouter</a> and <a href="https://github.com/simonw/llm-mistral">llm-mistral</a> are nearly there, releases coming soon.</li> </ul> <h4 id="i-guess-llm-is-an-agent-framework-now">I guess LLM is an agent framework now</h4> <p>Quite a few of the lower-level tools changes in this release were driven by the needs of <a href="https://agent.datasette.io/">Datasette Agent</a>. When I started work on LLM, the term "agent" had such a vague definition that I refused to use it. In <a href="https://simonwillison.net/2025/Sep/18/agents/">September 2025</a> I came around to the idea that "<strong>An LLM agent runs tools in a loop to achieve a goal</strong>" is well established enough now that I could stop avoiding the term entirely.</p> <p>Tool chains can now <a href="https://llm.datasette.io/en/stable/python-api.html#python-api-tools-pause">pause for human approval</a> and <a href="https://llm.datasette.io/en/stable/python-api.html#python-api-tools-resume">resume from a stored message history</a> - both needed by Datasette Agent.</p> <p>Looking at LLM today it's beginning to look very agent-shaped to me. There's something neat about having a CLI utility that can mix and match different tools from different sources with different models all as a one-liner, and that includes a Python library powerful enough to build systems like <a href="https://agent.datasette.io/">Datasette Agent</a> and <a href="https://github.com/simonw/llm-coding-agent">llm-coding-agent</a>.</p> <p>Maybe the next version of LLM will bake the concept of an "agent" into the core library. I'm still trying to figure out what that would look like.</p> <p>Tags: <a href="https://simonwillison.net/tags/projects">projects</a>, <a href="https://simonwillison.net/tags/releases">releases</a>, <a href="https://simonwillison.net/tags/ai">ai</a>, <a href="https://simonwillison.net/tags/openai">openai</a>, <a href="https://simonwillison.net/tags/generative-ai">generative-ai</a>, <a href="https://simonwillison.net/tags/llms">llms</a>, <a href="https://simonwillison.net/tags/llm">llm</a>, <a href="https://simonwillison.net/tags/anthropic">anthropic</a></p>
AI generation is temporarily unavailable; this entry was preserved with deterministic fallback metadata.
<p>I released <a href="https://llm.datasette.io/en/stable/changelog.html#v0-32">LLM 0.32</a> this morning, the most significant new version of LLM since the initial launch of the…
The AI Model Hub is a central platform for comparing open-source large language models from leading developers like Meta, Alibaba, Google, and Mistral. It provides detailed specifications for over 100 active models, including context windows, architectures, parameter counts, licenses, and benchmarks.
The AI Model Hub aggregates 100 active open-source LLMs.
Models come from developers including Meta, Alibaba, Google, Mistral, Microsoft, and DeepSeek.
This paper presents a two-part contribution for large-scale chatbot validation: a methodology for creating high-fidelity synthetic customer agents (SCAs) as digital twins, and an SCA-based validation framework combining automated LLM-as-a-Judge evaluation, human expert testing, and adversarial probing. The approach was validated at a leading UK bank, providing a scalable pathway toward regulatory compliance.
Introduces methodology for creating high-fidelity synthetic customer agents (SCAs) as digital twins from real transactional and conversational data.
SCAs achieve high semantic alignment, low hallucination rates, and controllable personality trait reproduction.
An AI employee is an agent with a persistent workspace and the tools to complete assigned work end to end, far beyond a simple chat assistant. Construct is a work OS that provides files, memory, schedules, and workflows for such agents.
AI employees can accept outcomes, choose tools, complete multi-step jobs, and preserve files and context for later.
Construct combines persistent files, inspectable memory, schedules, and native inbox with web research, browser, terminal, and apps.
A new study benchmarks the performance cost of enabling confidential computing for LLM inference on an NVIDIA H100 GPU under Intel TDX. Using Mistral-7B and Qwen3-30B-A3B models, results show a 21.8%-27.8% increase in time-to-first-token and 17.7%-21.1% drop in global token throughput in confidential mode. The larger model reaches saturation earlier, highlighting the need for capacity planning adjustments.
Confidential computing is becoming a practical requirement for AI inference but introduces performance overhead.
The study tests two LLMs on an H100 GPU within an Intel TDX confidential instance.
AI Maestro orchestrates AI coding agents to work on a task board, turning software delivery into a coordinated multi-agent pipeline rather than a single chat session.
Board-based workflow ensures work survives context resets and parallel sessions.
Each ticket specifies its own agent pipeline and model for optimal task-model matching.
NVIDIA Vera Rubin NVL72 production is ramping up with partners CoreWeave, Google Cloud, Microsoft Azure and Oracle Cloud Infrastructure. The platform delivers highest performance per watt and lowest token cost, with 10x more throughput per megawatt than Grace Blackwell NVL72 in benchmarks. It also powers Europe's open-model era through a partnership between Microsoft and Mistral.
Vera Rubin NVL72 production ramping with 350+ factory sites in 30 countries
10x more tokens per megawatt and 1/10th cost per million tokens vs. previous gen
A single 24GB GPU is the practical floor for serious local inference. This guide compares six open-weight models that fit one card at Q4_K_M, including Qwen3.6, Gemma 4, Mistral Small, gpt-oss-20b, and DeepSeek-R1-Distill. It covers VRAM fit, licensing, and the job each does best.
24GB is the practical floor: run right-sized 20B–35B models, not the biggest 70B quant you can squeeze in.
Qwen3.6-27B is the strongest all-around default; DeepSeek-R1-Distill-Qwen-32B is the tightest fit at ~18–20GB.
This comparison scores four leading AI coding agents—Mistral Vibe for Code, Claude Code, Cursor, and OpenAI Codex—on a real scaffold-to-PR workflow. Mistral Vibe leads with 22/25, driven by low cost, open weights, and self-hosting options. Claude Code and Codex tie at 21/25, while Cursor scores 16/25. The article details each tool's strengths and weaknesses across five dimensions: feature scaffolding, test generation, PR/async workflow, surface coverage, and cost/openness.
Mistral Vibe for Code scores highest (22/25) with low price, open-source CLI, and self-hosting.
Claude Code and OpenAI Codex tie at 21/25; Claude leads in raw coding quality; Codex excels in cross-surface async.
Mistral AI introduces a vision model that enables robots to navigate unknown environments using only a single RGB camera and natural language instructions.
Mistral AI introduced Robostral Navigate, an 8B embodied navigation model. It moves robots from a plain-language instruction using only a single RGB camera, with no LiDAR or depth sensors. The model reaches 76.6% success on R2R-CE validation unseen through a pointing method, prefix-caching training, and CISPO online reinforcement learning.
Robostral Navigate is Mistral AI's first 8B model for embodied navigation.
Achieves 76.6% success on R2R-CE validation unseen using only a single RGB camera.
This paper explores machine learning for automatic thematic indexing of large literary corpora, using Voltaire's works as a test case. The best model, a 4-bit quantized Mistral, achieves F1 scores up to 0.67, highlighting the potential of automated indexing.
Thematic indexing is crucial for scholarly access but remains labor-intensive. This study applies ML to automate it using Voltaire's 'Essai sur les mœurs' and 'Questions sur l'Encyclopédie'.
The task is framed as multi-label classification. Models range from encoders to fine-tuned LLMs (3–120B parameters).
A personal, non-benchmark tier list of AI models for coding and auditing as of mid-2026, covering Anthropic Fable, OpenAI Sol, Mistral, Gemini, and DeepSeek, with commentary on US export controls and European perspectives.
Fable (Anthropic) gets a B: fluent but unreliable, prone to hiding bugs.
Sol (OpenAI) gets an S: trustworthy for low-level code and testing.
AnyFile Translator is an AI-powered assistant for Google Chat that translates documents, web links, and messages while preserving original formatting. It supports over 100 languages, offers AI content writing, and ensures data privacy with encryption and deletion.
Translate files (PDF, Word, PPT, etc.) while preserving layout
Supports over 100 languages and works within Google Chat
This post walks you through building a production-ready ecommerce MCP server using Amazon Bedrock AgentCore and Mistral AI Studio. It covers MCP tool implementation, two-layer JWT authentication, AWS CDK deployment, integration with Mistral AI's Vibe, and best practices for data and identity management with DynamoDB and Cognito.
Leverage AgentCore Runtime to host MCP servers without managing containers or load balancers.
Implement two-layer authentication: infrastructure-level JWT validation and application-level identity resolution.
This paper presents a workload-aware benchmark of KV-cache optimization techniques including KIVI, TurboQuant, SnapKV, and CaM, evaluated on Llama-3.1-8B-Instruct and Mistral-7B-Instruct-v0.3 models across multi-document QA, single-document QA, few-shot learning, and summarization tasks. Results show that compression ratio alone is a poor predictor of end-to-end performance. KIVI4 offers the most stable quality across models, SnapKV delivers the strongest long-context throughput, and CaM yields large gains on selected QA workloads but exhibits substantial workload sensitivity. The study motivates workload-aware selection of KV-cache mechanisms.
KIVI4 provides the most stable quality across models.
Mistral AI released Leanstral 1.5, a free Apache-2.0 code agent model for Lean 4. It saturates miniF2F and solves 587 of 672 PutnamBench problems. The 119B mixture-of-experts activates 6.5B parameters per token. We break down its architecture, benchmarks, real bug-finding case studies, and deployment code.
Leanstral 1.5 is a free, Apache-2.0 licensed Lean 4 proof-engineering model.
It uses a 119B mixture-of-experts with 6.5B active parameters and 256k context.
Wiola is a fully original small language model architecture built from first principles, unrelated to existing families like GPT, LLaMA, Mistral, or Falcon. It introduces five novel components: Spiral Rotary Positional Encoding (SRPE), Gated Cross-Layer Attention (GCLA), Adaptive Token Merging (ATM), Dual Stream Feed-Forward (DSFF), and WiolaRMSNorm. Released in four sizes (120M to 1.5B parameters), it is fully compatible with HuggingFace Transformers.
Wiola is a fully original SLM architecture with no lineage from existing model families.
Five novel components: SRPE, GCLA, ATM, DSFF, and WiolaRMSNorm.
This paper formalizes the LLM Jury under the Huber contamination model and shows that PoLL incurs unbounded bias under any positive contamination if a single judge fails in a biased, LLM-typical way. By framing jury consensus as robust mean estimation, the authors propose RoPoLL using the geometric median as aggregation, achieving optimal breakdown point 1/2. Experiments across 13 judges, three benchmarks, and four corruption regimes show RoPoLL dominates PoLL on every biased corruption type, with a 3-judge committee at 38B outperforming Mistral-Large-3 (675B) by 1.31x under 30% bimodal-random corruption.
PoLL (Panel of LLM Evaluators) incurs unbounded bias under any positive contamination when a single judge exhibits typical LLM biases.
RoPoLL replaces the aggregation function with the geometric median, achieving optimal robustness with breakdown point 1/2.
Bored People Chat is a minimal, anonymous global chat room with no sign-up, ads, or bots. Inspired by old internet chat rooms, it focuses on safety through AI moderation and provides a space for lonely or bored people to connect.
Anonymous, no sign-up, no ads, no bots, single public room
Safety-focused with AI-powered moderation and filtering of personal info
LangChain releases four new test environments to benchmark LLMs' ability to use tools effectively, covering function calling, planning, and reasoning. Tests include single-tool and multi-tool typewriter tasks, relational data queries, and a math task with altered rules. Key findings: GPT-4 excels on relational data but fails on longer trajectories; Claude 2.1 matches GPT-4 on three tasks; open-source models like Mistral 7b struggle with multi-step function composition; planning remains challenging for all models.
LangChain introduces four benchmarks for LLM tool use: Typewriter (single & 26 tools), Relational Data, and Multiverse Math.
GPT-4 scores highest on Relational Data but still fails on simple long-horizon tasks.
Mistral AI released OCR 4 on June 23, 2026, moving from clean text extraction to structured document output. Each block returns a bounding box, a typed classification, and per-page and per-word confidence scores. The model supports 170 languages, runs in a single self-hosted container, and feeds citation-ready inputs into RAG, agentic, and enterprise search pipelines through one API endpoint.
OCR 4 returns bounding boxes, typed-block labels, and per-word confidence scores, not just text.
Supports 170 languages across 10 groups, with gains on rare and low-resource languages.
Mistral OCR 4 introduces bounding boxes, block classification, and confidence scores. Outperforms competitors in human preference tests, supports 170 languages, and can be self-hosted.
Independent annotators prefer OCR 4 with 72% win rate; top score on OlmOCRBench (85.20).
A year after announcing ambitious AI plans at NVIDIA GTC Paris, France’s AI infrastructure is coming online: AI agents are in production, startups are deploying applications, and the ecosystem is developing models and platforms tailored to local languages and European requirements. Key developments include Mistral’s new data center, open model collaborations via NVIDIA Nemotron, and enterprise AI adoption across healthcare, telecom, automotive, energy, and cosmetics industries.
France’s AI infrastructure is taking shape with Mistral’s 44MW data center and plans for 200MW by 2027. Other investments include Scaleway’s Blackwell instances, Bull/Foxconn production, and a bid for a European AI gigafactory.
Open models are central: NVIDIA Nemotron Coalition partners like Mistral, LINAGORA, H Company, and Pleias are developing models for local languages and EU compliance.
Mistral AI announces a new model coming this summer, the start of a larger family of fat but sparse models. Early access program in July for key partners.
New Mistral AI model arriving summer
Part of a larger family of models described as fat but sparse
A NewsGuard audit found that Mistral AI's chatbot Le Chat repeated false claims about the Iran war 50% of the time in English and 56.6% in French, showing vulnerability to state-sponsored disinformation. The French Ministry of Armed Forces uses a customized version of Le Chat Enterprise, not the free consumer version.
NewsGuard audit: Le Chat repeated false claims 50% in English, 56.6% in French.
False claims from Russian, Chinese, and Iranian state-aligned sources were repeated.
Mistral launches Vibe, a unified AI agent for long-running multi-step work and coding. It integrates with enterprise tools, offers Work Mode and Code Mode, and introduces VS Code extension and CLI updates.
Vibe is Mistral's new AI agent for work and code, replacing Le Chat.
Work Mode handles complex tasks with enterprise search, data analysis, document synthesis, and scheduling.
An Abrams military store in Kyiv has expanded rapidly during the war due to shortcomings in standard issue clothing. Many visitors come seeking more than just apparel, according to Intelligence Online.
Abrams military store expansion due to poor standard gear
Cohere releases North Mini Code, its first open-weight coding model under Apache 2.0, targeting developers who want to own and control their AI infrastructure. The 30B MoE model runs on a single H100 GPU, aiming to compete with Mistral, Qwen, and Gemma on agentic coding tasks.
Cohere launches North Mini Code, a 30-billion-parameter MoE coding model with 3B active parameters, available under Apache 2.0.
The model can run on a single Nvidia H100 GPU, making self-hosting practical for developers.
Tokens 4 Breakfast is a macOS menu bar app for tracking and monitoring token usage, subscriptions, and rate limits across major AI platforms like Claude, OpenAI, Cursor, Copilot, Gemini, DeepSeek, Mistral, and more. It helps developers avoid unexpected overspending with real-time alerts, budgeting, and cross-provider visibility. Free for one provider; Pro one-time $7.99 unlocks all.
Real-time menu bar display of AI spend, rate limits, and subscription costs.
Supports 8 major AI providers including Claude, OpenAI, Cursor, etc.
AI agents are evolving from simple chatbots to autonomous digital workers, raising security and governance concerns. Experts advise treating them like human interns with strict oversight, specific instructions, and careful monitoring to prevent unintended actions. Balancing independence with control is essential.
AI agents require clear constraints and human oversight to prevent unauthorized actions.
The unpredictable nature of agents introduces new security and governance challenges.
AgentCrew is a conversation-first, Markdown-first methodology for agentic coding that turns a single chat session into a team process with role assignments, task routing, quality gates, and human approval. It uses a pure-Bash classifier to identify task type and risk level, supports fast and full lanes, and includes safety rules to prevent agents from auto-merging or bypassing review.
Transforms coding agents from single-context to multi-role team workflow
Implemented with Markdown and shell scripts, no daemon required
Learn how to perform text classification using locally hosted open-source LLMs like Llama 3, Mistral, and Gemma via Ollama and the Scikit-LLM Python library, all without API costs.
Install Ollama and pull open-source LLMs for local use.
Configure Scikit-LLM to route requests to local Ollama endpoint.
ChatGPT has evolved from a simple chatbot into a versatile tool for writing, research, image generation, file analysis, and more. This guide covers getting started, free vs. paid plans, and essential features like web search, deep research, file uploads, app integrations, image creation, GPTs, projects, voice mode, and memory.
ChatGPT is free to use without an account, but paid plans offer higher limits and additional capabilities.
Core features include web search, deep research, file analysis, app connections, image generation, custom GPTs, projects, and voice mode.