Simon Willison published the May 2026 sponsors-only monthly newsletter covering rising AI costs, Anthropic's strong month, disappointing model releases, and the launch of Datasette Agent.

NVIDIA has released Cosmos 3, a unified world foundation model that combines world generation, physical reasoning, and action generation. Based on a Mixture-of-Transformers architecture, it handles multiple input and output modalities including text, image, video, and action. Two sizes are available: Cosmos 3 Nano (8B params) and Cosmos 3 Super (32B params). It integrates with Hugging Face Diffusers and includes several synthetic data generation datasets for physical AI.

Parallax is a new attention mechanism that retains softmax attention and adds a learned covariance correction branch, replacing the per-query conjugate gradient solver of Local Linear Attention (LLA). It doubles arithmetic intensity by reusing the FlashAttention key-value stream and achieves lower perplexity in LLM pretraining at 0.6B and 1.7B scales. However, its gains are heavily dependent on the Muon optimizer, shrinking significantly under AdamW.

Personal agents are exploding in popularity, with open source projects like OpenClaw and Hermes seeing rapid adoption by AI developer communities on GitHub. Built to adapt to individual preferences and workflows, these agents can interact with applications, generate content, automate repetitive processes and manage multi-step tasks — all while running locally on device. Today at NVIDIA GTC Taipei at COMPUTEX, NVIDIA unveiled NVIDIA RTX Spark — a new class of Windows PCs purpose-built for personal agents — alongside a wave of updates that expand local agents across the broader NVIDIA RTX and DGX ecosystems.

ZAPS-DA is a framework that reduces high-frequency action jitter in RL continuous control policies using a decoupled actor trained via supervised imitation of zero-phase filtered targets, achieving significant jitter reduction without phase lag or post-processing.

The ARISTO Hand, a tendon-driven robotic hand, integrates active distal hyperextension with hybrid fingertip sensing to increase pull-out force by 2.76x for thin objects and successfully perform SD card extraction and insertion.

VLM-GLoc presents a hierarchical semantic Monte Carlo Localization method that uses open-vocabulary Vision-Language Models (VLMs) as a unified semantic observation front-end. It addresses global localization challenges in geometrically aliased quasi-static environments like grocery stores and offices. The method benefits from discriminative text features, implicit quality filtering, and permanence reasoning for data augmentation, plus an inverse semantic proposal mechanism. Evaluated in a 3,500 sq. ft. grocery store and a 3,700 sq. ft. lab, it achieves 70% and 74% success rates, outperforming traditional baselines.

CoMo3R-SLAM is the first collaborative monocular dense RGB SLAM system that leverages robust learned feed-forward 3D reconstruction priors for outdoor multi-agent mapping. Each agent runs a prior-guided front-end for real-time tracking and local dense fusion, while a coordinator performs dense pointmap matching, closed-form Sim(3) gauge synchronization, and GPU-accelerated global bundle adjustment with segment-level depth optimization. Requiring neither depth sensors nor parametric intrinsics, the system produces robust cross-agent constraints and globally consistent metric maps from monocular RGB alone. On Tanks and Temples and Waymo sequences, CoMo3R-SLAM achieves the best ATE on three of four Tanks and Temples scenes and competitive Waymo accuracy, matching or exceeding state-of-the-art RGB-D methods while running online at 8 FPS.

ELAN4D is a training framework for robotic manipulation that enhances VLA policies with future robot keypoint tracks as predictive spatio-temporal supervision, using only forward kinematics and a plug-and-play auxiliary branch discarded at inference. Experiments across multiple benchmarks and real-world tasks show consistent improvements under out-of-distribution perturbations.

This paper presents a learning-based navigation framework for indoor mobile robots that combines a supervised neural global planner with a learning-based DWA local planner refined by PPO. Experiments in simulation and real environments show feasible global routes and reliable local motion commands for safe obstacle avoidance. Source code will be released.

This study uses a real-world transport-and-mapping task with 10 physical robots to show that restructuring communication from fully connected to modular hierarchical interactions improves normalized performance by 47 points, while doubling neural network hidden size yields at most 9 points. Nested mixed-effects models confirm topology's larger impact. Performance saturates beyond 1024 hidden units in simulation-calibrated extrapolation. Results suggest interaction structure can dominate gains within tested settings, but broader generalization remains unestablished.

This paper presents a privacy-by-design pipeline for visual monitoring that confines all inference to the edge device. Using YOLOv5n-seg on a Raspberry Pi 5 with Hailo-8L accelerator, raw pixel buffers are discarded immediately after inference. A stateful trigger engine sends minimal JSON event payloads to a locally hosted Phi-3 Mini LLM, which generates natural language alerts. No image data ever leaves the device, ensuring GDPR compliance.

Researchers propose GCSER-UNet, a deep neural network that integrates global context and attention mechanisms for precise brain tumor segmentation from multimodal MRI. It achieves 94% Dice score on TCGA LGG and 95%/92%/90% on BraTS 2020 for Whole Tumor, Tumor Core, and Enhancing Tumor, surpassing state-of-the-art results.

This study introduces an Arctic-focused remote sensing foundation model combining diversity-aware image curation with masked autoencoder (MAE) pretraining on a Vision Transformer. It achieves 5-8% F1 improvement over ImageNet baseline on four Arctic datasets and outperforms Prithvi-EO-2.0 by at least 15%, demonstrating the value of domain-specific pretraining.

Dex2HOI is a unified diffusion model for generating single- and two-object human-object interaction from text. It employs a Dual-Stream Diffusion approach with bidirectional cross-attention, a Motion Fusion Network with hand-relative object representations, and autoregressive sampling for real-time arbitrary-length sequence generation, achieving up to 540x speedup over prior methods.

The paper proposes a post-processing framework that fuses an input image with its GenAI-enhanced counterpart to preserve perceptual enhancements while enforcing structural faithfulness, effectively mitigating spatial misalignment, texture distortion, and content hallucination. Experiments show it better preserves aesthetic quality and pixel-level structural consistency.

Recent progress in video diffusion models has enabled remarkable generative fidelity, yet leveraging these priors for restoration remains limited by the strong coupling between conditional and unconditional branches in standard classifier-free guidance. We introduce a training-free framework that enhances distorted and low-resolution videos by decoupling these signals in time. Our proposed Decoupled Time Guidance (DTG) evaluates the unconditional branch at a cleaner diffusion timestep, providing a lookahead prior that preserves geometry while suppressing replication of warped content. This temporal bias is annealed throughout sampling, allowing the model to transition from structure correction to detail refinement without retraining. Combined with any off-the-shelf restoration module in a plug-and-play manner, our approach improves perceptual coherence and restores plausible structure in AIgenerated and real-world videos alike. To facilitate evaluation, we curate GenWarp480, a benchmark of 4,400 distorted 480p videos synthesized from diverse text-to-video models. GenWarp480 focuses on characteristic generative degradations such as warped faces, body misalignments, and spatial artifacts, providing a purpose-built testbed for assessing robustness to generative errors. Extensive experiments demonstrate that our method achieves significant improvements in structural fidelity and temporal stability without any model training.

SANA-Streaming is a system-algorithm co-designed framework for high-resolution real-time streaming video editing on consumer GPUs. It features a Hybrid Diffusion Transformer, Cycle-Reverse Regularization, and efficient system co-design, achieving 24 FPS at 1280x704 on an RTX 5090. Experiments show significant improvements in temporal coherence and throughput over state-of-the-art.

The paper proposes SURGE, a knowledge distillation framework that transfers relational geometry from a teacher to a student detector using contrastive InfoNCE in a shared embedding space. It achieves up to 6.2 mAP and 8.0 AP75 gains on two-stage detectors, surpassing teacher performance on SSDD and HRSID benchmarks.

This paper introduces CanLegalRAGBench, a Canadian legal QA benchmark based on realistic queries and expert-annotated answers grounded in case law. Evaluation shows retrieval performance is sensitive to design choices, open-source embedding models are competitive with closed-source, but automatic evaluations have limitations and generated answers often hallucinate or diverge. The benchmark aims to drive progress in legal RAG systems.

This study investigates how LLMs exhibit cultural bias by favoring global narratives over local contexts, introducing the concept of 'global narrative dominance'. Using the CulturalNB dataset for Bangla, they find that English questions increase global substitution and institutional framing, reducing local perspective coverage. Local evidence improves consistency but does not eliminate epistemic shifts.

A first systematic bias evaluation of multimodal speech recognition reveals significant accuracy disparities when different faces are paired with the same audio, with word error rate drops of up to 4.05 points across gender and ethnicity intersections. The study warns that adding modalities can introduce new biases.

This paper presents a zero-shot multi-label topic classification framework enhanced with per-article knowledge graphs, and systematically evaluates eight methods across fifteen LLMs and eight datasets. Keyword-enhanced classification (AK) is the best base method. Graph augmentation helps small models but hurts large ones, indicating large models already contain sufficient relational knowledge. Self-consistency decoding shows no benefit while increasing computation costs fivefold.

This study investigates whether internal signals for figurative language generation in multilingual LLMs are language-specific or cross-lingually reusable. Using activation steering, the authors estimate a direction from figurative-literal activation differences in one language and apply it to generation. Across five figurative categories, six languages, and four LLMs, these directions steer reliably within the same language, especially for metaphor and simile. More importantly, they transfer across languages: a direction learned in one language increases the target behavior in another, with German being most receptive. Combinations of directions from multiple languages can match or exceed native directions, while removing shared components weakens steering. These findings provide direct evidence of a reusable but target-dependent cross-lingual signal for figurative generation.

A new study uses historical cosmology as a controlled experiment to investigate how domain adaptation reshapes explanatory behavior in language models. Phase 1 trains a small model from scratch on a pre-Copernican corpus; Phase 2 fine-tunes a larger model with QLoRA. Results show domain adaptation primarily shifts explanatory framing (premodern vs modern) rather than directly altering cosmological stance, suggesting stance changes are secondary to framework shifts.

Large Language Models (LLMs) struggle to adapt to specialized domains without high-quality domain-specific data. This paper introduces Autonomous Agentic Data Engineering, where LLMs act as autonomous data engineers to plan, generate, and iteratively optimize training data. GPT-5.2 achieved a 57.29% improvement in a student model through agent-driven data adaptation.

We present a protocol to evaluate ChatGPT's ability to generate disease-centric biomedical associations. It outlines how we generate the associations, validate the biological entities using biomedical ontologies, and verify associations using literature. The protocol includes a self-consistency strategy to assess generative reliability across ChatGPT models. To address ontology exact-match limitations, we provide a use case performing semantic verification through a workflow enabled by Retrieval-Augmented Generation (RAG) powered by open-source large language models (LLMs). This enables LLMs to establish truth over content generated by other LLMs and expose hallucination.

This paper formalizes calibration for probabilistic label ranking, developing a hierarchy of notions covering full, sub-, and top-k rankings. It proves full-rank calibration implies others but not conversely. Empirical results show popular label ranking models are poorly calibrated, and calibration correlates with RLHF reward model accuracy beyond top-1.

Existing benchmarks mostly evaluate isolated or short interactive tasks, failing to test agents' ability to track evolving analytical context over long horizons. Researchers introduce LongDS, a benchmark with 68 tasks from real Kaggle notebooks spanning 2,225 turns across six domains. Evaluation shows best model achieves only 48.45% average accuracy, performance drops 47 points from early to late turns, and long-horizon errors account for 52%-69% of failures. Additional steps don't necessarily improve performance; key bottleneck is maintaining correct analytical state.

NumLeak is a measurement framework to detect memorization of public numeric benchmarks in foundation models. Top LLMs recall exact values with high Pearson r (0.97-0.99) on financial and economic data, but performance collapses on recent holdouts. White-box logprob ranking detects memorization better than open-ended generation, and a simple system prompt defense blocks 99.8% of suffix attacks.