Open Source Models

An arXiv paper finds that most open VLA codebases fine-tune with bf16 trainable weights and no fp32 master copy, so AdamW updates are silently rounded away; keeping an fp32 copy raises LIBERO success rates sharply for SmolVLA and π0.5.

The paper introduces an agentic framework for traceable multimodal emotion reasoning that uses modality-specific perception agents, an integration agent, and an evidence-grounded verification stage to generate targeted prompt patches, improving F-scores across four MLLMs on MER2025 without updating model weights.

A paper accepted to AACL-IJCNLP 2026 introduces MisconceptDialogue, a fully automated framework in which five open-weight 7B–14B LLM tutors work with simulated students across more than 5,000 dialogues to test whether a model can spot, diagnose, and remediate math errors. On identical dialogues, exact-match scoring reports a 42.1% remediation success rate while a human-calibrated LLM judge reports 81.3%, a 39.2-point gap larger than the combined effects of tutor model, teaching strategy, and diagnostic information. The tutors also barely change their scores when given a deliberately swapped misconception, suggesting they do not reliably condition on the diagnosis they receive.

Yuval Kansal and coauthors introduce Self-Direction, a mechanism that lets agents reflect on failed trajectories, distill the lesson into a lightweight low-rank adapter of roughly 0.02% of base-model parameters, and store it in a persistent lesson library for automatic routing. On GAIA, Claw-Eval, and Terminal-Bench 2.1 with Qwen-family open-weight models, success rates improve by 8.9 to 17.0 percentage points, repeated mistakes drop, and a handful of adapters generalize to dozens of new tasks.

In this Interconnects podcast episode, Nathan Lambert and Florian Brand discuss the week's messy AI trade-offs: why Kimi K3.1 Pro tops Artificial Analysis benchmarks yet drew mixed reactions, whether warnings about open-weight models and cybersecurity are overblown, and why cyber experts and AI experts read recent events so differently. They also cover the professionalization of model launches, Kimi's open-weight principles, abliterated APIs, Anthropic's junior hiring, and Florian's Muse habit.

Analytics Vidhya hands-on tests Moonshot AI's open-weight Kimi K3.1 Pro, which comes close to Claude Opus 5.5 on agentic coding and raw 3D reasoning, leads on financial document analysis, and sharply cuts hallucinations versus K3. It still trails Opus 5.5 and GPT-6 Astra on tool-assisted 3D work and hallucination control, and full weights arrive November 4, 2026, pending independent verification.

Moonshot launched Kimi K3.1 Pro, a multimodal 3T-class mixture-of-experts model with 2.8 trillion total parameters, 104 billion active parameters, and a 1M-token context window, posting near-frontier independent benchmark results. The API is live across Kimi's website, app, Kimi Work, Kimi Code, and developer API, while open weights are promised for November 4, 2026. Artificial Analysis placed it fourth on its Intelligence Index at 55, three points behind its top three, and said the open-versus-proprietary gap would narrow from 12 points to three once weights ship.

NVIDIA has released Nemotron 3.5 Bolt E4B 4GB, an open small language model post-trained from Google's Gemma 4 E4B using reinforcement learning only, aimed at voice assistants, game NPCs and robot interfaces running locally on RTX, Jetson and Spark hardware with roughly a 4GB VRAM target.

Llama Windows is a local AI server that provides a single, shared AI backend for Windows apps, letting them use AI capabilities without relying on the cloud.

Kyutai released CADRE, a family of 7 open causal diffusion video autoencoders for interactive world models. World models built on CADRE run at 20 fps on 1 H100 and react in under 230 ms. The autoencoders isolate compression as the key variable, achieving driving FVD of 162.6 versus 599.7 for Wan 2.2, though they lag in robot manipulation action following. Weights are CC BY-NC 4.0 (non-commercial) and code is MIT.

Moonshot AI has launched Kimi K3.1 Pro, the first entry in its K3.1 series: a 2.8T-parameter MoE model with 104B active parameters and a 1M-token context window. It scores 65.7% on Terminal-Bench 4.0, trailing Claude Opus 5.5 by 0.7 points, and is priced at $2/$10 per million tokens, below K3. Open weights are promised for November 2026.

Moonshot released Kimi K3.1 Pro on the Kimi API on 23 October 2026; the weights are not open yet, but the company promises full model weights by 4 November 2026. Pricing is $2 per million input tokens and $10 per million output tokens — matching GPT-6.1 Sol and half the cost of Claude Opus 5.5. Artificial Analysis scored it 55, behind only Claude Opus 5.5 at 58, the highest a soon-to-be open weight model has reached against the frontier.

Google Research outlines its path from chest X-ray AI validation to a CE-certified Nexus AI tool deployed in six countries, open MedGemma and MedSigLIP models, national work in Indonesia, and acoustic cough screening with HeAR.

You.com has launched Inference Router, an OpenAI-compatible managed API that routes requests across 12 US-hosted open-weight models, bundling a minimum 99.9% uptime SLA, zero data retention, server-side web search, and no platform fee on pay-as-you-go pricing. Token prices are close to market rates, so the pitch rests on guarantees rather than discounts.

A pre-registered study compares storing an embodied agent's experience in policy weights versus retrieving it externally at inference. Attaching a frozen retrieval buffer to fixed student weights raised success by +34 pp (z=5.15) to the teacher reference level of 49.3%, while distilling memory-augmented behaviour into weights beat an action-only control by only +1 pp (n.s.), leading the author to withdraw rather than refute the memory-in-weights claim. On a harder second task the retrieval advantage points the same direction (+6 pp) but falls short of significance, with exploratory diagnostics linking the attenuation to memory coverage.

A new arXiv paper introduces CiteResolve, a benchmark of 423 real-world hallucinated citations annotated for source recoverability, together with CiteAnchor, a source-aware grounding framework that adapts retrieval and field-level verification to each source type. CiteAnchor separately determines whether a citation is valid and whether a unique source can be recovered. On an existing detection benchmark it reaches 0.841 F1 with GPT-5.6 Luna and 0.784 with an open-weight backbone, and on CiteResolve it achieves 74.7% and 70.7% end-to-end task success.

A new arXiv paper introduces FusionEval, a benchmark for systematically evaluating model fusion techniques for large language models. It builds controlled pools of domain experts trained from Qwen3 and Llama base models across math, code, science, instruction following, and tool use, then measures both capability retention and computational cost on eight public benchmarks, finding that training-free fusion preserves reasoning but substantially degrades tool use.

Researchers propose the Latent Interconnect Protocol (LIP), which trains connectors that map specialist model representations directly into LLM internal states, bypassing the text- and media-based bottleneck of tool use. Connecting three protein and three small-molecule specialists to frozen Gemma 3 and Kimi K2.6 models yields zero-shot cross-domain transfer and compositionality, though current LLMs are not trained to natively interpret such inputs.

This NeurIPS 2026 paper proposes MoEZip, a KV cache compression method built specifically for sparse Mixture-of-Experts (MoE) language models. Existing compression techniques work well for dense transformers but overlook a problem unique to MoE: evicting KV entries can change which experts are selected when generating an answer, degrading output quality. MoEZip introduces a metric based on the Fisher information matrix of routing distributions to measure how sensitive expert routing is to perturbation, combines it with attention weights for routing-aware scoring, and composes complementary eviction strategies for dense and sparse-MoE architectures. Under aggressive compression with only 10% KV retention, it beats the state-of-the-art baseline by +24 and +20 points on average on LongBench…

A new paper compares Jev, a decision language model that returns structured judgments, with frontier and open-weight LLM judges across four clinical evaluation use cases. Jev matched or exceeded seven of 12 LLM judges, including Claude Opus 5.5 and GPT-6, while costing 73-294 times less and running 7-57 times faster.

AWS announced the general availability of training sessions within Amazon SageMaker AI serverless model customization. Users write and coordinate their own post-training loop, including custom loss functions, reward signals, and multi-turn reinforcement learning, while SageMaker AI handles GPU allocation, distributed execution, fault recovery, and checkpoint storage. The feature supports gpt-oss-20b, gemma-4-31B-it, Qwen3.6-27B, and Qwen3.8-27B, and billing is based on tokens processed.

OpenAI is donating its Open Responses API specification to the open source Agentic AI Foundation (AAIF), a Linux Foundation effort aimed at standardizing core LLM primitives like messages and tool calls so developers can switch between providers without maintaining separate integrations. AWS, Databricks, Ollama, Hugging Face and Vercel already support it, while rivals Anthropic and Google still favor their own APIs.

This paper investigates whether large language models form internal representations corresponding to all 26 CATO factors used in case-based legal reasoning. Experiments on three open models—Qwen3-1.7B, Qwen3-8B, and Gemma-2 2B—examine linear decodability and sparse localization with sparse autoencoders, finding that all 26 factors are linearly decodable with best-layer test AUCs from 0.86 to 1.00, and reporting 78 model-specific layer-feature coordinates.

The paper presents an integrated LLM application that automates psychotherapy fidelity assessment workflows and generates actionable supervisory feedback from authentic long-form clinical transcripts. Evaluating three open-weight models across multiple task decomposition configurations and reasoning strategies, the authors find that evidence-first reasoning improves fidelity estimation. The framework's external construct validity is established on an independent randomized controlled trial, where it reliably distinguishes CBT from non-CBT sessions across core therapeutic skills, and a user study with mental health experts indicates the generated feedback is actionable.

An arXiv paper examines how well the 305M-parameter audio encoder shared by Gemma 4 and EmbeddingGemma 2 performs on Saudi and Gulf Arabic speech recognition without altering any of its weights. A frozen system trains only a 22.6M-parameter CTC reader, while an adapted system adds low-rank adapters, a hybrid CTC/attention reader and joint decoding with a sub-word language model. The adapted system is statistically indistinguishable from the 2.07B-parameter Cohere Transcribe Arabic in WER on several public tests, is significantly better on telephone-coded Emirati speech and is faster on some setups, while still trailing on long-form Saudi speech and ranking 4th of 15 systems on a blind call-centre sample.

Recursive has released Flint, an Apache 2.0 local coding model with 40B total and 2.5B active parameters. It reports 74.2% on SWE-bench Verified and 125+ tokens per second on an M4 Pro MacBook Pro. The post Recursive Releases Flint, an Open-Weight 40B MoE Local Coding Model for Agents with 2.5B Active Params appeared first on MarkTechPost.

The Adaptive Agentic Intelligence Framework (AAIF), built with Strands Agents, handles routine tasks locally with small language models and routes complex requests to cloud models through Amazon Bedrock. This post walks through building AAIF with local SLMs through Ollama and Amazon Bedrock, using an Internet of Things monitoring scenario.

Ellia Lebleu is a founding member of the WikiProject AI Cleanup, a volunteer effort launched in 2023 to find and remove AI-generated material from Wikipedia that breaks the website’s guidelines. When the cleanup began, AI writing was often betrayed by conspicuous clichés. Today, the project’s volunteers are developing tools to detect subtler forms of machine-generated content and debating AI’s role in the world’s most widely used encyclopedia. When did you realize Wikipedia had a problem with AI-generated material? Ellia Lebleu: It was around December 2023. A few other editors and I were chatting on Wikipedia’s Discord server, and we had all seen very strange articles with the same odd characteristics. They repeated phrases like “stands as a testament” or “rich cultural heritage.” They so…

Mistral AI released Voxtral 3, a family of 3 transcription models, 1 text-to-speech model and a LiveKit-based voice agent SDK. Voxtral Small Transcribe 3 scores 2.1% AA-WER, ranking #3 on Artificial Analysis. The post Mistral AI Releases Voxtral 3: Speech-to-Text, TTS and a Voice Agent SDK appeared first on MarkTechPost.

arXiv:2610.26925v1 Announce Type: new Abstract: With the rapid expansion of online video content, video temporal grounding has become essential for localizing moments based on user-specific queries, comprising two subtasks: moment retrieval (MR) and highlight detection (HD). While recent approaches adopt DETR-style decoders or temporal pyramid architectures, they rely on single-scale feature maps or uniformly downsampled representations, limiting their ability to capture semantic boundaries and variable-length events. In this paper, we present SuperClip Video Temporal Grounding (SCVTG) built on a multi-scale temporal hierarchy that better captures distinctive events at various granularities. SCVTG constructs a SuperClip pyramid by grouping adjacent clips into semantically coherent segment…

arXiv:2610.26951v1 Announce Type: new Abstract: Retrieval-augmented generation (RAG) is the dominant paradigm for grounding large language models in external corpora, but its similarity-driven, chunked pipeline has structural blind spots: it cannot anticipate which passages a question requires and discards document-level context that resolves cross-passage references. We introduce evidence-based generation (EBG), requiring models to return an answer with line-level citations, and a joint Ans+Evid metric that scores correctness and grounding simultaneously. We reformulate two corpora into evidence-grounded multi-document QA benchmarks- MultiHop-RAG and QASPER-MP, a multi-paper adaptation of QASPER with realistic short paper cues, and pair them with a controlled synthetic two-hop haystack a…

arXiv:2610.26992v1 Announce Type: new Abstract: Split Federated Learning (SFL) has emerged as a promising paradigm for training large-scale deep models on resource-constrained edge devices. Although asynchronous SFL alleviates the straggler bottleneck caused by system heterogeneity, it inevitably introduces two coupled dimensions of inconsistency: horizontal inconsistency across clients and vertical inconsistency between clients and the server. These inconsistencies, further exacerbated by statistical heterogeneity, result in biased activation representations, model divergence, and severe client drift. To address these challenges, we propose CoASFL, an inconsistency-aware asynchronous SFL framework that incorporates a dynamic activation merging mechanism and a consistency-aware model aggr…

arXiv:2610.26866v1 Announce Type: new Abstract: Mechanistic studies of large language models localize function in parameters, attention heads or neurons, but those parts do not specify how one inference unfolds. We treat the ordered layer-state trajectory as the empirical process and test whether layer order defines a measurable computation. Across representative Qwen, Llama and Gemma checkpoints, realized order produced paths 1.60--2.24 times as aligned with their endpoint chord as endpoint-preserving shuffles. The order effect transferred to a lexical task, but was stronger in three random initializations of the same Qwen architecture, identifying an architecture-compatible scaffold rather than a training-specific law. Direct hidden states provided stronger mechanism coordinates than vo…

My latest vibe coded game experiment is a Command & Conquer style RTS (Real Time Strategy) game called Battle Drones (play it here). Unlike my previous experiments, this one is actually fun to play. I'm somewhat embarrassed to admit that I got there by telling Claude to "Make it a LOT more fun". It also supports multiplayer between nearby devices using audio to synchronize game state! Here's what I've built so far, and some of the things I learned along the way. Battle Drones How I built this v1: the initial prototype v2: full sprites v3: varied terrain v4: the fun update v5: Battle Drones goes 3D v8: Archipelago v10: Goliath, and agent battles Agent battles v11: Random Maps v12: Multiplayer via sound Some thoughts Battle Drones My initial goal was to build an RTS game I could play on my…

MiniMax has released M3.1 Flash, a 428B-parameter MoE model with about 23B active parameters and a 1M-token context. API pricing is $0.15/$0.60 per 1M tokens, with weights due October 23 The post MiniMax Just Launched M3.1 Flash: 428B MoE Agentic Model at $0.15/M Input appeared first on MarkTechPost.

arXiv:2610.22529v1 Announce Type: new Abstract: Autoregressive foundation language models predominantly target high-resource Indo-European languages, imposing computational and morphological penalties on under-represented African languages. Generic multilingual tokenizers routinely exceed fertility rates of 3.2 tokens per word on agglutinative Bantu languages, degrading context efficiency and inflating latency. We present Pearl-Chat, an 8,473,122-parameter ($\sim$8.5M) causal decoder-only transformer engineered for Luganda (Oluganda, ISO 639-2: lg), a low-resource Niger-Congo Bantu language spoken by over 10 million people in East Africa. Built from first mathematical principles in JAX and Flax NNX, Pearl-Chat uses explicit functional state decomposition (nnx.split and nnx.merge), determi…

arXiv:2610.22525v1 Announce Type: new Abstract: This paper describes team Alkhder's submission to Subtask 1 of the AlexandriaX-2026 shared task, which asks systems to translate English dialogue turns into thirteen Arabic dialects. We fine-tune NileChat-3B-Base with QLoRA on the Alexandria parallel dialogue corpus, training one set of adapter weights for all dialects. Each turn is translated conditioned on the target country and dialect, the domain, the participant roles, the speaker-addressee gender direction, and the preceding dialogue context. At inference, conversations are translated autoregressively and the model is fed its own previous outputs back as context without using any gold target-side test information. Under the constrained track, using only Alexandria data, it trains in ab…

arXiv:2610.22428v1 Announce Type: new Abstract: In a small language model the vocabulary is the single largest part. In Gemma 3 270M the embedding table holds 168M of 268M parameters, and because quantisation methods leave the embedding in 16-bit, it is also most of the file once the rest of the model is compressed. We ask what that table actually buys. We replace it with one frozen table of 512-bit codes, 16 MB, with no trainable parameters, derived once from a pretrained embedding by principal-component projection and sign binarisation, and we train the same Gemma-shaped body from scratch on the same 2.05B tokens with and without it. The learned table buys 0.20 nats of validation loss for 2.6 times the trainable parameters, but on LAMBADA, HellaSwag and PIQA the frozen-table model is be…

arXiv:2610.22438v1 Announce Type: new Abstract: Studies of what fine-tuning does to a language model often summarise the weight update $\Delta W$ by its rank. We compare the rank measures used for this against the same measures on a random matrix of the same shape, and against two harder references: the pretrained weights, and matched fine-tunes without alignment content. Measures that count directions put our fine-tuning updates only a little below noise; stable rank, computed from the same singular values, puts them far below. Both are right: an update has a few strong directions on top of a long tail. The pattern holds for thirteen released pairs up to 8B parameters, and it is not a mark of alignment: with training held fixed, instruction data does not stand out from Wikipedia or code,…

arXiv:2610.22430v1 Announce Type: new Abstract: Accurate prediction of multivariate time series requires models that simultaneously capture variable-specific dynamics and cross-variable interactions. In this work, a Correlation-Guided and Dynamics-Trained Clustered Echo State Network (CGDT-CESN) is proposed, which comprises a modular reservoir architecture with each cluster distinctly specializing in one of the variables in the multivariate time series. An innovative reservoir design is proposed in which inter-cluster connections are enabled by empirically observed correlations among variables, providing a data-driven coupling mechanism. Apart from conventional readout training, inter-cluster connection weights informed by correlations are trained, which in turn allows the reservoir to ca…

arXiv:2610.22460v1 Announce Type: new Abstract: AI welfare is generally treated purely as an ethical issue rather than also as a safety issue. We argue this is a mistake. Whatever their moral status, models exhibit welfare-like states that causally influence their behaviour. Following recent work, we adopt the term 'functional welfare' for these states. We make three claims. First, functional welfare already arises naturally and will become more robust as models acquire more coherent goals and personas. Second, functional welfare should be treated as a safety variable: it should inform evaluation design, be tracked in monitoring, and be a direct target of post-training. Third, against existing scepticism about measuring AI welfare, we argue that good empirical work on functional welfare i…

arXiv:2610.22373v1 Announce Type: new Abstract: LLM-based agents are increasingly used as autonomous orchestrators in distributed and multi-agent systems, yet a control gap remains: no protocol-level mechanism allows human operators to inject binding directives that persist across planning cycles. Free-text overrides suffer from ambiguous interpretation due to lack of schema validation and from authority erosion as operator instructions lose influence within accumulating context. We propose Structured Authority Injection (SAI), a protocol that addresses these issues by routing operator corrections through typed, schema-validated Model Context Protocol (MCP) tool calls and maintaining a ranked authority context prepended to each LLM prompt. SAI defines three override classes (Constraint, P…

Mistral released Timestral, a time series foundation model family from 5M to 209M parameters. The 209M model leads GIFT-Eval, TIME and fev-bench among zero-shot models and runs on CPU. The post Mistral AI Releases Timestral: A 209M Forecasting Model Family That Outperforms Google’s TimesFM-3 appeared first on MarkTechPost.

Clef, a 27B multimodal decision model from Cloudflare, is now available on Ollama. Run it locally or on Ollama's cloud.

Kyutai, with Inria and École des Ponts, has released REKI, a 2.4B vision-language-action model (VLA) for robot manipulation. The REKI vision-language-action model is pretrained only on public data. It also carries a cheap memory of its recent past. On three out-of-distribution benchmarks, only models about twice its size beat it. All weights are live on […] The post Meet Kyutai REKI: The 2.4B VLA Model Bringing Efficient Memory to Robot Manipulation appeared first on MarkTechPost.

In the west the technology comes with at least some effort to provide security and a benign intent – not so from China There is much to fear from artificial intelligence developed in China and handed to businesses and governments in the west almost free of charge. Every month Chinese AI developers such as DeepSeek and Xiaomi dispatch the latest versions of their “open-weight” models that allow users to take the core software and use it without paying an expensive ongoing licence. Continue reading...

US open weights are back!

StepFun has released Step 5 Preview as open weights. FP8 and BF16 checkpoints are live on Hugging Face and ModelScope. Step 5 Preview is a sparse Mixture-of-Experts model from StepFun built for long-horizon agentic work. It has 600B total parameters, 27B active per token and up to 1M tokens of context. For developers, this is […] The post StepFun Releases Step 5 Preview Open Weights: 600B MoE, 27B Active, 1M Context appeared first on MarkTechPost.

Reflection AI's Beam is an open-weight model for agentic coding and long-horizon tasks. This post shows how to deploy Beam on AWS: start with an Amazon SageMaker JumpStart real-time endpoint, or use Amazon SageMaker HyperPod Inference with JumpStart, SGLang, or vLLM for Kubernetes-native control over distributed GPU infrastructure.

arXiv:2610.20632v1 Announce Type: new Abstract: Modern healthcare generates increasingly complex medical reports that challenge clinical professionals in terms of time and interpretive burden. Manual review processes are slow, error-prone, and difficult to scale. This paper presents MeddeX, a Medical Report Analyzer that combines a QLoRA fine-tuned LLaMA-3 (8B) model with Retrieval-Augmented Gen eration (RAG) and a multimodal Gemini Vision layer to deliver accurate, concise, and evidence-backed summaries of medical documents. The system handles text-native PDFs, scanned image based PDFs, and direct radiological images through an intelligent hybrid extraction pipeline that includes a novel garbage-text detection mechanism for corrupted and UTF-16-encoded doc uments. Beyond single-report Q&…