Apple Machine Learning Research

Modern Large Language Models achieve impressive reasoning capabilities with long Chain of Thoughts, but they incur substantial computational cost during inference, and this motivates techniques to improve the performance-cost ratio. Among these techniques, Speculative Decoding accelerates inference by employing a fast but inaccurate draft model to auto-regressively propose tokens, which are then verified in parallel by a more capable target model. However, due to unnecessary rejections caused by token mismatches in semantically equivalent steps, traditional token-level Speculative Decoding…

Large Language Models (LLMs) have achieved state-of-the-art performance on a broad range of Natural Language Processing (NLP) tasks, including document processing and code generation. Autoregressive Language Models (ARMs), which generate tokens sequentially conditioned on all previous tokens, have been the predominant paradigm for LLMs. While these models have achieved high accuracy across a range of downstream tasks, they exhibit low arithmetic intensity due to the inherent sequential dependency in next-token prediction. Recently, Diffusion Language Models (DLMs) have emerged as a promising…

Continuous diffusion and flow matching models could represent a powerful alternative to autoregressive approaches for language modelling (LM), as they unlock a host of advantages currently reserved for continuous modalities, including accelerated sampling and tilting. Recently, several works have demonstrated the possibility of generating discrete data continuously by a simple flow matching process between a Gaussian and the one-hot encoded data distribution. They have further shown the feasibility of accelerated sampling via Categorical Flow Maps (CFMs), resulting in competitive sample…

Large language models (LLMs) with integrated search tools show strong promise in open-domain question answering (QA), yet they often struggle to produce complete answer set to complex questions such as “Which actor from the film Heat won at least one Academy Award?”, which requires (1) distinguishing between multiple films sharing the same title and (2) reasoning across a large set of actors to gather and integrate evidence. Existing QA benchmarks rarely evaluate both challenges jointly. To address this, we introduce DEEPAMBIGQAGEN, an automatic data generation pipeline that constructs QA…

The quality of open-weight language models has dramatically improved in recent years. Sharing weights greatly facilitates model adoption by enabling their use across diverse hardware and software platforms. They also allow for more open research and testing, to the extent that users can use them as checkpoints, fine-tune them according to their needs, and potentially redistribute them. In some cases, however, concerns on modifying these weights towards unauthorized uses may outweigh the pros of giving users such a freedom. Defending against such adaptation is non-trivial: since an adaptive…

We study outlier tokens in Diffusion Transformers (DiTs) for image generation. Prior work has shown that Vision Transformers (ViTs) can produce a small number of high-norm tokens that attract disproportionate attention while carrying limited local information, but their role in generative models remains underexplored. We show that this phenomenon appears in both the encoder and denoiser of modern Representation Autoencoder (RAE)-DiT pipelines: pretrained ViT encoders can produce outlier representations, and DiTs themselves can develop internal outlier tokens, especially in intermediate layers…

Preference alignment has become a crucial component in enhancing the performance of Large Language Models (LLMs), yet its impact in Multimodal Large Language Models (MLLMs) remains comparatively underexplored. Similar to language models, MLLMs for image understanding tasks encounter challenges like hallucination. In MLLMs, hallucination can occur not only by stating incorrect facts but also by producing responses that are inconsistent with the image content. A primary objective of alignment for MLLMs is to encourage these models to align responses more closely with image information. Recently…

To operate effectively across contexts, robots must adapt their action execution. This paper proposes MoMo, a two-stage imitation learning framework with a spatiotemporal action tokenizer and a behavior-cloning transformer that takes a continuous motion-mode condition. In six real-robot tasks, varying this condition produces steady, dynamic, and intermediate behaviors distinguishable by human raters. MoMo transfers unseen motion modes to tasks demonstrated in only one mode, largely preserving task success, showing compositional generalization.

While UMAP is widely used for exploring high-dimensional data, typical workflows focus on its lower-dimensional embedding, largely overlooking the rich k-nearest-neighbor (kNN) graph that UMAP constructs internally. This graph encodes the data manifold in its original high-dimensional space, before the distortion that UMAP’s 2D projection introduces. We demonstrate the untapped potential of this internal representation, showing how standard graph algorithms applied to this graph enhance data sensemaking: (1) PageRank identifies representative data points, (2) k-core decomposition reveals dense core regions versus sparse periphery, and (3) clustering coefficient detects tight-knit neighborhoods with highly-similar data points. Through quantitative and qualitative evaluation on MNIST and Fashion MNIST, we show that these graph-based analyses are not only practical but also competitive with or complementary to purpose-built methods (e.g., k-medoids for exemplar selection, HDBSCAN for density-based clustering).

Apple has released Siri Expressive Voices, powered by a memory-efficient audio synthesis architecture called a detokenizer. This architecture converts semantic audio tokens from the foundation model into high-fidelity audio within the tight compute and memory budget of the Apple Matrix Coprocessor (AMX). It features a three-component design—a streaming encoder, a temporal decoder, and a depth decoder—that systematically decouples temporal and depth processing. A single reusable depth decoder with Diffusion Transformer (DiT)-style stage conditioning generates all RVQ levels autoregressively, replacing dedicated per-level decoders. Causal sliding window attention with fixed-window key-value caching yields constant memory complexity independent of sequence length. Deployed on the AMX, the detokenizer sustains roughly 10ms per generation step (about 16x faster than real time), with a peak runtime memory of only ~21MB and 329MB of on-device assets, enabling continuous streaming synthesis of 20–320 seconds of audio. The architecture is deployed in production as part of Siri Expressive Voices, running within AFM 3 Core Advanced (1-billion-parameter activation size) and improving Mean Opinion Score (MOS) by +0.28 overall (4.15 vs. 3.87) and by +0.42 on conversational speech (4.24 vs. 3.82) over the prior on-device TTS system.

GH-ESD is a novel error slice discovery framework tailored for instance-level vision tasks such as object detection and segmentation. It reformulates slice discovery as a generate-and-verify process, leveraging LLM priors and vision-language models to identify hypothesis slices at the instance level and statistically verify them. Experiments show significant improvements over baselines on the new GESD benchmark.

Long-horizon execution in LLMs remains unstable even with high-level strategies. This work shows that while decomposition is essential for stability, extreme decomposition creates a 'no-recovery bottleneck' due to highly non-uniform error distribution. The proposed Lookahead-Enhanced Atomic Decomposition (LEAD) incorporates short-horizon future validation and aggregating overlapping rollouts, enabling the o4-mini model to solve Checkers Jumping up to complexity n=13, whereas extreme decomposition fails beyond n=11.

This paper introduces CalibAtt, a training-free sparse attention method that identifies stable sparsity patterns via offline calibration, skips unimportant connections at inference, achieving up to 1.58× end-to-end speedup without sacrificing video quality.

Training API-calling LLM agents requires high-quality trajectories. Apple researchers propose an environment-free method that uses LLMs as digital world models to generate synthetic trajectories from API specs alone. Fine-tuning on synthetic data yields significant gains on AppWorld and OfficeBench.

Apple researchers present the Length Value Model (LenVM), a token-level framework that predicts remaining generation length at each decoding step. By framing length modeling as a value estimation problem with a constant negative reward per token, LenVM provides annotation-free, dense, unbiased, and scalable supervision. Experiments on LLMs and VLMs show that on the LIFEBench exact length matching task, LenVM improves a 7B model's length score from 30.9 to 64.8, surpassing frontier closed-source models. LenVM also enables continuous control over the performance-efficiency trade-off and predicts total generation length from the prompt.

LVSum is a human-annotated benchmark for evaluating long-form video summarization with fine-grained temporal alignment. It comprises 72 diverse videos (avg. 16 min) across 13 domains, each with up to 10 human-generated summaries containing temporal references. Evaluation reveals that transcripts contribute more than visual frames, a significant gap remains between model and human summaries, and MLLMs exhibit weaknesses in temporal grounding, instruction adherence, and cross-modal coherence.

Vision-language models (VLMs) fail to infer shared visual concepts from sets of example images. The new Visual Concept Inference from Sets (VICIS) benchmark evaluates this capability. The authors propose a training framework and architecture that learns to extract concept-specific embeddings from image sets, improving generative accuracy and generalization to unseen concepts and modalities.

A new paper challenges the standard approach to unlearning in machine learning by proposing that not all data points requiring removal are equally important. By analyzing influence functions, the researchers identify subsets of training data with negligible impact on model outputs, enabling a framework that reduces dataset size before unlearning by up to 50% and achieving significant computational savings.

Apple researchers introduce SSD, a simple self-distillation method that improves LLM code generation without verifiers, teacher models, or reinforcement learning, achieving significant gains on LiveCodeBench.

A new personalization system for Apple TV search combines text-based multilingual embeddings and ID-based collaborative embeddings, fine-tuned via contrastive learning and interaction data. The approach improves ranking for ambiguous short queries (8.63% NDCG@10 lift) and users with longer histories (4.37% vs 2.13% for new users). Online A/B test shows +1.14% tap-through rate and +1.23% conversion rate.

We study doubly sub-linear interactive proofs of proximity (dsIPPs): proofs that are ultra-fast to generate, and can be used to prove approximate assertions about a huge input. Proof generation is ultra-fast in the sense that it only requires reading a small (sub-linear) portion of the input. Approximate verification of the proof is even faster (reading an even smaller portion of the input). We construct such proofs for any property decidable by constant-width read-once oblivious branching programs (ROOBPs), as well as for approximate verification of Hamming weight and a relaxation of bipartiteness.

A first evaluation of uncertainty quantification methods for LLM function-calling finds that multi-sample methods offer no advantage over single-sample methods, with improvements via abstract syntax tree clustering and token selection.

CLaRa is a unified RAG framework that compresses documents into continuous vectors and jointly optimizes the reranker and generator via end-to-end training. It introduces SCP for key-preserving data synthesis and uses a differentiable top-k estimator, achieving state-of-the-art compression and reranking at 16x compression rates.

Propose FAE (Feature Auto-Encoder) framework that adapts pretrained visual representations into low-dimensional generative latents using only a single attention layer, achieving near-SOTA FID of 1.29 (with CFG) and 1.48 (without CFG) on ImageNet 256×256 while preserving reconstruction and understanding information.

Apple Music serves listeners across 150+ storefronts in dozens of languages, with a catalog that grows by hundreds of thousands of new tracks daily. At this scale, search recall on misspelled, transliterated, and cross-lingual queries becomes a dominant driver of session quality. Apple researchers present a multilingual semantic retrieval system built on a 305M-parameter Siamese bi-encoder fine-tuned from GTE-multilingual-base with curriculum-scheduled multi-objective training. The system achieves a 69% relative improvement in Hit@10 offline, and in a worldwide online A/B test, a 2.28% conversion-rate lift overall with an 86% reduction in no-result rate. Tail queries see a 7.93% relative CR lift.

Researchers propose the Pare framework, which models applications as finite state machines to enable realistic user simulation for proactive agents, and release Pare-Bench with 143 tasks.

Accepted at the AI4TCI Workshop at ARES 2026, this paper addresses behavioral privacy leakage in autonomous negotiation agents. It proposes an adaptive stochastic policy that provides (ε,δ)-differential privacy, almost-sure convergence, and high utility. In 3,000 simulated negotiations, it reduces adversarial inference accuracy by 43-50% while maintaining over 90% success rate and utility.

On-policy distillation provides dense per-token supervision for reasoning models, but its effectiveness varies. This research introduces a training-free diagnostic framework that quantifies the alignment between distillation signals and the ideal gradient at the per-token level. Findings show that distillation guidance aligns better on incorrect rollouts, and the optimal context depends on student capacity and task, with no universal best configuration.

This paper introduces SRLM, a framework that augments recursive language models with self-reflective program search using uncertainty signals (self-consistency, reasoning trace length, verbalized confidence). It achieves up to 22% improvement over RLMs and shows that recursion itself is not the primary driver of performance.

Multimodal large language models (MLLMs) have recently shown strong performance in visual understanding, yet they often lack temporal awareness, particularly in egocentric settings. Apple researchers propose Temporal Global Policy Optimization (TGPO), a reinforcement learning algorithm that uses verifiable rewards to incentivize temporal reasoning over frame-level shortcuts.