2026-04-28 00:00 UTCOriginal source2 min readUpdated: 2026-06-27 00:25 UTC

Local Mechanisms of Compositional Generalization in Conditional Diffusion

The paper investigates how conditional diffusion models achieve compositional generalization, specifically length generalization—generating images with more objects than seen during training. Experiments on CLEVR show success in some cases but not all, pointing to the importance of local conditional scores. The authors prove an equivalence between compositional structure and local conditional scores, and demonstrate that enforcing locality enables generalization in failing models. Analysis of SDXL reveals spatial locality but absent conditional locality in pixel space, yet evidence of local conditional scores in feature space.

SourceApple Machine Learning Research

Local Mechanisms of Compositional Generalization in Conditional Diffusion - Apple Machine Learning Research

Machine Learning Research

Open MenuClose Menu

Overview

Research Highlights

Publications

Events

Work with us

research area Methods and Algorithms

content type paperpublished April 2026

Local Mechanisms of Compositional Generalization in Conditional Diffusion

AuthorsArwen Bradley

View publication

Copy Bibtex

Conditional diffusion models appear capable of compositional generalization, i.e., generating convincing samples for out-of-distribution combinations of conditioners, but the mechanisms underlying this ability remain unclear. To make this concrete, we study length generalization, the ability to generate images with more objects than seen during training. In a controlled CLEVR setting (Johnson et al.,2017), we find that length generalization is achievable in some cases but not others, suggesting that models only sometimes learn the underlying compositional structure. We then investigate locality as a structural mechanism for compositional generalization. Prior works proposed score locality as a mechanism for creativity in unconditional diffusion models (Kamb & Ganguli, 2024; Niedoba et al., 2024), but did not address flexible conditioning or compositional generalization. In this paper, we prove an exact equivalence between a specific compositional structure (conditional projective composition) (Bradley et al., 2025) and scores with sparse dependencies on both pixels and conditioners (local conditional scores). This theory also extends to compositions of concepts (such as style+content) in feature-space. We validate our theory empirically: CLEVR models that succeed at length generalization exhibit local conditional scores, while those that fail do not. Furthermore, we show that a causal intervention explicitly enforcing local conditional scores enables length generalization in a previously failing model. Finally, we investigate SDXL and find that in pixel-space, spatial locality is present but conditional-locality is mostly absent; however, we find quantitative evidence of local conditional scores in the network’s learned feature-space.