2026-06-19原文2 min readUpdated: 2026-06-19

Human-like autonomy emerges from self-play and a pinch of human data

Self-play reinforcement learning can train driving policies without human data but often learns alien behaviors. A new method uses just 30 minutes of human demonstrations as a regularizer on top of a minimal safe reward, achieving human-compatible policies with 2500x less data than imitation learning, training in 15 hours on a single consumer GPU.

SourcearXiv Machine LearningAuthor: Daphne Cornelisse, Julian Hunt, Zixu Zhang, Wa\"el Doulazmi, Kevin Joseph, Jaime Fern\'andez Fisac, Eugene Vinitsky

[2606.19370] Human-like autonomy emerges from self-play and a pinch of human data

[Submitted on 11 Jun 2026]

Title:Human-like autonomy emerges from self-play and a pinch of human data

View a PDF of the paper titled Human-like autonomy emerges from self-play and a pinch of human data, by Daphne Cornelisse and 6 other authors

View PDF HTML (experimental)

Abstract:Self-play reinforcement learning has recently emerged as a way to train driving policies without any human data. It uses cheap, large-scale simulations to substitute expensive, large-scale human driving demonstrations. A key limitation of this approach is that policies trained through pure self-play can learn effective but alien driving conventions incompatible with people. Previous works attempt to mitigate such behavioral misalignments through extensive reward engineering and domain randomization, which are brittle and labor-intensive. Instead of completely discarding human demonstrations, our method treats them as a regularization objective on top of a minimal safe goal-reaching reward. Like the spice in a good stew, we find that a little human data goes a long way: our method uses only 30 minutes of human demonstrations, 2500x fewer than comparable imitation learning approaches. Resulting policies coordinate with held-out human trajectories and complete training in 15 hours on a single consumer-grade GPU. Videos and full source code are available at this https URL.

Comments: 10 pages

Subjects:

Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA)

Cite as: arXiv:2606.19370 [cs.LG]

(or arXiv:2606.19370v1 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2606.19370

arXiv-issued DOI via DataCite

Submission history

From: Daphne Cornelisse [view email] [v1] Thu, 11 Jun 2026 19:16:53 UTC (6,632 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled Human-like autonomy emerges from self-play and a pinch of human data, by Daphne Cornelisse and 6 other authors

View PDF

HTML (experimental)

TeX Source

view license

Current browse context:

cs.LG

new | recent | 2026-06

Change to browse by:

cs cs.AI cs.MA

References & Citations

NASA ADS

Google Scholar

Semantic Scholar

Data provided by:

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)