GigaWorld-1: A Roadmap to Build World Models for Robot Policy Evaluation
This systematic study introduces WMBench, a benchmark for evaluating world models as surrogate robot policy evaluators. Analyzing 7 video world models, 4 action schemes, and over 324,000 simulated rollouts paired with real robot executions, the authors identify three key insights: evaluator quality depends on long-horizon action-faithful consistency, pretraining requires balancing general knowledge with robot-specific controllability, and architectural choices critically determine alignment with real-world behavior. Based on these, they present GigaWorld-1, an optimized world model, and release all code, models, datasets, and toolkits.
-->
[Submitted on 2 Jul 2026]
Title:GigaWorld-1: A Roadmap to Build World Models for Robot Policy Evaluation
View a PDF of the paper titled GigaWorld-1: A Roadmap to Build World Models for Robot Policy Evaluation, by GigaWorld Team and 26 other authors
View PDF HTML (experimental)
Abstract:Evaluating embodied robot foundation models remains a critical bottleneck; unlike large language models efficiently assessed via digital benchmarks, robotic policies require slow, costly real-world rollouts limited by hardware and human supervision, which has driven interest in world models as surrogate policy evaluators, yet the key properties that make a world model reliable for policy assessment remain poorly understood. This work presents a systematic study of world models for robotic policy evaluation and introduces WMBench, a benchmark constructed from real-robot teleoperation data and matched policy rollouts covering diverse manipulation tasks to enable controlled comparisons across model families, action encodings, rollout horizons, and evaluation metrics. Using WMBench, we analyze 7 video world models, 4 action representation schemes, and over 324,000 simulated policy rollouts paired with real robot executions, further enriching our analysis with large-scale community submissions from the CVPR 2026 GigaBrain Challenge, curated synthetic trajectories, and a training videos spanning more than 12,000 hours. Our experiments deliver three core insights: evaluator quality is dominated by long-horizon, action-faithful rollout consistency rather than short-term visual realism; pretraining gains stem not only from data scale but from balancing general world knowledge with robot-specific controllability; and architectural choices including action encoding, memory design, and evaluator-focused post-training strongly determine alignment with real-world robot behavior. Drawing on these results, we derive a practical design roadmap and realize it in \textit{GigaWorld-1}, a world model specially optimized for policy evaluation, and we fully release our code, models, datasets, and toolkits to advance scalable evaluation research for embodied foundation models.
Comments: Project page: this https URL
Subjects:
Robotics (cs.RO)
Cite as: arXiv:2607.02642 [cs.RO]
(or arXiv:2607.02642v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2607.02642
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: Xiaofeng Wang [view email] [v1] Thu, 2 Jul 2026 17:02:43 UTC (22,772 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled GigaWorld-1: A Roadmap to Build World Models for Robot Policy Evaluation, by GigaWorld Team and 26 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.RO
new | recent | 2026-07
Change to browse by:
cs
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
Loading...
Data provided by:
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)