Skip to content
AI News HubLIVE
Original source3 min read

How Uncertain Is My Action? Self-Correcting Visual Planning with Frozen World Models

Summary

This paper shows that a frozen latent world model can be run in reverse to detect and compensate for actuation shifts. By predicting what should have been observed under a hypothesized shift and comparing it with actual observations, the method quantifies shifts, estimates them with Cramer-Rao error bars, and corrects the planner. Experiments on simulated tasks and a real robotic arm demonstrate significant improvements in success rates and precision.

SourcearXiv RoboticsAuthor: Siqing Li, Luming Yang, Yijia Jiao, Junze Deng, Alper Yilmaz
How Uncertain Is My Action? Self-Correcting Visual Planning with Frozen World Models
Report an error

The correction channel is not available yet. You can copy the article reference below for later.

Correction instructions
Read article

[Submitted on 17 Nov 2026]

Title:How Uncertain Is My Action? Self-Correcting Visual Planning with Frozen World Models

View a PDF of the paper titled How Uncertain Is My Action? Self-Correcting Visual Planning with Frozen World Models, by Siqing Li and Luming Yang and Yijia Jiao and Junze Deng and Alper Yilmaz

View PDF HTML (experimental)

Abstract:A frozen latent world model combined with sampling-based model predictive control is a visual planner. At deployment, the executed action can depart from the commanded one, and the frozen model has no way to notice. We show that the model can be run in reverse: for any hypothesized actuation shift, it predicts what it should have observed under that shift, and comparing this prediction with what was actually observed quantifies the shift and lets the planner correct itself. Unlike the residual energy, which mixes the shift with the model's own error, the score keeps only the part of the residual that a shift can produce. This shift score detects the shift with a locally optimal test, estimates it with Cramer-Rao error bars, names the dominant generator, and compensates the planner once an alarm fires. On PushT, TwoRoom and OGBench-Cube with a frozen latent world model, measured detection follows the prediction to within 0.05 AUROC. Self-correction raises success from 0.650 to 0.862 on PushT and from 0.378 to 0.432 on the harder Cube task, levels with an oracle that is told the true shift at the same block, better than five common test-time adaptation methods. On a real robotic arm, the score identifies a base-joint offset that no joint encoder registers and reduces the reach error from 36 mm to 3-5 mm, and under shifts drawn at random and sealed before the session, estimates recover 92-94% of the error reduction achieved with the true shift. The broader implication is that a frozen world model can act as a filter by predicting the future, testing action-interface uncertainty and validity, and finally correcting failures.

Subjects:

Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

Cite as: arXiv:2611.23620 [cs.RO]

(or arXiv:2611.23620v1 [cs.RO] for this version)

https://doi.org/10.48550/arXiv.2611.23620

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Siqing Li [view email] [v1] Tue, 17 Nov 2026 23:23:42 UTC (954 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled How Uncertain Is My Action? Self-Correcting Visual Planning with Frozen World Models, by Siqing Li and Luming Yang and Yijia Jiao and Junze Deng and Alper Yilmaz

View PDF

HTML (experimental)

TeX Source

view license

Additional Features

Audio Summary

Current browse context:

cs.RO

new | recent | 2026-11

Change to browse by:

cs cs.CV cs.LG

References & Citations

SciX

Google Scholar

Semantic Scholar

Loading...

Data provided by:

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)

Related Papers

Recommenders and Search Tools

Link to Influence Flower

Influence Flower (What are Influence Flowers?)

Core recommender toggle

CORE Recommender (What is CORE?)

Author

Venue

Institution

Topic

About arXivLabs

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)

Key points and analysis

Article intelligence

ResearchersAdvanced

Key points

  • A frozen world model can be run in reverse: for any hypothesized actuation shift, predict expected observations and compare with actual observations to compute a shift score that isolates the effect of the shift.
  • The shift score enables locally optimal detection, Cramer-Rao-bounded estimation, identification of the dominant generator, and compensation once an alarm fires.
  • Self-correction raises success from 0.650 to 0.862 on PushT and from 0.378 to 0.432 on OGBench-Cube, matching an oracle and outperforming five test-time adaptation methods.
  • On a real robotic arm, the score identifies an unregistered base-joint offset and reduces reach error from 36 mm to 3-5 mm, recovering 92-94% of the error reduction from the true shift.

Highlights and analysis are generated automatically and may contain errors. Check the original source.

How Uncertain Is My Action? Self-Correcting Visual Planning with Frozen World Models | AI News Hub