Internalizing the Future: A Unified Agentic Training Paradigm for World Model Planning
This paper proposes a unified three-stage training paradigm to enable large language model (LLM) agents to internalize a world model for foresight planning. By addressing the format-capability gap through World Model Agentic Mid-Training, Format-Eliciting SFT, and Foresight-Conditioned Reinforcement Learning, the approach outperforms baselines on search and mathematical reasoning tasks.
[2606.27483] Internalizing the Future: A Unified Agentic Training Paradigm for World Model Planning
[Submitted on 25 Jun 2026]
Title:Internalizing the Future: A Unified Agentic Training Paradigm for World Model Planning
View a PDF of the paper titled Internalizing the Future: A Unified Agentic Training Paradigm for World Model Planning, by Xuan Zhang and 8 other authors
View PDF
Abstract:Large language model (LLM) agents have demonstrated strong capability in sequential decision-making, yet they remains fundamentally reactive in long-horizon tasks. Unlike humans who employ "what-if" reasoning to evaluate potential plans before commitment, standard agents lack an internal world model to simulate future outcomes. Therefore, we propose to internalize future-aware planning by training a single autoregressive model to verbalize both a prospective state rollout and a plan-conditioned success estimate-a textual analogue of the Q-value. Crucially, we identify a format-capability gap: simply fine-tuning agents on look-ahead traces during post-training leads to superficial mimicry of foresight without genuine predictive grounding. To bridge this gap, we introduce a three-stage training paradigm: (i) World Model Agentic Mid-Training (WM-AMT) to inject latent predictive capabilities into the policy; (ii) Format-Eliciting SFT (FE-SFT) to structure this injected capability; and (iii) Foresight-Conditioned Reinforcement Learning (FC-RL) to refine the calibration and utility of the generated simulations. Evaluated on search and mathematical reasoning tasks, our approach consistently outperforms other training baselines. Our results demonstrate that effective internal world modeling in LLM agents requires a capability-first training pipeline to achieve grounded and calibrated foresight.
Subjects:
Artificial Intelligence (cs.AI)
Cite as: arXiv:2606.27483 [cs.AI]
(or arXiv:2606.27483v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2606.27483
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: Xuan Zhang [view email] [v1] Thu, 25 Jun 2026 19:05:44 UTC (19,366 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled Internalizing the Future: A Unified Agentic Training Paradigm for World Model Planning, by Xuan Zhang and 8 other authors
View PDF
TeX Source
view license
Current browse context:
cs.AI
new | recent | 2026-06
Change to browse by:
cs
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
Loading...
Data provided by:
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)