Skip to content
AI News HubLIVE
Original source2 min read

In-Context Reinforcement Learning under Non-Stationarity: A Survey

Summary

This paper surveys in-context reinforcement learning (ICRL) under non-stationarity, where pretrained decision models infer latent task rules and improve behavior from interaction context without parameter updates. In changing environments, accumulated context can become stale or misleading, requiring the policy to infer both the current decision rule and which past evidence is still valid. The literature is organized around three questions: what changes, how the change unfolds, and how observable the change is, linking ICRL to meta-RL, decision sequence modeling, retrieval-augmented RL, and related approaches.

SourcearXiv AIAuthor: A Run, Ziluo Ding
In-Context Reinforcement Learning under Non-Stationarity: A Survey
Report an error

The correction channel is not available yet. You can copy the article reference below for later.

Correction instructions
Read article

[Submitted on 1 Jul 2026]

Title:In-Context Reinforcement Learning under Non-Stationarity: A Survey

View a PDF of the paper titled In-Context Reinforcement Learning under Non-Stationarity: A Survey, by A Run and Ziluo Ding

View PDF HTML (experimental)

Abstract:The development of decision-pretrained transformers, algorithm distillation, long-context meta-RL, and retrieval-augmented agents has renewed interest in in-context reinforcement learning (ICRL): the ability of a pretrained or fine-tuned decision model to infer latent task rules and improve future behavior from interaction context, without test-time parameter updates. This line of work asks when trial-and-error evidence, rewards, transitions, demonstrations, feedback, or retrieved experience can make learning-like computation happen inside the context window. However, existing surveys of ICRL mainly organize the field around pretraining objectives, architectures, context formats, evaluation protocols, and theoretical mechanisms, while the non-stationary setting remains comparatively underexamined. In changing environments, accumulated context is not merely more evidence about a fixed task: the reward specification, transition kernel, observation channel, action interface, constraint model, or demonstration and memory distribution can fall out of alignment with the current regime. Previously useful context can therefore become stale, misleading, or useful again when an old regime returns. We survey non-stationary ICRL as the problem of adapting through context while deployed policy parameters remain fixed: the policy must infer both the current decision rule and which parts of its accumulated evidence still support that rule. We define non-stationary ICRL, relate it to meta-RL, decision sequence modeling, retrieval-augmented RL, value- and model-aware ICRL, and reward-feedback agents, and organize the literature along three questions: what changes, how the change unfolds, and how observable the change is to the agent.

Subjects:

Artificial Intelligence (cs.AI)

Cite as: arXiv:2607.11906 [cs.AI]

(or arXiv:2607.11906v1 [cs.AI] for this version)

https://doi.org/10.48550/arXiv.2607.11906

arXiv-issued DOI via DataCite

Submission history

From: Ziluo Ding [view email] [v1] Wed, 1 Jul 2026 12:55:17 UTC (672 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled In-Context Reinforcement Learning under Non-Stationarity: A Survey, by A Run and Ziluo Ding

View PDF

HTML (experimental)

TeX Source

view license

Current browse context:

cs.AI

new | recent | 2026-07

Change to browse by:

cs

References & Citations

NASA ADS

Google Scholar

Semantic Scholar

Loading...

Data provided by:

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)

Related Papers

Recommenders and Search Tools

Link to Influence Flower

Influence Flower (What are Influence Flowers?)

Core recommender toggle

CORE Recommender (What is CORE?)

Author

Venue

Institution

Topic

About arXivLabs

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)

Key points and analysis

Article intelligence

EngineersIntermediate

Key points

  • ICRL enables decision models to learn from interaction context without parameter updates.
  • Existing surveys focus on pretraining objectives, neglecting non-stationarity.
  • In non-stationary environments, historical context may become stale and must be dynamically evaluated.
  • The paper systematically surveys non-stationary ICRL along three dimensions: what changes, how, and observability.

Highlights and analysis are generated automatically and may contain errors. Check the original source.

More recent stories on this topic · Models