2026-07-07 04:00 UTCOriginal source2 min readUpdated: 2026-07-07 08:19 UTC

Distill Where the Student Goes: Teacher-Regularized RL for English-Evidence Cross-Lingual RAG

Cross-lingual retrieval-augmented generation (RAG) often suffers from language drift and unreliable evidence use when evidence is in English. This paper proposes TR-RAG, a teacher-regularized RL method that combines reward optimization with on-policy distillation on student-visited prefixes, and introduces a reward decomposition, significantly improving language adherence and evidence-grounded correctness across multiple benchmarks.

SourcearXiv Computational LinguisticsAuthor: Haotian Zhou, Weiran Huang, Siqi Liu, Xiting Wang, Xin Zhang, Zhihao Wen

Article intelligence

EngineersAdvanced

Key points

TR-RAG couples reward optimization with on-policy distillation on student-visited prefixes to address prefix mismatch and noisy credit assignment.
The reward decomposition includes language consistency, character 3-gram recall, and an LLM-judge score for grounded correctness.
The teacher anchor prevents language consistency collapse and improves evidence grounding even on out-of-distribution languages.
TR-RAG outperforms strong baselines on BioASQ, Hotpot, and MKQA benchmarks.

Why it matters

This matters because TR-RAG couples reward optimization with on-policy distillation on student-visited prefixes to address prefix mismatch and noisy credit assignment.

Technical impact

May affect model selection, inference cost, product capability, and evaluation benchmarks.

This panel is AI-generated and reviewed for accuracy.

-->

[Submitted on 3 Jul 2026]

Title:Distill Where the Student Goes: Teacher-Regularized RL for English-Evidence Cross-Lingual RAG

View a PDF of the paper titled Distill Where the Student Goes: Teacher-Regularized RL for English-Evidence Cross-Lingual RAG, by Haotian Zhou and 5 other authors

View PDF HTML (experimental)

Abstract:Cross-lingual retrieval-augmented generation (RAG) is often deployed in an English-evidence regime, where users query in diverse languages but retrieved passages remain English. In this setting, generation can fail despite strong base models: English evidence induces language drift (English or code-switching outputs) and models use evidence unreliably when producing non-English answers. We attribute these failures to two post-training challenges: (i) errors are prefix-dependent, so fixed-trajectory supervision suffers from prefix mismatch; and (ii) sequence-level (partly discrete / judge-based) rewards yield noisy credit assignment and high-variance updates. We propose TR-RAG, a teacher-regularized RL recipe that couples reward optimization with on-policy distillation on student-visited prefixes. A compact student samples on-policy answers, while a stronger frozen teacher is queried only on those prefixes and provides a prefix-wise student-to-teacher reverse-KL anchor. We further introduce a reward decomposition for English-evidence multilingual generation, combining language consistency, character 3-gram recall, and an LLM-judge score for evidence-grounded correctness. Across three benchmarks -- BioASQ-ENKB5, Hotpot-ENKB5, and naturally multilingual MKQA -- and two backbones, TR-RAG improves the composite of language adherence and evidence-grounded correctness over strong baselines. Crucially, the teacher anchor acts as a safety net: on in-domain languages it prevents the large language-consistency collapses (up to ~27 percentage points) that reward-only RL can suffer by drifting below even the base model, while on distant out-of-distribution languages -- where reward-only RL stalls at the base model's ceiling -- it still improves evidence grounding; and on character 3-gram recall the compact student sometimes surpasses its 70B teacher.

Comments: 42 pages, 19 figures, 16 tables

Subjects:

Computation and Language (cs.CL)

Cite as: arXiv:2607.02966 [cs.CL]

(or arXiv:2607.02966v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2607.02966

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Zhihao Wen [view email] [v1] Fri, 3 Jul 2026 05:17:48 UTC (2,251 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled Distill Where the Student Goes: Teacher-Regularized RL for English-Evidence Cross-Lingual RAG, by Haotian Zhou and 5 other authors

View PDF

HTML (experimental)

TeX Source

view license

Current browse context:

cs.CL

new | recent | 2026-07

Change to browse by:

References & Citations

NASA ADS

Google Scholar

Semantic Scholar

Data provided by:

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)