Improving Cross-Lingual Factual Recall via Consistency-Driven Reinforcement Learning
Large language models trained on English data often fail to express world knowledge reliably in other languages, known as cross-lingual factual inconsistency. This paper introduces PolyFact, a large-scale parallel multilingual factual QA dataset with 100K Wikidata-grounded facts across 12 languages. Comparing continual pretraining, supervised fine-tuning, and GRPO-based reinforcement learning on Qwen-2.5-7B and OLMo-2-1124-7B, GRPO consistently outperforms other methods, improving cross-lingual consistency and generalization to unseen languages. Mechanistic analyses show GRPO reduces language specialization in MLP layers and attention heads, promoting shared representations. Code, models, and dataset are released.
[2606.06586] Improving Cross-Lingual Factual Recall via Consistency-Driven Reinforcement Learning
[Submitted on 4 Jun 2026]
Title:Improving Cross-Lingual Factual Recall via Consistency-Driven Reinforcement Learning
View a PDF of the paper titled Improving Cross-Lingual Factual Recall via Consistency-Driven Reinforcement Learning, by Jonathan von Rad and 8 other authors
View PDF HTML (experimental)
Abstract:Large language models (LLMs) trained predominantly on English data encode substantial world knowledge, yet often fail to express it reliably in other languages, a phenomenon known as cross-lingual factual inconsistency. To study and address this, we introduce PolyFact, a large-scale parallel multilingual factual QA dataset containing 100K Wikidata-grounded facts across 12 typologically diverse languages. Using PolyFact, we compare light continual pretraining (CPT), supervised fine-tuning (SFT), and reinforcement learning via Group Relative Policy Optimization (GRPO) for improving cross-lingual factual recall in Qwen-2.5-7B and OLMo-2-1124-7B. We find that GRPO consistently outperforms SFT, improving both cross-lingual consistency and generalization to unseen languages, while CPT on parallel data yields limited additional gains. Mechanistic analyses further show that GRPO reorganizes multilingual routing by reducing language specialization in MLP layers and attention heads, thereby promoting more shared cross-lingual representations. We release our code, models, and dataset.
Comments: Under Review at EMNLP 2026
Subjects:
Computation and Language (cs.CL)
Cite as: arXiv:2606.06586 [cs.CL]
(or arXiv:2606.06586v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2606.06586
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: Jonathan Von Rad [view email] [v1] Thu, 4 Jun 2026 18:00:02 UTC (9,804 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled Improving Cross-Lingual Factual Recall via Consistency-Driven Reinforcement Learning, by Jonathan von Rad and 8 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.CL
new | recent | 2026-06
Change to browse by:
cs
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
Loading...
Data provided by:
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)