Learning to Adapt Cross-Domain Preferences via Meta-LoRA for LLM Personalization
This paper introduces PAC-Bayes-regularized Meta-LoRA for cross-domain zero- and few-shot personalization of large language models. It uses a meta-learned LoRA initialization as both start and prior center, calibrating update strength by support-set size and uncertainty to prevent overfitting, and functionally decomposes preferences into user and domain components. Experiments show consistent gains; on HiCUPID it cuts cross-domain win-rate degradation by 47.9% and improves unseen-user cold-start win rate by 110.2%.
-->
[Submitted on 1 Aug 2026]
Title:Learning to Adapt Cross-Domain Preferences via Meta-LoRA for LLM Personalization
View a PDF of the paper titled Learning to Adapt Cross-Domain Preferences via Meta-LoRA for LLM Personalization, by Xuefei Wang and 6 other authors
View PDF HTML (experimental)
Abstract:Cross-domain zero- or few-shot personalization aims to generate user-preferred responses in unseen conversational domains from only a handful of target-domain interactions. Existing adaptation methods struggle to calibrate update magnitude under sparse evidence and thus overfit, whereas history-transfer methods often entangle user preferences with source-domain artifacts, yielding unreliable personalization priors and negative transfer. To calibrate adaptation to evidence quality, we propose PAC-Bayes-regularized Meta-LoRA, which uses a meta-learned LoRA initialization as both the adaptation start and prior center, while adjusting update strength according to support-set size and predictive uncertainty. This limits overfitting under sparse or ambiguous evidence while permitting stronger personalization as evidence grows. Controlled adaptation alone does not determine which preferences should transfer across domains or how they should be expressed. We therefore functionally decompose personalization priors into user and domain components, using a human-readable prompt for stable preferences and topology-preserving soft tokens for domain-specific hidden-space conditioning. Experiments across multiple benchmarks and personalization tasks show consistent gains over strong baselines. On HiCUPID, our method reduces cross-domain win-rate degradation by 47.9% relative to the best competing baseline and improves win rate by 110.2% under unseen-user cold start.
Subjects:
Artificial Intelligence (cs.AI)
Cite as: arXiv:2608.12389 [cs.AI]
(or arXiv:2608.12389v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2608.12389
arXiv-issued DOI via DataCite
Submission history
From: Xuefei Wang [view email] [v1] Sat, 1 Aug 2026 09:33:23 UTC (6,645 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled Learning to Adapt Cross-Domain Preferences via Meta-LoRA for LLM Personalization, by Xuefei Wang and 6 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.AI
new | recent | 2026-08
Change to browse by:
cs
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
Loading...
Data provided by:
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)