2026-07-07 04:00 UTCOriginal source2 min readUpdated: 2026-07-07 08:13 UTC

Reinforcement Learning for Data-Efficient Code-Switched ASR

Researchers propose a reinforcement learning with verifiable rewards (RLVR) method to adapt audio-language models for code-switched automatic speech recognition. Using only 10% of the data, RLVR matches the performance of full-dataset supervised fine-tuning on Qwen2-Audio across 10 language pairs, with gains transferring zero-shot to human-recorded speech.

SourcearXiv Computational LinguisticsAuthor: Ziwei Ye, Peter Vickers

-->

[Submitted on 2 Jul 2026]

Title:Reinforcement Learning for Data-Efficient Code-Switched ASR

View a PDF of the paper titled Reinforcement Learning for Data-Efficient Code-Switched ASR, by Ziwei Ye and 1 other authors

View PDF HTML (experimental)

Abstract:Audio-language models can be prompted for code-switched speech, but their decoding is not optimized for code-switching and often fails at language boundaries. We propose a practical reinforcement learning with verifiable rewards recipe for data-efficient adaptation of audio-language models to code-switched ASR using group relative policy optimization, combining an error rate reward with a script fidelity reward that penalizes wrong writing systems and a two-pass draft-and-refinement procedure. Using Qwen2-Audio as a reproducible testbed across 10 language pairs, training on only TTS code-switched speech, we show that RLVR with 10% of the data matches LoRA supervised fine-tuning trained on the full dataset, with the largest gains on typologically distant pairs. The error rate reward eliminates translation errors while the script fidelity reward separately reduces script contamination without degradation. These gains transfer zero-shot to a human-recorded code-switching corpus.

Comments: Accepted at Interspeech 2026

Subjects:

Computation and Language (cs.CL); Sound (cs.SD)

Cite as: arXiv:2607.02757 [cs.CL]

(or arXiv:2607.02757v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2607.02757

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Peter Vickers [view email] [v1] Thu, 2 Jul 2026 20:44:53 UTC (5,114 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled Reinforcement Learning for Data-Efficient Code-Switched ASR, by Ziwei Ye and 1 other authors

View PDF

HTML (experimental)

TeX Source

view license

Ancillary-file links:

Ancillary files (details):

csfleurs_read_test_all_rewards_10pct_sampling.json

csfleurs_read_test_lora_100pct_sampling.json

csfleurs_read_test_raw_qwen2.json

eval_switchlingua_csfleurs_xtts_train_cgpr_plus_n4625_e4_twostep_novad_all_n500000_20260217_095543.json

eval_switchlingua_csfleurs_xtts_train_format_n100_e4_all_n500000_20260219_170200.json

eval_switchlingua_lora_csfleurs_xtts_train_n4625_e8_s42_n999999_20260223_142343.json

Current browse context:

cs.CL

new | recent | 2026-07

Change to browse by:

cs cs.SD

References & Citations

NASA ADS

Google Scholar

Semantic Scholar

Data provided by:

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)