2026-06-18原文2 min readUpdated: 2026-06-18

Want Better Synthetic Data? Steer It: Activation Steering for Low-Resource Language Generation

This paper proposes activation steering as an alternative for synthetic data generation in low-resource languages, using Language Steering and Quality Steering strategies. Experiments on four open-source LLMs and 11 languages show that steering on early layers improves data diversity and downstream task performance.

SourcearXiv Computational LinguisticsAuthor: Jan Cegin, Daniil Gurgurov, Yusser Al Ghussin, Simon Ostermann

[2606.18389] Want Better Synthetic Data? Steer It: Activation Steering for Low-Resource Language Generation

[Submitted on 16 Jun 2026]

Title:Want Better Synthetic Data? Steer It: Activation Steering for Low-Resource Language Generation

View a PDF of the paper titled Want Better Synthetic Data? Steer It: Activation Steering for Low-Resource Language Generation, by Jan Cegin and 3 other authors

View PDF HTML (experimental)

Abstract:Large language models (LLMs) have become an effective tool for synthetic data generation, including for low-resource languages, where generated data can improve downstream task performance. Current best-performing approaches typically rely on few-shot prompting with target-language examples, which increases inference costs and may reduce diversity through lexical anchoring. In this work, we investigate activation steering as an alternative for low-resource synthetic data generation. We study two steering strategies: Language Steering, which targets the linguistic identity of a language, and Quality Steering, which captures well-formedness by contrasting human-written and backtranslated text representations. We evaluate these methods across four open-source LLMs, multiple layers, and 11 typologically diverse languages by generating sentiment and topic classification data and finetuning smaller classifiers. Steering is applied in both zero-shot and few-shot prompting settings and compared against non-steered counterparts. Our results show that steering on early layers consistently improves the diversity of generated data while often yielding stronger downstream model performance, particularly for low-resource languages.

Comments: 25 pages

Subjects:

Computation and Language (cs.CL)

Cite as: arXiv:2606.18389 [cs.CL]

(or arXiv:2606.18389v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2606.18389

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Jan Cegin [view email] [v1] Tue, 16 Jun 2026 18:34:21 UTC (1,318 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled Want Better Synthetic Data? Steer It: Activation Steering for Low-Resource Language Generation, by Jan Cegin and 3 other authors

View PDF

HTML (experimental)

TeX Source

view license

Current browse context:

cs.CL

new | recent | 2026-06

Change to browse by:

References & Citations

NASA ADS

Google Scholar

Semantic Scholar

Data provided by:

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)