2026-07-30 04:00 UTCOriginal source2 min readUpdated: 2026-07-30 08:33 UTC

Steering Instruction Hierarchies at Inference Time

A new method called V-Steer uses inference-time editing of cached value vectors to enforce instruction hierarchies in LLMs, improving primary constraint accuracy from under 18% to 92% on role conflict benchmarks without training.

SourcearXiv Computational LinguisticsAuthor: Siqi Zeng, Sewoong Lee, Han Zhao, Julia Hockenmaier

Article intelligence

EngineersAdvanced

Key points

V-Steer is a training-free method that restores instruction hierarchy by editing cached value vectors at inference time.
It uses direct logit attribution to identify heads where lower-priority instructions dominate, then boosts privileged ones.
The method works with fused attention backends and adds only a one-time prefill overhead, with negligible decoding speed impact.
Across 7B to 70B models, V-Steer raises primary constraint accuracy from under 18% to 92% on controlled benchmarks.

Why it matters

This matters because v-Steer is a training-free method that restores instruction hierarchy by editing cached value vectors at inference time.

Technical impact

May affect model selection, inference cost, product capability, and evaluation benchmarks.

This panel is AI-generated and reviewed for accuracy.

-->

[Submitted on 28 Jul 2026]

Title:Steering Instruction Hierarchies at Inference Time

View a PDF of the paper titled Steering Instruction Hierarchies at Inference Time, by Siqi Zeng and 3 other authors

View PDF HTML (experimental)

Abstract:Instruction hierarchies are a core safety assumption of language model deployment: higher priority inputs, such as system prompts, should override conflicting lower priority inputs from users or tools. Yet frontier LLMs often violate this hierarchy. We introduce V-Steer, a training-free inference time method that restores privileged influence by editing cached value vectors at prompt positions. Using direct logit attribution on the first next token prediction, V-Steer identifies heads where lower priority spans dominate privileged ones, then boosts privileged spans and suppresses conflicting lower priority spans through in-place multiplicative edits to cached V tensors. Since the method acts only on cached values, it remains compatible with fused attention backends and adds only a one time prefill overhead. Across models from 7B to 70B, this attribution guided intervention raises primary constraint accuracy from under 18% up to 92% on controlled role conflict benchmarks, and on broader instruction hierarchy evaluations substantially outperforms prompt only baselines while matching or exceeding SoTA training based methods on 3 of 4 scales of LLMs, with negligible decoding-speed overhead. The code is available at this https URL.

Comments: Published as a conference paper at COLM '26; the first two authors contributed equally to the work. 24 pages, 9 figures

Subjects:

Computation and Language (cs.CL)

Cite as: arXiv:2607.26228 [cs.CL]

(or arXiv:2607.26228v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2607.26228

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Siqi Zeng [view email] [v1] Tue, 28 Jul 2026 20:06:31 UTC (489 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled Steering Instruction Hierarchies at Inference Time, by Siqi Zeng and 3 other authors

View PDF

HTML (experimental)

TeX Source

view license

Current browse context:

cs.CL

new | recent | 2026-07

Change to browse by:

References & Citations

NASA ADS

Google Scholar

Semantic Scholar

Data provided by:

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)