Skip to content
AI News HubLIVE
Original source2 min read

Probe Generalization as Subspace Selection for OOD Deception Detection

Summary

The paper treats out-of-distribution generalization of deception-detection probes as a subspace selection problem. Using Llama-3.1-8B-Instruct, the authors show that projecting activations onto a handful of training-distribution principal components enables cross-domain transfer close to probes trained on test data, and an LLM judge can identify transferable directions to close much of the remaining gap.

SourcearXiv Computational LinguisticsAuthor: Daniel Yoo, Adrians Skapars
Probe Generalization as Subspace Selection for OOD Deception Detection
Report an error

The correction channel is not available yet. You can copy the article reference below for later.

Correction instructions
Read article

[Submitted on 1 Jul 2026]

Title:Probe Generalization as Subspace Selection for OOD Deception Detection

View a PDF of the paper titled Probe Generalization as Subspace Selection for OOD Deception Detection, by Daniel Yoo and Adrians Skapars

View PDF HTML (experimental)

Abstract:Linear probes can be used to detect behaviors and concepts inside language model activations, but may fail to transfer to out-of-distribution examples. When studying the generalization performance of Llama-3.1-8B-Instruct probes over 3 held-out deception detection datasets, we find that projecting inputs onto a small subset of principal components (PCs) from the training distribution of activations enables cross-domain transfer that nearly matches the performance of probes trained directly on the test distribution. Furthermore, we find that PC interpretations can be used to find a subset of those transferable PCs. By using an LLM judge to score each PC on whether its most/ least activating examples imply a transferable deception direction, then probing on the highest-scoring PCs, we close the baseline-to-oracle gap by 78% on Insider Trading Report and by 25% on Sandbagging. The directions a source probe weights heavily appear to encode source-specific surface features, while the directions that actually transfer appear to encode the same contrast more abstractly, in a way natural language descriptions can capture. Broadly, our results suggest that the OOD robustness of probes is largely determined by subspace selection.

Subjects:

Computation and Language (cs.CL)

Cite as: arXiv:2609.02893 [cs.CL]

(or arXiv:2609.02893v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2609.02893

arXiv-issued DOI via DataCite

Submission history

From: Daniel Yoo [view email] [v1] Wed, 1 Jul 2026 08:33:10 UTC (1,753 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled Probe Generalization as Subspace Selection for OOD Deception Detection, by Daniel Yoo and Adrians Skapars

View PDF

HTML (experimental)

TeX Source

view license

Current browse context:

cs.CL

new | recent | 2026-09

Change to browse by:

cs

References & Citations

NASA ADS

Google Scholar

Semantic Scholar

Loading...

Data provided by:

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)

Related Papers

Recommenders and Search Tools

Link to Influence Flower

Influence Flower (What are Influence Flowers?)

Core recommender toggle

CORE Recommender (What is CORE?)

Author

Venue

Institution

Topic

About arXivLabs

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)

Key points and analysis

Article intelligence

EngineersAdvanced

Key points

  • Projecting probe inputs onto a small subset of principal components from the training distribution enables OOD deception detection transfer that nearly matches probes trained directly on the test distribution.
  • An LLM judge scores each principal component by whether its most and least activating examples imply a transferable deception direction.
  • The approach closes 78% of the baseline-to-oracle gap on Insider Trading Report and 25% on Sandbagging.
  • Heavily weighted source-probe directions encode source-specific surface features, while transferable directions encode the deception contrast more abstractly.

Highlights and analysis are generated automatically and may contain errors. Check the original source.

More recent stories on this topic · Models

Probe Generalization as Subspace Selection for OOD Deception Detection | AI News Hub