Skip to content
AI News HubLIVE
Original source2 min read

Shared SFT Lessons Across Alignment, Model Organisms, and Toy Models

Summary

This research investigates the transferability of supervised fine-tuning (SFT) lessons across alignment training, model organisms, and toy models. Through three experiments, the authors demonstrate that lessons in behavior generalization, capability preservation, and robustness can transfer across domains, suggesting researchers should borrow techniques from other areas.

SourcearXiv Machine LearningAuthor: Anton de la Fuente, Arthur Conmy
Shared SFT Lessons Across Alignment, Model Organisms, and Toy Models
Report an error

The correction channel is not available yet. You can copy the article reference below for later.

Correction instructions
Read article

[Submitted on 28 Jul 2026]

Title:Shared SFT Lessons Across Alignment, Model Organisms, and Toy Models

View a PDF of the paper titled Shared SFT Lessons Across Alignment, Model Organisms, and Toy Models, by Anton de la Fuente and Arthur Conmy

View PDF HTML (experimental)

Abstract:Alignment training, model organisms, and toy models are usually treated as separate research areas. But projects in all three frequently use supervised fine-tuning (SFT) to pursue the same underlying goals. When projects share a goal, we should test whether lessons learned from one area transfer to the other areas. We study three such transfers, each taking a lesson developed in one SFT setting and testing it in another. First, we port a lesson about behavior generalization from alignment training into toy models. Training on the reason for a behavior, as in Teaching Claude Why, can make the behavior generalize better than training on examples of the behavior alone. Second, we port a lesson about capability preservation from model organisms into the Model-Spec Midtraining alignment setting. SFT on outputs written by a model other than the student (off-model outputs) can damage capabilities when trained on. Mixing in benign on-model (and on-policy) data into our training can prevent most of this damage while still embedding the target behavior. Third, we port a lesson about robustness from model organisms into the same alignment setting. We find that follow-up benign SFT can erase the alignment behavior while preserving capabilities, showing that capability preservation alone does not ensure robustness to subsequent training. Our work illustrates how porting SFT lessons between different research fields can uplift them all, suggesting more researchers should borrow techniques from outside their own areas.

Comments: 31 pages, 13 figures

Subjects:

Machine Learning (cs.LG)

Cite as: arXiv:2607.26173 [cs.LG]

(or arXiv:2607.26173v1 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2607.26173

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Anton De La Fuente [view email] [v1] Tue, 28 Jul 2026 18:29:45 UTC (1,977 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled Shared SFT Lessons Across Alignment, Model Organisms, and Toy Models, by Anton de la Fuente and Arthur Conmy

View PDF

HTML (experimental)

TeX Source

view license

Current browse context:

cs.LG

new | recent | 2026-07

Change to browse by:

cs

References & Citations

NASA ADS

Google Scholar

Semantic Scholar

Loading...

Data provided by:

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)

Related Papers

Recommenders and Search Tools

Link to Influence Flower

Influence Flower (What are Influence Flowers?)

Core recommender toggle

CORE Recommender (What is CORE?)

IArxiv recommender toggle

IArxiv Recommender (What is IArxiv?)

Author

Venue

Institution

Topic

About arXivLabs

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)

Key points and analysis

Article intelligence

EngineersAdvanced

Key points

  • Porting behavior generalization lessons from alignment to toy models: training on reasons improves generalization.
  • Porting capability preservation from model organisms to alignment: mixing benign on-model data mitigates capability damage.
  • Subsequent benign SFT can erase alignment behavior, showing capability preservation alone does not ensure robustness.

Highlights and analysis are generated automatically and may contain errors. Check the original source.

More recent stories on this topic · Models