2026-07-09 04:00 UTCOriginal source2 min readUpdated: 2026-07-09 08:22 UTC

Behavior Leverage Imbalance in Multi-Teacher On-Policy Distillation

This paper analyzes over-calling in multi-teacher on-policy distillation for tool-using language models and proposes Soft Clamp, a per-token divergence calibration method that reduces over-calling from 13.7% to 9.0% while matching decision accuracy.

SourcearXiv Computational LinguisticsAuthor: Jiabin Shen, Guang Chen, Chengjun Mao

-->

[Submitted on 8 Jul 2026]

Title:Behavior Leverage Imbalance in Multi-Teacher On-Policy Distillation

View a PDF of the paper titled Behavior Leverage Imbalance in Multi-Teacher On-Policy Distillation, by Jiabin Shen and 2 other authors

View PDF HTML (experimental)

Abstract:Agentic language models must learn when to call tools, when to consume tool responses, and when to answer directly. This makes multi-teacher on-policy distillation a natural training strategy: one teacher can specialize in tool calls, another in direct responses, and the student can learn from both on

its own generated distribution. We show that this strategy can induce a behavior shift that is invisible from aggregate losses alone. In a two-teacher tool-use setting, vanilla generalized knowledge distillation improves tool-call recall but also moves the model toward over-calling, where it calls tools

on examples that should be answered directly. Aggregate explanations are insufficient: tool-call samples do not receive more token exposure, and full-sequence per-token divergence is not larger for the tool-call teacher. We instead analyze behavior leverage imbalance: local token-level signals at mode-

entry and structural positions, such as and function names, can have disproportionate control over the global generation mode. We propose Soft Clamp, a per-token divergence calibration method that dynamically compresses extreme token-level Jensen-Shannon divergence while preserving nonzero

gradients. On APIGen-MT, Soft Clamp reduces over-calling from 13.7% to 9.0% relative to vanilla GKD while matching its decision accuracy. In a BFCL multi-turn diagnostic, it also lowers tool-call loops and repeated calls among GKD variants. These results suggest that multi-teacher OPD should monitor

where teacher signals act, not only how large they are in aggregate.

Comments: 17 pages including appendix, 6 figures

Subjects:

Computation and Language (cs.CL); Machine Learning (cs.LG)

Cite as: arXiv:2607.07050 [cs.CL]

(or arXiv:2607.07050v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2607.07050

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Jiabin Shen [view email] [v1] Wed, 8 Jul 2026 06:26:13 UTC (135 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled Behavior Leverage Imbalance in Multi-Teacher On-Policy Distillation, by Jiabin Shen and 2 other authors

View PDF

HTML (experimental)

TeX Source

view license

Current browse context:

cs.CL

new | recent | 2026-07

Change to browse by:

cs cs.LG

References & Citations

NASA ADS

Google Scholar

Semantic Scholar

Data provided by:

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)