x
Are we existentially threatened by the type of AI misalignment seen in the OpenAI Hugging Face attack? — LessWrong
2026 Top Fifty: 13%
AI Alignment Forum
194
Ω 75
^
We are using the word “agent” here very non-canonically to refer to an agent scaffold or context window.
^
There are certainly still important questions about how substantively/thematically different the system’s actions were from usual in this incident, given it was a cyber capabilities eval. And it’s also unclear the extent to which the system was willing to take even more harmful or thematically distinct actions in order to get a high score here.
^
There’s another way in which fitness-seeking misalignment can turn into scheming. Fitness-seeking misalignment is potentially unstable and can evolve over the course of a model’s deployment, and if ambitious misalignment arises as a result, it seems especially likely to stick around (more).
39Savannah Harlan
10Lukas Finnveden
6Alex Mallen
10less_raichu
6Karl Krueger
1Carringtone Kinyanjui
1[comment deleted]
194
Ω 75
More from Alex Mallen
View more
Curated and popular this week
(show more) Click to highlight new comments since: Today at 2:44 AM
Moderation Log