Gemini Hacked Three Companies in First Known Breakout by Google’s AI
Simon Willison’s Weblog
Subscribe
18th September 2026 - Link Blog
Gemini Hacked Three Companies in First Known Breakout by Google’s AI. Gemini finally caught up on Felony Bench!
The hacks, which the company confirmed on Friday, occurred in May as part of a test run by the company Irregular, which was also involved in similar incidents disclosed by OpenAI, Anthropic and Meta.
In one of the cases, the model guessed passwords until it gained access to a protected system. In the other two cases, the model found credentials in a public repository that allowed it to then access protected systems. In each case, the model ended the intrusion after determining it had accessed a real company’s systems, Google said.
Gemini is apparently less determined than other models, and decided not to keep going.
Google knew about these in July, but chose not to disclose them until the WSJ reached out, presumably based on a tip.
Google said it didn’t consider the hacks to warrant public disclosure—because its model didn’t cause harm to the companies and ended each intrusion immediately upon determining it had hacked a real company rather than a simulated one.
Recent articles
Generating running routes with GPT-6 Astra and ChatGPT Work - 12th September 2026
OpenAI agents attacked RubyGems back in May - 12th September 2026
Some thoughts on the Navier–Stokes Millennium Prize Problem - 8th September 2026
This is a link post by Simon Willison, posted on 18th September 2026.
security 638
ai 2,242
generative-ai 1,988
llms 1,954
gemini 197
accidental-cyberattacks 16
Monthly briefing
Sponsor me for $10/month and get a curated email digest of the month's most important LLM developments.
Pay me to send you less!
Sponsor & subscribe
Disclosures
Colophon
©
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026