How AI guardrails are impeding the work of offensive cybersecurity researchers
AI companies' guardrails to prevent malicious use are inadvertently hindering legitimate offensive security researchers, as seen in recent U.S. export controls on Anthropic's models.
For months, AI giants have devised special vetted programs and strict guardrails to limit the use of their models by malicious hackers. But these limits are now hindering the work of legitimate network defenders, as well as that of offensive cybersecurity researchers.
In June, the U.S. government slapped export control restrictions on Anthropic’s much-hyped AI models Mythos and Fable. The move was prompted at least in part by a report that claimed it was possible to bypass the models’ guardrails designed to prevent users from using them to build and execute malicious cyberattacks.