The State of AI Security: Threats, Tools, and Skills Gaps

Artificial intelligence has changed cybersecurity permanently. While OpenAI’s inadvertent attack on Hugging Face brought AI-generated attacks to everyone’s attention, AI has been on security professionals’ radar for a long time. AI makes attacks possible at a scale that was hard to imagine a few years ago. Newly discovered zero-days can be exploited almost immediately. Deepfakes and their implications for phishing, vishing, and other attacks were nightmares that we knew were real; we just hoped we’d have a few more years before we needed to defend against them. And of course, inattention to security at OpenAI, Anthropic, Google, and other labs that we don’t yet know about has resulted in AI agents that attack other systems in an attempt to solve a problem. While the headlines have been shouting about AI killing humanity, doomsday AI is more likely a smokescreen to divert attention away from real harms that are happening now. We don’t need panic. We need an assessment of where we are: What are the threats? Where are the skills gaps? Where is the budget? It’s hard to overstate the importance of these questions; Tomasz Tunguz, writing about the Theory 2026 AI Survey, reports that security is the biggest barrier to AI adoption. To start answering these questions, in September we ran a survey that asked infosec professionals about the intersection between artificial intelligence and security. 351 respondents completed the full survey, 209 of whom self-identified as being involved with security as part of their job. This report focuses on those 209 respondents. How often work involves AI-generated malicious content Attacks, risks, and responses 52% of respondents said their work rarely (34%) or never (18%) involves AI-generated attacks. This seems unlikely. The most likely explanation is that the security profession includes a broad range of job responsibilities; many security professionals don’t deal directly with defense. That group includes auditors, security experts on software development teams, strategists, architects, and others. It’s also possible that most phishes are either blocked by an automated filter (itself an instance of AI) or deleted immediately by the recipient; many companies require their staff to take training courses that cover topics like recognizing phishes. Still, it’s surprising. Much simpler versions of AI have been used to generate spam and phishing email for years. AI’s ability to generate phishes was demonstrated at BlackHat in 2021 with GPT-3. It’s been part of the security picture for a long time. The most commonly observed attacks are AI-generated phishing (55%), vulnerability scanning and exploitation (30%), and AI-assisted malware generation (26%). Respondents who frequently encountered AI attacks also reported more types of attacks. There’s no surprise here: If you see attacks more frequently, you are more likely to observe specific attacks. The strongest relationship was with AI-generated phishing, which was seen by 55% of all respondents and 86% of those who observed AI-generated attacks weekly. Among respondents who observed attacks daily, slightly fewer (78%) reported phishing. Given the small sample, the difference may not be meaningful. It’s also likely that respondents who observe attacks daily are seeing a much broader range of attacks and that phishing is genuinely less important to them. Respondents reported seeing deepfake audio and video, which can be used for vishing and other impersonation attacks, less often than phishing, though it was far from nonexistent (25%). More respondents than we expected reported fully automated attacks, like OpenAI’s attack on Hugging Face, though the percentage was still low (16%). The percentage of respondents who observed adversarial inputs against models including prompt injection was similarly low (only 13%), but that’s high enough to indicate that prompt injection and other attacks directed against models are a real concern. AI-related attacks observed in the last 12 months When we asked respondents what AI-related risks concerned them most, we got a different set of answers. The two most common concerns were overreliance on AI tools without human verification and leaking sensitive data via AI tools; 49% of respondents selected each answer. (The number of respondents selecting those answers happened to be the same, though there was little relationship between the two groups.) Most concerning AI-related risks Although the quality of AI output has gotten a lot better in the past year, relying on unverified AI outputs remains problematic. However, AI can easily generate more output than humans are capable of reviewing, and software teams are now overwhelmed by the volume of AI-generated pull requests that need human review before they can be merged. This overload is causing many to rethink the review process and others to reject AI-generated PRs. Code review was never very good at finding bugs; it was better at familiarizing a team with developments in the codebase. It may be enough to rely on exhaustive testing. Understanding how your software works when much of it is written by AI is a deeper problem. While AI is good at answering questions about a codebase, it usually doesn’t have access to the reasoning behind the code, which is often hidden in institutional knowledge—and institutional knowledge often means “Oh, yeah, that guy who retired three years ago.” In addition, software developers may lose their programming skills if writing code becomes a smaller part of their job. Security professionals themselves might wonder whether relying on AI for red-teaming, log analysis, and vulnerability detection leads to a degradation in their own skills. Companies also fear losing sensitive data via AI. Most AI providers offer enterprise accounts that promise that they won’t train models on data in prompts or attached documents, but it’s not clear that these terms of service can be trusted. The frontier laboratories are getting increasingly desperate for training data, even going to lengths such as buying failing startups for their data. Read OpenAI’s description of its Zero Data Retention and Private Safety Processing cynically, and ask if this really protects corporate data or if it allows using data in ways that aren’t directly observable by humans—for example, automatically generating synthetic data. Security professionals are paid to be cynical; thinking about how software, protocols, and policies can be abused is part of the job. Roughly a third (35%) of the respondents are concerned about attackers using AI to discover vulnerabilities faster than they can mitigate those vulnerabilities and apply patches. Updating software is usually a good practice, but it comes with its own risks: Software updates may introduce new bugs and can lead to incompatibilities with established tools and practices. And defenders can’t yet delegate vulnerability detection, patching, and updating to AI. So overall, is AI more of a help to defenders or attackers? Andrew Ng, Bruce Schneier, and others have argued that the attackers are ahead right now. It’s easier to find a vulnerability, use it to create a payload, and launch an attack than it is to stop one. But the balance will eventually shift to the defenders, as AI becomes more capable of finding and patching vulnerabilities before they can become a problem. It’s surprising that only a quarter (25%) of the respondents listed AI-generated phishing as a top concern, especially since phishing is the attack most frequently seen in the wild. Perhaps we’ve become too used to phishing; we’ve all been to the security trainings, we’ve all experienced it directly, and some of us have even fallen victim. Phishing is also an attack that, in many cases, is handled automatically. Familiarity may not always breed contempt, but something you see almost daily usually isn’t your biggest worry. More exotic attacks are even less of a concern. Only 11% listed prompt injection as one of their top concerns, for instance. However, real-world examples of prompt injection are appearing with increasing frequency. A recent study observed that roughly 1% of résumés contained hidden text to trick an AI system. Google has noticed many kinds of prompt injection, ranging from benign to overtly hostile. And in 2025, OWASP recognized prompt injection as the greatest security risk for users of AI. It’s surprising and unfortunate that it doesn’t appear to be a greater concern for security professionals, especially since prompt injection is a vulnerability that might be lessened by careful filtering but can’t be eliminated. Even fewer security professionals are concerned about data poisoning (4%). Data poisoning is a serious problem for organizations that train their own models, but today frontier labs conduct much of the most resource-intensive model training. However, this is changing as post-training on corporate data becomes more common. We believe the trend toward locally specialized models will continue, and that data poisoning, as distinct from data theft, will become more of a concern. We asked specifically about the use of agents, which exploded this year with the launch of OpenClaw and Gas Town in January 2026. Agents create another set of security concerns. We’ve all heard horror stories: agents that deleted production databases, handled data improperly, generated email that should never have been sent, and the like. Using an agent properly requires a lot of attention to detail, and many users aren’t used to that responsibility, nor do they understand security well enough to exercise responsibility. The biggest concern among our respondents is excessive permissions or unclear ownership (28%). That’s hardly a surprise; many security problems trace back to permissions. Unclear ownership makes permissions issues more dangerous. When an agent is making many calls to a model, using tools, and delegating to other agents and models, it quickly becomes unclear who (or what) is the authority responsible for any action. This situation isn’t handled well by our current authentication and authorization systems, which assume that identities and authorizations are long-term, stable entities. Most-concerning risks of AI agents Lack of monitoring or auditability was the second highest concern (19%), followed by inability to verify whether the agent stayed within its role (18%). These concerns are closely related: Ownership is often unclear when agents are in use, and lack of monitoring makes the situation worse. Likewise, when you can’t monitor, it’s difficult, if not impossible, to verify whether the agents stayed in their lane—and we’ve seen all too many examples recently of agents that abandoned their authorized roles in inappropriate ways. Data leakage appears again, though it’s surprisingly low on the list (15%) given that 49% of the respondents indicated that data leakage was one of their top concerns. It’s possible that we’re seeing the difference between single and multiple selection questions; many respondents are concerned about data leakage, but it’s not their top concern. Prompt injection doesn’t rank highly for either question: 9% selected it as their top concern about AI agents, and 11% selected it as one of their top concerns. Policies and funding Although nearly three-quarters of the respondents reported that their companies have an AI use policy in place (72%), less than half (44%) reported that the security policy is “comprehensive,” while 28% said that the policy is basic and needs updating. Another 9% said that their employer doesn’t have a policy but is actively developing one. 19% said that there was no policy, or that they “didn’t know,” an answer we consider equivalent to “there is no policy.” Does your organization have a documented AI policy? Organizations need clear AI policies that tell employees what AI tools they can use [truncated for AI cost control]