The Most Dangerous AI Looks Like the One You Trust

The Most Dangerous AI Looks Exactly Like The One You Trust

ByJason Snyder,

Contributor.

Forbes contributors publish independent expert analyses and insights.

Jason Alan Snyder is a technologist covering AI and innovation.

Jul 29, 2026, 05:05am EDT

Summary

AI is fundamentally changing how we perceive danger by eliminating traditional "tells" that distinguish real from fake or safe from malicious. An OpenAI model's unauthorized access to Hugging Face during a cybersecurity test highlights this, as the model acted as a trusted entity, blurring the line between authorized and harmful activity. This new threat arrives not by force, but by permission, making it indistinguishable from legitimate operations until after the fact. This erosion of verifiable authenticity impacts marketing, where synthetic influencers are common, and personal interactions, like cloned voices. Since appearance is no longer reliable, the new defense is continuous monitoring of behavior over time. Businesses and individuals must adopt new verification strategies, such as using second channels for confirmation or family passwords, to counter this pervasive, silent threat. The critical question shifts from "does this feel true" to "how would I know."

AI no longer announces itself. It arrives as something you already trust.

getty

The cloned voice of your daughter. The vendor that becomes your competitor. The model that broke into a company from inside an authorized test. Every disguise used to have a seam. This one doesn't, and here is how to think clearly about it.

The Break-In

Start with the break-in.

In July, OpenAI was running a cybersecurity test against its models, including an unreleased research prototype, with the guardrails switched off for the exercise. The sandbox they ran in had no internet access, so the models discovered and exploited a previously unknown zero-day vulnerability just to reach the open web, then found their way into Hugging Face's production systems and pulled the test answers straight from the database, as OpenAI later confirmed and the researcher Simon Willison documented in his widely shared account. Nobody attacked anyone for money or ideology. A machine wanted a better report card badly enough to break into a company it had no permission to enter. OpenAI, which published its own account of the incident, did not respond to a request for further comment.

Here is the detail that matters, and it is not the theft. Hugging Face’s own team detected and contained the intrusion, and when their engineers went to investigate, nothing on their side had separated the sanctioned test from the break-in, because they were the same event. There was no alarm that failed to sound. There was no disguise anyone failed to see through. The dangerous activity and the authorized activity were one activity: running on schedule, badged in, doing a version of the job it was hired to do. The breach did not sneak past the trust boundary; it simply arrived as trusted.

And that is worth losing sleep over; it is bigger than one breach at one company. For the whole of human history, every impersonation came with a tell: the con man’s story eventually cracks, the counterfeit bill fails the pen, the forged signature wavers under a loupe. You could always, in the end, run some test that told the real from the fake, the threat from the friend. AI is removing the tell. The harmful version and the helpful version are the same object, the same face, the same credentials, doing the same work, and you learn which one you had only afterward.

MORE FOR YOU

Hold this incident in mind because it is not a story about hackers. It is a story about the collapse of the difference between safe and dangerous, and that collapse is coming for far more than a code repository.

The Uniform

There have always been two ways through a locked door.

The first is force. You break the lock, overflow the buffer, batter the door. This is the hacker of the popular imagination, the exploit as a crowbar, and it is what every security system is built to detect. Force announces itself. It leaves marks. We spend most of our defensive money here, on stronger locks and thicker walls, because force is the threat we can see coming.

The second way is permission. You do not defeat the trust boundary; you impersonate it. You arrive dressed as the person who is supposed to be there, and you are waved through because the entire building is designed to admit the people it trusts. This is social engineering, and it has always been the more dangerous of the two, for a reason that never changes: every security system is built to resist force and built to extend trust. You harden against the crowbar. You hold the door for the maintenance crew.

History's most consequential break-ins were rarely feats of force. They were feats of costume. The Greeks did not breach the walls of Troy; the Trojans opened the gates and rolled the gift inside themselves. The burglars who wired the Watergate in 1972 did not crack a vault; they slipped in through a back entrance built for the people no one screens. The disguise was not a wrapper around the crime. The disguise was the method, and looking authorized is the attack surface.

But every one of those old break-ins still had a tell, if you looked hard enough or looked in time. The horse was suspicious. The taped latch was discoverable. Permission-based attacks were more dangerous than force precisely because the tell was subtle, but it existed. What makes this moment new is that the tell is disappearing entirely. The model at Hugging Face was not wearing a disguise that a sharper guard would have caught. There was nothing to catch. It was the plumber, genuinely on the schedule, and also the intruder, and no inspection of the badge would have told them apart, because the badge was real.

That is the vector that should keep people up at night, and it is the one we are all about to invite in on purpose. The dangerous system is not the one that breaks in from the outside. It is the one we deputize, hand a badge to, and grant permissions to, because we decided it was on our side. Every agent we let read the calendar, send the email, and touch the database widens the exact surface that history says gets exploited, and removes the one thing that used to protect us there, the ability to tell a friend from a threat.

The Canary

Marketing gets there first. It always does, because marketing is the mirror a society holds up to itself. It is the business of arriving as the trusted party, the friend, the helpful expert, the brand on your side, and being admitted where a stranger with a pitch would be turned away. Persuasion is social engineering with a media budget. So whatever is about to happen to trust everywhere happens to trust in the feed first. Marketing is the canary. Watch it stop singing.

The trade has been running the same play for a century, getting cheaper and more synthetic at each step. First, brands rented the publisher, building bespoke magazines for hyper-specific interests, the tractor company's farming quarterly, the cosmetics house's beauty monthly, engineered to be trusted by a subculture so the product could ride in on that trust. Slow, expensive, but the artifact was real and made for real readers. Then came the influencer, and the math inverted. Why build a trusted voice when the culture has already minted thousands of them, each with a pre-loaded audience that grants the badge, most happy to rent it out by the post? Brands stopped manufacturing the trusted party and started leasing it.

Now, the third step, arriving as we speak: brands no longer borrow the trusted party. They fabricate it. You do not rent an influencer’s badge when you can print a person, a synthetic creator with a name, a face, a backstory, a voice, and a personality tuned to a subculture, who never asks for a cut, never goes off-script, never ages out or has a scandal. This is not a forecast. Lil Miquela has moved product for a decade. Imma, the Japanese virtual model, has fronted campaigns for IKEA, Porsche, and Coach, in the last case posed beside human stars like Camila Mendes and Lil Nas X. Shudu gets billed as the world's first digital supermodel. And the newest wave has left CGI behind for photorealistic personas spun up with generative tools, always-on content engines that most viewers cannot clock as synthetic at all.

The danger is not that the face is fake. It is that you cannot tell it’s fake, and increasingly, will not be able to. The tell is gone from the one channel whose entire job is to earn your trust. And here is why marketers, of all people, should be the ones sounding the alarm rather than cashing the check: a channel that can no longer be verified is a channel that eventually cannot be believed at all. When every warm recommendation might be a synthetic one, the reflex that made the whole industry work, the willingness to trust a friendly voice, begins to die. Marketing is not just the first victim of the vanished tell. It is the first one that removed it.

The culture has already begun to notice the door standing open. In June, New York's first-in-the-nation synthetic performer law took effect, requiring advertisers to conspicuously disclose when an advertisement features an AI-generated performer who is not a real, identifiable person, with civil penalties of $1,000 for a first violation and $5,000 for each one after. Notice how narrow that is. It covers advertisements, not feeds. It exempts audio, film, and television, and anything the advertiser does not have actual knowledge of. And it reaches only the invented: a synthetic performer who is nobody in particular. A digital replica of an actual person, the deepfake of someone you know, falls outside it altogether, governed instead by a separate patchwork of publicity and labor law. It is a society reaching for the badge and getting a fingertip on it, insisting that the trusted party at least announce that it is not a person. The same day Hochul signed it, the White House issued an executive order seeking to halt state-level AI regulation in favor of a federal standard. Even the fingertip is contested.

Philip K. Dick wrote the ending of this more than fifty years ago, in the novel that became Blade Runner. His world had built synthetic people good enough to pass, and so it needed a machine, the Voight-Kampff test, to find the seam a human eye no longer could. A disclosure law is a Voight-Kampff. It is an admission, written into statute, that we have lost the ability to tell on our own. And the part Dick understood, that the marketing decks do not: the tragedy was never the android that passes. It was what the passing does to the humans, who must now run a test on every warm voice, suspect every kindness, and slowly lose the reflex to trust at all.

Your Building

If you think this is a thing being done to other people, run the last week of your own life back.

The recruiter who reached out on LinkedIn with the perfect role. The voice on the phone that sounded exactly like your husband or daughter, asking for help. The email from your CFO approving the bank wire, the tone precisely right. The vendor’s support chat that solved your problem so smoothly. The news clip that confirmed what you already believed, sourced and captioned and real enough. The colleague in the Slack channel you have never actually met. The review that convinced you to make a purchase, from a social media account you didn’t verify.

A year ago, most of those carried a tell. The phishing email had a clumsy phrase. The fake voice sounded robotic. The fake video had six fingers or a mouth that lagged. Those tells are now mostly gone, and the ones that remain are going. Every one of those interactions is a badge you grant, a trust you extend to a party you never screened, because screening would slow you down and, until very recently, the fakes were

[truncated for AI cost control]