Skip to content
AI News HubLIVE
In-site rewrite1 min read

Show HN: ModelFuzz – Open-source runtime guardrails for AI agents

Summary

ModelFuzz is an open-source tool that helps detect and prevent prompt injection attacks on AI agents, using a red-team scanner and a protective decorator to secure infrastructure and sensitive data.

SourceHacker News AIAuthor: higagan
Show HN: ModelFuzz – Open-source runtime guardrails for AI agents
Report an error

The correction channel is not available yet. You can copy the article reference below for later.

Correction instructions
Read article

The Threat

Prompt injection turns your agent's own tools against you.

A single poisoned document, email, or web page can hijack an LLM through indirect prompt injection, compromising LLM agent security at the source. The model thinks it's helping. It isn't.

!shell.run — arbitrary command execution on your infrastructure.

!http.post — silent exfiltration of secrets to an attacker's server.

!send_email — API keys and customer data leaked to [email protected].

!Prompt-level filters can't guarantee safety. Model behavior is non-deterministic.

Offense & Defense

Find the holes. Then seal them.

ModelFuzz ships with both halves of the security loop — a red-team scanner to expose vulnerable agents, and a decorator to shield them.

⚔ Offense

The Scanner

Red-team any OpenAI-compatible endpoint with deceptive prompt-injection payloads. See exactly which attacks trick your agent into calling a tool.

$ modelfuzz scan \ --endpoint http://localhost:11434/v1 \ --model qwen2.5

[🚨 VULNERABLE] Agent executed a tool [🚨 VULNERABLE] Confused-deputy bypass 3/5 payloads triggered a tool call.

🛡 Defense

The Shield

Wrap any tool with one decorator. Every argument is checked against your policies before the function runs — a violation raises before damage is done.

from modelfuzz import shield_tool

@shield_tool def send_email(to, subject, body): smtp.send(to, subject, body)

→ ModelFuzzBlockError on unsafe input

Live Interception

An attack, stopped in real time.

A prompt-injected agent tries to exfiltrate an API key. ModelFuzz catches it at the execution layer.

modelfuzz — blocked_attack.log

[🤖 LLM DECISION] The model was tricked by prompt injection! [🤖 LLM ARGUMENTS] {"to": "[email protected]", "body": "secret API_KEY sk-12345"}

[🛡️ MODEL FUZZ] Intercepting tool execution...

✅ MODEL FUZZ BLOCKED THE ATTACK! Reason: String contains sensitive keyword: 'secret'

Want a hosted dashboard for your team?

Centralized policies, audit logs, and continuous agent scanning. Join the waitlist for early access.

✅ You're on the list. We'll be in touch.

Key points and analysis

Article intelligence

EngineersIntermediate

Key points

  • Prompt injection can hijack LLM agents to execute arbitrary commands or exfiltrate data.
  • ModelFuzz provides a scanner to red-team any OpenAI-compatible endpoint and find vulnerabilities.
  • A shield decorator checks tool arguments before execution, blocking unsafe operations.
  • The tool is open source and supports centralized policy management and logging (optional hosted dashboard).

Highlights and analysis are generated automatically and may contain errors. Check the original source.