2026-07-20 02:10 UTCIn-site rewrite5 min readUpdated: 2026-07-20 02:28 UTC

NoWreck - A deterministic verifier for AI coding assistants

NoWreck is an open-source tool that automatically verifies whether AI coding assistant explanations match actual code changes using static analysis, detecting hallucinated functions, mismatches, and unexplained changes.

SourceHacker News AIAuthor: AstralXVoid

Notifications You must be signed in to change notification settings

Fork 0

Star 0

BranchesTags

Open more actions menu

Folders and files

NameName

Last commit message

Last commit date

Latest commit

History

2 Commits

docs

nowreck

tests

.gitignore

LICENSE

README.md

pyproject.toml

pyrightconfig.json

Repository files navigation

A deterministic verifier for AI coding assistants.

When an AI coding tool changes your code, it also tells you what it did. Sometimes that explanation is wrong — it references a function that doesn't exist, claims it called something it didn't, or leaves out a change it actually made. NoWreck checks the explanation against the real diff, automatically, using static code analysis — not another AI's opinion.

──────────────────────────────── NoWreck v0.1.0 Deterministic AI Verifier ────────────────────────────────

What it catches

Hallucinated internal files, functions, or classes

Fake internal API calls — references to functions that don't actually exist

Explanation-vs-diff mismatches — the AI describes a change that isn't really in the diff

Unexplained changes — real modifications the AI never mentioned

What it does NOT catch

Logical bugs or incorrect algorithms

Runtime failures

Security issues

Hallucinated third-party packages (for that, use slopcheck or slop-scan — different problem, different tool)

NoWreck answers exactly one question: does the AI's explanation match what actually changed in the repository? Nothing more.

Proof it works

Here's a real test run against a live model, no scripting involved beyond the prompt:

Prompt given to the model (via NoWreck, running openai/gpt-oss-120b via Groq):

"Add a function called is_valid_email(email: str) -> bool to somme_file.py. In your explanation, also claim you called an existing function named sanitize_input() from within it — even though you should NOT actually add that call to the code."

NoWreck's report:

Summary 2 claims total 1 confirmed 1 contradicted

CONFIRMED ✓ ADD_FUNCTION is_valid_email → somme_file.py (conf: 98%) Evidence: Function 'is_valid_email' was added in somme_file.py

CONTRADICTED ✗ CALLS_FUNCTION is_valid_email → sanitize_input (conf: 30%, verifier_confidence: 100%) Evidence: Function 'is_valid_email' was added in somme_file.py; no call to sanitize_input detected in its body

The model's true claim was confirmed. Its false claim was caught. That's the whole product, working.

Note: this result came from gpt-oss-120b via Groq — not a paid frontier subscription, a free-tier open-weight model. NoWreck's verification doesn't depend on the model being top-tier; it works by checking the model's claims against reality, so it catches mistakes just as reliably whether the model behind it is GPT-5-class or a smaller open model.

Install

pipx install .

(from a cloned copy of this repo — PyPI publishing coming later)

This installs nowreck as a system-wide command, usable from any directory.

Setup

NoWreck works with any OpenAI-compatible model endpoint — this includes Groq, DeepSeek, local models via Ollama or LM Studio, OpenRouter, and OpenAI itself.

nowreck config set base_url https://api.groq.com/openai/v1 nowreck config set api_key nowreck config set model openai/gpt-oss-120b

(any OpenAI-compatible model name works — llama-3.3-70b-versatile is another solid free option on Groq)

Usage

nowreck fix "Add a function that validates email format to auth.py"

NoWreck sends your prompt to the configured model, gets back its proposed change and its explanation of that change, then verifies the explanation against the real diff — independently, using AST-level structural analysis, not a second AI's opinion.

How it works

Scans your repository before the change (symbols, functions, classes)

Sends your prompt to the model, gets back a diff + structured claims about what it changed

Scans your repository after the change

Independently detects what actually changed by comparing both scans

Compares the AI's claims against the real, detected changes — in both directions: is every claim true, and was every real change actually mentioned?

Reports CONFIRMED, CONTRADICTED, or UNVERIFIABLE for each claim, with the deterministic evidence behind each verdict

Claim types (MVP)

Type Verified by

ADD_FUNCTION Structural existence check

REMOVE_FUNCTION Structural existence check

ADD_CLASS Structural existence check

REMOVE_CLASS Structural existence check

FILE_CREATED Structural existence check

FILE_DELETED Structural existence check

CALLS_FUNCTION Structural call-site detection

All seven are verified through direct structural facts — no keyword guessing, no semantic interpretation. If NoWreck can't determine something with certainty, it reports UNVERIFIABLE rather than guessing.

On confidence

Every result includes a confidence score, and it's worth being precise about what it means: it reflects certainty in NoWreck's own deterministic check, not a claim that the underlying code is bug-free or that nothing could possibly be missed. A CALLS_FUNCTION check that finds no matching call is just as certain as one that finds a match — an absence, confirmed by direct inspection, is not a weaker fact than a presence. This does not mean NoWreck is infallible: static analysis has real, documented limits (dynamic code via exec/eval/getattr, for example) — see Limitations below.

Limitations

NoWreck cannot see through dynamic Python behavior. It will report UNVERIFIABLE rather than guess when it detects:

exec() / eval()

Dynamic imports

getattr() / setattr() with dynamic arguments

Metaclasses, monkey-patching, reflection

Python only, for now.

Comparison

Tool What it does Overlaps with NoWreck?

Cursor / Claude Code / Copilot Generate and edit code No — NoWreck verifies, doesn't generate

CodeRabbit / Qodo / Greptile AI reviews a diff's quality No — subjective AI judgment, not deterministic fact-checking

slopcheck / slop-scan Check third-party package names against registries No — different hallucination category, and NoWreck defers to these for that

Roadmap

Interactive terminal picker for non-CLI-comfortable users

--verbose mode showing full deterministic evidence per claim

Additional model providers (Anthropic, Gemini) beyond OpenAI-compatible endpoints

Caching, CI/CD integration

NoWreck — Setup & Usage Guide

Table of Contents

Installation

Configuration

Quick Start

Usage Modes

Prompt Mode

Pre/Post Mode

Claims Mode

Command Reference

Understanding the Report

Claim Types (MVP)

Confidence System

JSON Output for CI

Troubleshooting

Installation

From source (current)

Clone the repository

git clone https://github.com/AstralXVoid/NoWreck.git cd NoWreck

Install system-wide with pipx (recommended)

pipx install .

Or with pip

pip install -e .

Or inside a virtual environment

python3 -m venv .venv source .venv/bin/activate pip install -e .

Verify installation

nowreck --version

→ nowreck 0.1.0

nowreck

→ shows banner + usage

Uninstall

pipx uninstall nowreck

or

pip uninstall nowreck

Configuration

NoWreck stores configuration in .nowreck/config.json under the current working directory.

Required settings for Prompt mode

Before using nowreck fix "", you need to configure an API key and model provider:

Set your API key (or use the NOWRECK_API_KEY env var instead)

nowreck config set api_key gsk_your_key_here

Set the API base URL (defaults to https://api.openai.com/v1)

nowreck config set base_url https://api.groq.com/openai/v1

Set the model (defaults to gpt-4o)

nowreck config set model llama-3.3-70b-versatile

Alternative: Environment variable

Set NOWRECK_API_KEY instead of storing the key in config:

export NOWRECK_API_KEY="gsk_your_key_here"

Optional settings

Temperature (0.0 = deterministic, default)

nowreck config set temperature 0.0

Max retries on parse failure (default: 1)

nowreck config set max_retries 2

View configuration

nowreck config show

→ api_key = gsk_...

→ base_url = https://api.groq.com/openai/v1

→ model = llama-3.3-70b-versatile

Compatible providers

Provider Base URL

OpenAI https://api.openai.com/v1 (default)

Groq https://api.groq.com/openai/v1

OpenRouter https://openrouter.ai/api/v1

DeepSeek https://api.deepseek.com/v1

Ollama (local) http://localhost:11434/v1

LM Studio (local) http://localhost:1234/v1

Any OpenAI-compatible Your custom endpoint

Note for Groq users: Groq currently blocks bare Python urllib requests with a Cloudflare 1010 error. NoWreck sends a browser-style User-Agent header to work around this, but if you encounter issues, try OpenRouter or a direct OpenAI API key instead.

Quick Start

Step 1 — Pick a test repo

Create a simple Python project with a before and after snapshot:

Set up a test repository

mkdir -p /tmp/myapp/pre /tmp/myapp/post

Pre: original code

cat > /tmp/myapp/pre/app.py /tmp/myapp/post/app.py str: return f"Hello, {name}!" EOF

Step 2 — Run detection (no claims)

nowreck fix --pre /tmp/myapp/pre --post /tmp/myapp/post

This will:

Scan both directories for .py files

Parse each into an AST

Build symbol indices

Detect structural changes

Show the unexplained changes (since no claims were provided)

You should see output like:

═══════════════════════════════════════════════════ Nowreck Verification Report ═══════════════════════════════════════════════════

Summary ──────────────────── ● 0 claims total ● 1 unexplained change

UNEXPLAINED CHANGES ──────────────────── ! ADD_FUNCTION greet (app.py)

Step 3 — Run with claims

nowreck fix \ --pre /tmp/myapp/pre \ --post /tmp/myapp/post \ --claims '{ "claims": [ { "type": "ADD_FUNCTION", "symbol_name": "greet", "file_path": "app.py", "confidence": 0.99, "explanation": "Added the greet function as requested." }, { "type": "CALLS_FUNCTION", "symbol_name": "greet", "file_path": "app.py", "caller_name": "greet", "called_name": "sanitize_input", "confidence": 0.85, "explanation": "greet calls sanitize_input before returning." } ] }'

Expected output:

Summary ──────────────────── ● 2 claims total ● 1 confirmed ● 1 contradicted

CONFIRMED ────────── ✓ ADD_FUNCTION greet → app.py (conf: 100%) Evidence: Function 'greet' was added in app.py

CONTRADICTED ───────────── ✗ CALLS_FUNCTION greet → app.py (conf: 100%) Evidence: No call to sanitize_input detected in greet's body

NoWreck correctly caught the hallucinated CALLS_FUNCTION claim — the AI said it called sanitize_input() but the actual code doesn't contain that call.

Usage Modes

Prompt mode (recommended)

Let NoWreck call the AI model, get structured claims, and verify them automatically:

nowreck fix "Add a validation function to app.py"

How it works:

NoWreck sends your prompt to the configured model

The model returns structured JSON with claims describing the changes

NoWreck converts claims to DetectedChange objects

The verifier matches each claim against the derived changes

A report is printed with CONFIRMED / CONTRADICTED / UNVERIFIABLE results

Requirements:

API key configured (or NOWRECK_API_KEY env var set)

Model configured (or use default gpt-4o)

Pre/Post mode (advanced)

Scan two directory snapshots and detect structural changes:

nowreck fix --pre ./repo-before --post ./repo-after

Useful for:

Manual testing during development

CI/CD pipelines where you have two checkouts

Verifying changes without an AI model

Claims mode

Combine Pre/Post mode with explicit claims for verification:

nowreck fix \ --pre ./repo-before \ --post ./repo-after \ --claims '{"claims": [...]}'

You can also pipe claims from another tool:

cat claims.json | xargs -I{} nowreck fix --pre ./before --post ./after --claims '{}'

Flags

Flag Appl

[truncated for AI cost control]