NoWreck v0.4.0 – Deterministic AI Verifier

Notifications You must be signed in to change notification settings

Fork 0

Star 1

BranchesTags

Open more actions menu

Folders and files

NameName

Last commit message

Last commit date

Latest commit

History

14 Commits

14 Commits

docs

docs

nowreck

nowreck

test_js_samples

test_js_samples

test_milestone1

test_milestone1

tests

tests

.gitignore

.gitignore

LICENSE

LICENSE

README.md

README.md

pyproject.toml

pyproject.toml

pyrightconfig.json

pyrightconfig.json

use.md

use.md

Repository files navigation

What it catches

Hallucinated functions or classes — the AI claims it added something that isn't there

Fake internal API calls — the AI says it called a function it didn't

Explanation-vs-diff mismatches — the AI describes a change that doesn't match the actual diff

Unexplained changes — real modifications the AI never mentioned

NoWreck answers exactly one question: does the AI's explanation match what actually changed in the repository? Nothing more, nothing less.

Install

pipx install .

(from a cloned copy of this repo — PyPI publishing coming later)

Requires Python 3.10+. Installs nowreck as a system-wide command.

Setup

NoWreck works with any OpenAI-compatible model endpoint — Groq, DeepSeek, Ollama, LM Studio, OpenRouter, or OpenAI itself.

nowreck config set base_url https://api.groq.com/openai/v1 nowreck config set api_key nowreck config set model llama-3.3-70b-versatile

Or set the NOWRECK_API_KEY environment variable instead of storing it in config.

Quick start

Interactive picker — menu-driven interface (great for new users)

nowreck --interactive

Prompt mode — NoWreck calls the model, gets claims, verifies them

nowreck fix "Add a rate-limiting decorator to api/client.py"

Pre/Post mode — advanced: scan two snapshots manually

nowreck fix --pre ./repo-v1 --post ./repo-v2

Pre/Post with claims — verify specific claims against a diff

nowreck fix --pre ./before --post ./after --claims '{"claims": [...]}'

JSON output for CI pipelines

nowreck fix "Add validation to auth.py" --json

View or change configuration

nowreck config show nowreck config set base_url https://api.openai.com/v1

Interactive Mode (in v0.2.0)

Menu-driven interface for users who prefer exploring options without memorizing commands:

nowreck --interactive

This launches a terminal picker where you can:

Choose from Prompt, Pre/Post, or Claims modes

Select your repository path

Configure options interactively

See verification results with formatted output

Great for:

Beginners exploring NoWreck

One-off verifications without typing complex commands

Learning the tool's capabilities

Command reference

Command Description

nowreck Show help / usage

nowreck --version Show version

nowreck --interactive Launch the interactive terminal picker — menu-driven interface for all operations

nowreck fix "" Prompt mode — describe a change in natural language. NoWreck calls the configured model, gets a diff + claims, and verifies them automatically.

nowreck fix --pre PATH --post PATH Pre/Post mode — scan two directory snapshots and detect structural changes. Add --claims JSON to verify specific claims against the detected chang[...]

nowreck fix --json Output structured JSON instead of coloured terminal text (for CI). Works with both prompt and pre/post modes.

nowreck fix --no-colour Disable coloured terminal output.

nowreck config show Display current configuration.

nowreck config set Set a configuration value. Keys: api_key, model, base_url, temperature, max_retries.

How it works

┌─────────────────────────────────────────────────────────┐ │ Prompt mode │ │ │ │ Your prompt ──► AI model ──► diff + claims │ │ │ │ │ ▼ │ │ Pre-scan ──► Symbol index ──► Change Detector │ │ Post-scan ──► Symbol index ────────┘ │ │ │ │ │ Claims ──► Claim Verifier ◄────────────┘ │ │ │ pure comparison — no AI judgment │ │ ▼ │ │ Verification Report │ │ ✓ CONFIRMED ✗ CONTRADICTED ? UNVERIFIABLE │ └─────────────────────────────────────────────────────────┘

NoWreck's verification pipeline has three stages:

Scan — recursively discovers .py and .js files in both snapshots, parses each with the appropriate parser, and builds a symbol index of every function, class, and method.

Python files → parsed with Python's built-in ast module

JavaScript files → parsed with Tree-sitter (the tree-sitter-javascript grammar)

Both parsers produce the same Symbol / SymbolType data shapes, so the rest of the pipeline never knows (or cares) which language produced the data.

Detect — compares the pre and post symbol indices to find structural changes: added/removed functions, classes, files, and new function calls. This produces the single source of truth — a list[DetectedChange] that the verifier references exclusively.

Verify — for each claim from the AI model, the verifier looks for a matching DetectedChange. If one exists with the same type and identity fields → CONFIRMED. If a contradicting change exists (e.g., claim says "added" but detection shows "removed") → CONTRADICTED. If nothing matches → UNVERIFIABLE.

The verifier never parses AST, never queries the symbol index, and never applies AI judgment. Its decisions are purely field-based comparison.

Claim types (MVP)

Claim type What it means Verified by

ADD_FUNCTION A function was added Structural existence check

REMOVE_FUNCTION A function was removed Structural existence check

ADD_CLASS A class was added Structural existence check

REMOVE_CLASS A class was removed Structural existence check

FILE_CREATED A new file appeared File-list diff

FILE_DELETED A file was removed File-list diff

CALLS_FUNCTION A function now calls another AST call-site detection

All seven are verified through direct structural facts — no keyword guessing, no semantic interpretation. If NoWreck can't determine something with certainty, it reports UNVERIFIABLE rather than guessing.

On confidence

Every result includes a confidence score. This reflects NoWreck's certainty in its deterministic check — not a claim that the underlying code is bug-free.

CONFIRMED at 100% — the structural fact was found and matched

CONTRADICTED at 100% — the opposite structural fact was found

UNVERIFIABLE at 50% — no matching fact exists either way

A CALLS_FUNCTION check that finds no matching call is just as certain as one that finds a match. An absence, confirmed by direct inspection, is not a weaker fact than a presence.

This does not mean NoWreck is infallible. Static analysis has real, documented limits — see Limitations below.

Comparison

Tool What it does Overlaps with NoWreck?

Cursor / Claude Code / Copilot Generate and edit code No — NoWreck verifies, doesn't generate

CodeRabbit / Qodo / Greptile AI review of a diff's quality No — subjective AI judgment, not deterministic fact-checking

Agent Verifier (aurite-ai) AI agent skill for code quality/security No — checks code quality, not claim truthfulness

slopcheck / slop-scan Check third-party package names against registries No — different hallucination category

ESLint / Ruff / Black Linting and formatting No — syntax/code style, not structural verification

NoWreck occupies a unique niche: deterministic verification of AI claims about code changes. No other tool does this.

Limitations

Python + JavaScript — NoWreck supports both languages. Python files are parsed with the built-in ast module; JavaScript files use Tree-sitter with the tree-sitter-javascript grammar.

Cannot see through dynamic behavior — exec(), eval(), dynamic imports, getattr()/setattr() with dynamic arguments, metaclasses, monkey-patching, and reflection will all yield UNVERIFIABLE

Simple calls only — detects name() calls, not obj.method() or chained calls

No cross-file resolution beyond direct name matching

No semantic analysis — it verifies structure, not intent

No TypeScript support — TypeScript syntax is not yet supported (files will be skipped during scanning).

Roadmap

Interactive terminal picker for non-CLI users ✅ (done in v0.2.0)

JavaScript core support (Tree-sitter scanner + symbol index) ✅ (done in v0.3.0)

JavaScript polish (generators, export default, IIFEs) ✅ (done in v0.4.0)

--verbose mode showing full deterministic evidence per claim

Additional model providers (Anthropic, Gemini)

Caching for large repositories

TypeScript support

CI/CD integration

Table of Contents

Installation

Configuration

Quick Start

Usage Modes

Prompt Mode

Pre/Post Mode

Claims Mode

Command Reference

Understanding the Report

Claim Types (MVP)

Confidence System

JSON Output for CI

Troubleshooting

Installation

From source (current)

Clone the repository

git clone https://github.com/AstralXVoid/NoWreck.git cd NoWreck

Install system-wide with pipx (recommended)

pipx install .

Or with pip

pip install -e .

Or inside a virtual environment

python3 -m venv .venv source .venv/bin/activate pip install -e .

Verify installation

nowreck --version

→ nowreck 0.3.0

nowreck

→ shows banner + usage

Uninstall

pipx uninstall nowreck

or

pip uninstall nowreck

Configuration

NoWreck stores configuration in .nowreck/config.json under the current working directory.

Required settings for Prompt mode

Before using nowreck fix "", you need to configure an API key and model provider:

Set your API key (or use the NOWRECK_API_KEY env var instead)

nowreck config set api_key gsk_your_key_here

Set the API base URL (defaults to https://api.openai.com/v1)

nowreck config set base_url https://api.groq.com/openai/v1

Set the model (defaults to gpt-4o)

nowreck config set model llama-3.3-70b-versatile

Alternative: Environment variable

Set NOWRECK_API_KEY instead of storing the key in config:

export NOWRECK_API_KEY="gsk_your_key_here"

Optional settings

Temperature (0.0 = deterministic, default)

nowreck config set temperature 0.0

Max retries on parse failure (default: 1)

nowreck config set max_retries 2

View configuration

nowreck config show

→ api_key = gsk_...

→ base_url = https://api.groq.com/openai/v1

→ model = llama-3.3-70b-versatile

Compatible providers

Provider Base URL

OpenAI https://api.openai.com/v1 (default)

Groq https://api.groq.com/openai/v1

OpenRouter https://openrouter.ai/api/v1

DeepSeek https://api.deepseek.com/v1

Ollama (local) http://localhost:11434/v1

LM Studio (local) http://localhost:1234/v1

Any OpenAI-compatible Your custom endpoint

Note for Groq users: Groq currently blocks bare Python urllib requests with a Cloudflare 1010 error. NoWreck sends a browser-style User-Agent header to work around this, but if you encounter issues, try OpenRouter or a direct OpenAI API key instead.

Quick Start

Step 1 — Pick a test repo

Create a simple Python project with a before and after snapshot:

Set up a test repository

mkdir -p /tmp/myapp/pre /tmp/myapp/post

Pre: original code

cat > /tmp/myapp/pre/app.py /tmp/myapp/post/app.py str: return f"Hello, {name}!" EOF

Step 2 — Run detection (no claims)

nowreck fix --pre /tmp/myapp/pre --post /tmp/myapp/post

This will:

Scan both directories for .py files

Parse each into an AST

Build symbol indices

Detect structural changes

Show the unexplained changes (since no claims were provided)

You should see output like:

═══════════════════════════════════════════════════ Nowreck Verification Report ═══════════════════════════════════════════════════

Summary ──────────────────── ● 0 claims total ● 1 unexplained change

UNEXPLAINED CHANGES ──────────────────── ! ADD_FUNCTION greet (app.py)

Step 3 — Run with claims

nowreck fix \ --pre /tmp/myapp/pre \ --post /tmp/myapp/post \ --claims '{ "claims": [ { "type": "ADD_FUNCTION", "symbol_name": "greet", "file_path": "app.py", "confidence": 0.99, "explanation": "Added the greet function as requested." }, { "type": "CALLS_FUNCTION", "symbol_name": "greet", "file_path": "app.py", "caller_name": "greet", "called_name": "sanitize_

[truncated for AI cost control]