Epistemic Engine – verify AI-generated code and forecast what will break

Notifications You must be signed in to change notification settings

Fork 0

Star 0

BranchesTags

Open more actions menu

Folders and files

NameName

Last commit message

Last commit date

Latest commit

History

6 Commits

6 Commits

.github/workflows

.github/workflows

src/epistemic_engine

src/epistemic_engine

tests

tests

.gitattributes

.gitattributes

.gitignore

.gitignore

CHANGELOG.md

CHANGELOG.md

LICENSE

LICENSE

NOTICE

NOTICE

PUBLISHING.md

PUBLISHING.md

README.md

README.md

pyproject.toml

pyproject.toml

Repository files navigation

Computational epistemology for software. It reads a codebase as a system of justified beliefs — claims the code makes about itself ("this function authenticates the caller", "this value is never null", "this uses approved crypto") — reconstructs how well-justified each one is from git history, and tracks how that confidence changes over time.

On top of that foundation it ships two tools you can use today:

ee guard — a pre-commit / CI verifier that blocks unsafe or unjustified code before it lands (great for reviewing AI-generated changes).

ee predict-chain — forecasts which beliefs are about to collapse, why, roughly when, and what it would cost to fix now vs. later — and it tells you when it can't trust its own forecast.

Everything is deterministic (no LLM, no network, no wall-clock in any computed value), runs offline, and is backed by 167 tests.

pip install epistemic-engine

Requires Python 3.10+. Apache-2.0 licensed.

What it does, in one screen

ee ingest ./my-repo # read git history into a local graph ee analyze ./my-repo # extract beliefs, justifications, trajectories

ee guard ./my-repo --all # find unsafe / unjustified code ee predict-chain ./my-repo # forecast which beliefs will collapse ee dashboard ./my-repo # explore all of it in your browser

1. Verify code before it ships — ee guard

ee guard extracts the beliefs a change makes about itself and blocks on risky ones, then layers on direct OWASP-class scanners and known-CVE matches. Each finding carries a severity, the reason, a concrete fix, and a stable fingerprint.

It catches, among others:

Class Examples

Weak crypto md5/sha1/mt_rand used as a security primitive

Injection SQL built by string concatenation, os.system/popen on user input

Unsafe sinks eval, innerHTML, unserialize, pickle.loads on untrusted data

Secrets hardcoded API keys / tokens (the value is never printed or stored)

Misconfig TLS verification off, Access-Control-Allow-Origin: *, debug enabled

ee guard ./repo --staged # verify staged changes (exit 1 = would block the commit) ee guard ./repo --all # verify the whole tree ee guard ./repo --format sarif # SARIF 2.1.0 for GitHub code scanning ee guard ./repo --emit-workflow # print a ready .github/workflows/ee-guard.yml ee hook install ./repo # run it automatically on every git commit

Adoption features so it fits a real team: inline # ee-ignore[rule] suppression, a disabled_rules config, and a baseline mode (--update-baseline / --baseline FILE) that fails only on new findings so you can adopt on an existing repo without fixing everything first.

Field-tested on 1,500+ real production files: precision / recall / false-positive rate of 1.0 / 1.0 / 0.0 (95% CI lower bound 0.92 on a 114-sample corpus).

1. Forecast what's about to break — ee predict-chain

The engine already knows how each belief's confidence is eroding. predict-chain projects that forward and, for each at-risk belief, gives you:

a causal chain — the ranked factors driving the predicted collapse, each tagged measured (from your repo's history) or assumption (your cost model);

a collapse probability within a horizon, from a seeded Monte Carlo;

a calendar ETA from your repo's own commit cadence;

a fix + ROI, computed under your cost assumptions and labelled as such.

ee predict-chain ./repo --horizon 30 # forecasts with causes, ETA, ROI ee calibrate ./repo # is the forecast trustworthy on THIS repo?

It refuses to lie. ee calibrate runs a leakage-free back-test against your repo's own history and reports a Brier skill score vs. a base-rate baseline. If the model doesn't beat guessing on your repo, the report says so — and predict-chain prints that verdict above every forecast. It never fabricates probabilities for undisclosed future CVEs, and never presents a dollar figure as fact.

1. See everything — ee dashboard

ee dashboard ./repo # loopback-only web UI; Ctrl-C to stop

A local, offline, self-contained dashboard with three tabs:

Beliefs — every claim the code makes, by domain, coloured by confidence.

🔮 Predictions — the collapse forecasts, with the calibration verdict on top.

🛡 Guard — the ee guard findings, grouped by severity, with a BLOCK/PASS banner. Scans the whole repo by default and caches the result.

Why you can trust the output

Deterministic. Identical inputs produce byte-identical output. No LLM, no sampling, no wall-clock in any computed value — so results are reproducible and diffable in CI.

Honest about uncertainty. Gates are evaluated on the lower bound of a Wilson confidence interval, not a point estimate. Predictions ship with a calibration verdict. Cost/ROI figures are labelled assumptions.

Offline-first. No network access except an explicit ee sync.

Bounded formalism. Beliefs outside the closed PO-1 predicate ontology are recorded as unformalised and excluded from reasoning — never silently coerced.

Install

pip install epistemic-engine # core (zero heavy dependencies) pip install "epistemic-engine[parsing]" # + tree-sitter for entity-level beliefs

Without the parsing extra the engine degrades gracefully to file-granularity analysis. Python 3.10+.

Command reference

Command What it does

ee ingest Read git history into the local epistemic graph

ee analyze Extract beliefs, justifications, change events, trajectories

ee guard Verify changed/all code; block unsafe or unjustified beliefs

ee hook install Install a git pre-commit hook running ee guard --staged

ee predict-chain Forecast belief collapse with causes, ETA, and ROI

ee calibrate Back-test the forecast model against the repo's own history

ee dashboard Serve the local web dashboard

ee report Epistemic health report (text / json / markdown / html)

ee beliefs / falsifiers / debt / timeline Inspect specific slices

ee doctor Validate environment, ontology, and configuration

Run ee --help for options, or ee man for a full man page.

Development

git clone && cd epistemic-engine pip install -e ".[parsing,dev]" pytest # 167 tests

Documented deviations from the design spec

Both are portability choices for a single-developer, offline-first, cross-platform (incl. Windows) build; neither changes observable semantics.

Graph store is abstracted behind storage.GraphStore with a SQLite default backend (zero-install, deterministic); RocksDB is optional.

Git access goes through the git CLI via subprocess, abstracted behind ingestion.GitExtractor.

License

Apache-2.0 © 2026 Aswin S. See LICENSE and NOTICE.

Resources

Readme

Apache-2.0 license

Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks

Report repository