Resilience testing for AI models against radiation-induced bit flips

Notifications You must be signed in to change notification settings

Fork 0

Star 0

BranchesTags

Open more actions menu

Folders and files

NameName

Last commit message

Last commit date

Latest commit

History

4 Commits

4 Commits

.ananth-skills

.ananth-skills

.cursor

.cursor

benchmarks

benchmarks

docs

docs

resilience_engine

resilience_engine

tests

tests

.gitignore

.gitignore

LICENSE

LICENSE

README.md

README.md

pyproject.toml

pyproject.toml

requirements-lock.txt

requirements-lock.txt

resilience_test.py

resilience_test.py

Repository files navigation

(codename: OrbitTest)

A command-line tool that tests whether an ONNX model's predictions survive the kind of memory corruption a satellite's onboard computer actually experiences in orbit — radiation-induced single event upsets (SEUs), a.k.a. bit flips — rather than just checking that the model runs.

Given a model and a validation set, it injects statistically-realistic bit flips into the model's weights, re-runs inference many times, and reports a 0-100 resilience score plus a per-layer sensitivity breakdown showing which weight tensors matter most for surviving corruption.

This was built as a 3-day CLI validation sprint to test one thing before investing in a full SaaS product: does this fault-injection methodology actually work and generalize, and is the finding credible enough to show a skeptical aerospace engineer? This README is deliberately honest about what that sprint did and didn't prove — see Current Limitations before drawing conclusions from a report this tool produces.

Quickstart

git clone https://github.com/ananfauh7/MRVPlatform.git cd MRVPlatform python -m venv .venv && source .venv/bin/activate pip install -e .

python resilience_test.py \ --model benchmarks/mnist/model.onnx \ --data benchmarks/mnist/dataset.npz \ --preset leo-typical \ --mission-days 365 \ --trials 20

This runs the tool against a real pretrained CNN and a real MNIST validation subset that ship with the repo — no external downloads needed to try it out. Since everything is seeded deterministically, running the exact command above reproduces this exact output:

Using preset 'leo-typical': fault_rate=2e-07 errors/bit-day Loading model from benchmarks/mnist/model.onnx... Loading validation dataset from benchmarks/mnist/dataset.npz... Running clean baseline on 50 example(s)... Running 20 Monte Carlo trial(s) at fault_rate=2e-07, mission_days=365.0... Running per-layer sensitivity analysis (8 trial(s) per layer)...

============================================================ RESILIENCE REPORT ============================================================ Clean accuracy: 1.0000 Faulted accuracy (mean): 0.8190 Faulted accuracy (std): 0.3622 Faulted accuracy (range): [0.0600, 1.0000] Resilience score: 81.9 / 100 Trials run: 20

Top sensitive layers: #1 Parameter87 mean accuracy drop: 0.1125 #2 Parameter193 mean accuracy drop: 0.0000 #3 Parameter5 mean accuracy drop: 0.0000 #4 Parameter6 mean accuracy drop: 0.0000 #5 Parameter88 mean accuracy drop: 0.0000

Full results written to: results.json Chart written to: accuracy_vs_fault_rate.png ============================================================

What it does

Loads your ONNX model and validation dataset, and establishes a clean-run accuracy baseline.

Computes how many bit flips to expect in each weight tensor, given a fault rate (errors per bit-day) and a mission exposure duration (days): expected_flips = fault_rate * tensor_size_bits * mission_days.

Injects those bit flips directly into the IEEE-754 float32 bit representation of the model's weights — sign, exponent, and mantissa bits alike, chosen uniformly at random (whether exponent flips are disproportionately damaging is a finding the tool surfaces, not an assumption it bakes in).

Re-runs inference against the validation set with the corrupted weights and measures the resulting accuracy.

Repeats this as a Monte Carlo simulation (default: 20 trials) to characterize the distribution of outcomes, not just one corrupted run — a single trial's accuracy can vary enormously depending on which bits happen to get hit.

Computes a resilience score: 100 * (mean faulted accuracy / clean accuracy), clipped to [0, 100].

Runs a per-layer sensitivity analysis: repeats the trial loop once per weight tensor, corrupting only that tensor each time, and ranks tensors by how much they degrade accuracy when hit — this is the tool's most useful non-obvious output: it tells you which layers would benefit most from redundancy/hardening, not just an aggregate score.

Writes a JSON report and a chart, plus a console summary.

How it works (methodology)

Fault model

Bit flips are simulated at the level of each weight tensor's raw IEEE-754 float32 bit pattern:

bit 31 : sign bits 30-23 : exponent (8 bits, biased by 127) bits 22-0 : mantissa (23 bits)

For a tensor of N bits, at fault rate r (errors/bit-day) over a mission of D days, the expected number of flips is r * N * D, rounded to the nearest integer. Exact bit positions are then chosen uniformly at random (without replacement) and flipped via XOR on the tensor's uint32 bit view. Everything is fully deterministic given a --seed — the same seed, model, dataset, and parameters always reproduce the same result.

Monte Carlo aggregation

A single corrupted run tells you almost nothing on its own (an unlucky mantissa-bit flip is invisible; an unlucky exponent-bit flip on a critical weight can be catastrophic). The tool runs --trials independent corruption-and-evaluate trials (each with its own derived sub-seed) and reports the mean, standard deviation, min, and max of the resulting accuracy and loss deltas relative to the clean baseline.

Per-layer sensitivity

Separately from the headline whole-model score, the tool repeats the trial loop once per weight tensor — corrupting only that tensor each time, leaving every other tensor clean — and ranks tensors by mean accuracy drop. This answers "if I could only harden/replicate one layer, which one matters most?", which is more actionable than an aggregate score alone.

Resilience score

score = 100 * (mean_faulted_accuracy / clean_accuracy) # clipped to [0, 100]

Deliberately simple for this sprint. It does not currently factor in loss degradation, prediction-confidence calibration, or weight how "close" a near-miss is versus total failure — see Current Limitations.

Fault-rate presets

Rather than requiring you to guess a --fault-rate value, --preset accepts a name grounded in published Single Event Upset (SEU) rate research:

Preset Fault rate (errors/bit-day) Basis

leo-typical 2×10⁻⁷ Measured: derived from an in-flight SEU rate of 0.85 errors/day for a 4-Mbit SRAM aboard the SAC-C satellite (707 km, 98.2° polar LEO orbit). Falguère et al., IEEE Trans. Nucl. Sci., vol. 49, no. 6, pp. 2782–2787, Dec. 2002 (doi:10.1109/tns.2002.805380).

saa-transit 1×10⁻⁵ Estimate: two orders of magnitude above leo-typical, reflecting the well-documented finding that South Atlantic Anomaly trapped-proton flux dominates total LEO SEU counts (Mullen et al., IEEE Trans. Nucl. Sci., vol. 42, no. 6, pp. 1988–1994, 1995). No single directly-measured per-bit-day SAA-only figure was found within this project's research timebox.

solar-storm-worst-case 1×10⁻³ Estimate: matches the industry-standard "October 1989" CREME96 worst-day benchmark used in radiation hardness assurance (referenced in a 2023 RADECS SER-prediction paper, doi:10.1109/radecs59069.2023.10767005).

Every preset's exact citation and derivation reasoning is documented alongside its value in resilience_engine/presets.py — read it before treating a preset as a validated figure for a real mission decision. Presets marked "estimate" are order-of-magnitude figures, not device-specific measurements.

Usage

resilience-test --model MODEL.onnx --data DATA (--fault-rate RATE | --preset NAME) [--mission-days DAYS] [--trials N] [--layer-trials N] [--seed SEED] [--output-dir DIR] [--debug]

Flag Required Default Description

--model Yes — Path to the ONNX model file

--data Yes — Path to a .npz file (with inputs/labels arrays) or a per-class directory of .npy files

--fault-rate One of these two — Fault rate in errors/bit-day (e.g. 1e-6)

--preset required — leo-typical, saa-transit, or solar-storm-worst-case

--mission-days No 365 Mission exposure duration in days

--trials No 20 Monte Carlo trials for the headline resilience score

--layer-trials No 8 Trials per layer for the sensitivity breakdown

--seed No 42 Base random seed (for reproducibility)

--output-dir No . Where to write results.json and accuracy_vs_fault_rate.png

--debug No off Show full Python tracebacks instead of a clean error message

Either --fault-rate or --preset is required (mutually exclusive).

Examples

Using a named preset (recommended starting point):

python resilience_test.py --model my_model.onnx --data ./val_data --preset leo-typical

Using an explicit fault rate and a shorter mission:

python resilience_test.py --model my_model.onnx --data ./val_data \ --fault-rate 1e-6 --mission-days 90 --trials 50

Using a per-class directory dataset (one subdirectory per integer class label, each containing .npy files):

val_data/ 0/example_001.npy 0/example_002.npy 1/example_001.npy ...

python resilience_test.py --model my_model.onnx --data ./val_data --preset saa-transit --output-dir ./results

Output

Console summary: clean/faulted accuracy, resilience score, and top-5 most sensitive layers.

results.json: the full Monte Carlo result (every trial's outcome, distribution statistics, resilience score, model/dataset content hashes for auditability) plus the complete per-layer sensitivity ranking.

accuracy_vs_fault_rate.png: a chart of clean vs. faulted accuracy (mean ± std error bars) at the tested fault rate.

Current Limitations

Being explicit about these matters more to us than looking impressive — an aerospace engineer will find these gaps immediately if we don't disclose them first.

Weight corruption only. This tool corrupts model weights, not activations, gradients, control flow, or the surrounding flight software/OS. Real onboard SEUs can hit any of those; weight corruption is one well-motivated slice of the real fault surface, not the whole thing.

No thermal or power-throttling modeling. The MVP spec's broader vision (radiation + power throttling + memory corruption) is only partially implemented here — this sprint validated the radiation/bit-flip piece specifically.

Accuracy-only resilience score. The score doesn't currently factor in loss degradation, prediction-confidence calibration, or distinguish a near-miss from a catastrophic failure beyond their effect on top-1 accuracy.

Single operating point per run. Each run reflects one fault-rate/mission-days combination, not a full sensitivity curve across exposure levels (though nothing stops you from scripting multiple runs).

Validated against two model architectures, not proven to generalize universally. The core pipeline was built and unit-tested against a small synthetic linear model, then validated end-to-end against a real pretrained CNN (see benchmarks/mnist/) to check the approach generalizes beyond the synthetic case. That's meaningful evidence, not proof — it hasn't been tested against, for example, transformer architectures, models with custom/exotic ops, or anything beyond ONNX opset 8-era graphs.

Two of three fault-rate presets are estimates, not direct measurements. leo-typical is derived from a specific published in-flight measurement; saa-transit and solar-storm-worst-case are order-of-magnitude estimates extrapolated from qualitative literature findings (see resilience_engine/presets.py for exactly how each was derived). Don't treat them as a substitute for mission-specific radiation analysis.

CPU-only, single-image-at-a-time inference. No GPU o

[truncated for AI cost control]