The AI Disagreement Index: 8 models agreed on the "best tool" 0 of 16 times
An open, rigorous, living measurement of how much AI engines disagree on which B2B tools to trust per category. In the recorded sample, across 16 categories, all eight models named the same single best tool zero times, with a mean pairwise agreement of 44% and Fleiss' kappa of 0.41. The index is updated monthly and provides raw data for reproducibility.
Featured dataset
A living measurement, updated monthly
The AI Disagreement Index
The first open, rigorous, living measurement of how much the AI engines disagree on which tools to trust, per category, over time, with the receipts. Not a ranking. A record of where the machines cannot agree.
By Vincent Wesley Couey ORCID 0009-0005-6869-308X DOI: 10.5281/zenodo.20767877 CC-BY-4.0
The recorded finding, B2B software sample
0 times
Across 16 B2B software categories, all eight AI models named the same single best tool zero times in our recorded sample.
Source: B2B / GTM index, captured Jun 19 and Jul 8, 2026
44%
Mean pairwise agreement between engines on which tool tops a category. The rest of the time, they diverge.
Recorded across 8 models, 16 categories
0.41 κ
Fleiss' kappa across all engines and categories: only moderate agreement, measured, not asserted.
163 tools were named by just one model
01 What it measures, and why it is different
Everyone else ranks brands. We measure where the engines fall apart.
The incumbents, Profound and Evertune and the rest, tell a vendor its generic "share of voice" across AI answers: one blended score, one direction, brand-level. Useful for a marketing team. It is still a ranking.
This measures something none of them publish: cross-engine disagreement on tools. For a given category we ask several AI engines the same buying question and record who each one names. The headline is not the winner. It is the spread: how far apart the engines land, which tools only one model has ever heard of, and how that picture changes when a model stops reciting training memory and starts searching the live web.
Three commitments make it a research object rather than a listicle. It is open: the raw recorded answers are retained as JSONL and the standings are reproducible on challenge. It is rigorous: consensus is a computed statistic, Fleiss' kappa and pairwise agreement, not a vibe. And it is temporal: we re-capture monthly, so you can watch the AI change its mind.
Rank-first tools
One blended visibility score
Brand-level, generic
Snapshot, sold to the vendor
Closed methodology
The Disagreement Index
Cross-engine consensus and spread
Tool-level, per category
Living, re-captured monthly
Open JSONL, DOI, reproducible
02 The divergence map
Where each engine sends you, side by side.
Part of the featured AI Disagreement Index. The interactive layer plots who each AI engine trusts per category and how those picks drift month over month. It is fed directly by the recorded dataset.
Most-named tool in the row Tools that broke away No tool named
The AI Disagreement Index divergence matrix. Rows are 16 B2B software categories ordered from most cross-engine agreement at the top to most fractured at the bottom. Columns are 8 AI engines. Each cell is the single top tool that engine named for that category, colored by tool within its row.
Hover, tap, or focus a cell to read who each engine sends you.
03 The datasets
The full collection. Each one citable, sourced, and reproducible.
Every dataset below is published as a citable research object with its own DOI under ORCID 0009-0005-6869-308X. Open one to read the data and the receipts. The three that carry a full web writeup link to it as well.
AI & Business Data 11 datasets
Featured
The AI Disagreement Index
DOI 10.5281/zenodo.20767877
Eight-engine audit of which B2B/GTM software AI assistants recommend across buying categories; the cross-engine disagreement measurement.
View dataset → Read the page →
New
The AI Creative Rights Contradiction Index
DOI 10.5281/zenodo.21302650
Pre-registered measurement of where AI creative tools' grants and platform rules contradict each other: 83% of good-faith creator paths carry a rights contradiction, and it is tool-intrinsic, not platform-driven. The rights-side twin of the Disagreement Index.
View dataset → Read the page →
The AI Citation Autopsy
DOI 10.5281/zenodo.20632767
Cross-engine citation analysis of which GTM tools AI assistants name and which sources they cite (464 classified citations).
View dataset → Read the page →
The Nesyona Effective-Price Index 2026
DOI 10.5281/zenodo.20675137
The first public computation of AI consumer effective price per task, accounting for tokenizer inflation and free-tier changes.
View dataset →
The LLMOps Stack 2026
DOI 10.5281/zenodo.20738670
Structured comparison of 19 production LLM-operations tools across gateway/routing, observability, evaluation, and guardrails layers.
View dataset →
Vector Databases for RAG 2026
DOI 10.5281/zenodo.20738949
Structured comparison of 11 vector databases for retrieval-augmented generation across deployment shape, license, and native hybrid search.
View dataset →
EduBracket Certification ROI 2026
DOI 10.5281/zenodo.20768041
Sourced ROI dataset for 16 IT and data certifications: program cost and duration paired with target-role U.S. BLS wage data.
View dataset →
AI Creative Tool Commercial Rights & Pricing 2026
DOI 10.5281/zenodo.20638385
Nine-tool dataset of the commercial-use terms behind major generative AI creative tools across image, music, video, and voice.
View dataset →
AI Content Platform Policy Matrix 2026
DOI 10.5281/zenodo.20638383
Nine-platform matrix of how major creator and marketplace platforms treat AI-generated content (allowed, disclosure, monetization).
View dataset →
AI Creative Policy Change Log 2026
DOI 10.5281/zenodo.20638387
Dated, sourced change log of how platforms and US law shifted their treatment of AI-generated creative content from January 2025 to April 2026.
View dataset →
U.S. 1099-K Reporting Thresholds 2026
DOI 10.5281/zenodo.20632599
Fifty-state 1099-K reporting-threshold dataset for tax year 2026, compiled against the restored federal floor.
View dataset →
Physics · Substrate Geometry 5 datasets
Substrate Geometry Research Program
DOI 10.5281/zenodo.20674508 · software
Computational framework for evaluating geometric primitives as engineering substrates, classified by operational invariants rather than symmetry groups.
View dataset →
Mono-monostatic Body Catalog
DOI 10.5281/zenodo.20674393
Geometry and analysis behind the mono-monostatic catalog paper (arXiv:2604.17120): generated body meshes plus landscape and basin-geometry data.
View dataset →
Gömböc Oracle Evaluation
DOI 10.5281/zenodo.20674390
Geometry and computational results behind the Gömböc paper (arXiv:2604.17095): multi-resolution meshes plus oracle-evaluation data.
View dataset →
TPMS Electrode Thermal Evaluation
DOI 10.5281/zenodo.20674388
Computational results behind the TPMS thermal-electrode paper: single-arc methodology and FEniCS thermal validation.
View dataset →
Oracle-Based Computational Metrics
DOI 10.5281/zenodo.20673963 · working paper
Methodological characterization of the trust boundaries of discrete mesh operators in oracle-based geometric evaluation.
View dataset →
04 Methodology
Research-grade, so it holds up when someone checks the math.
One rigor stance governs every dataset in this collection, the AI and business measurements and the physics releases alike: claims are framed, figures are sourced, results are reproducible from the retained data, and where the data does not reach we stay silent. The mechanics below describe the flagship AI capture; the same honesty floor applies across the board.
Cross-engine capture
The same buying question is put to several AI engines per category. Every answer is recorded verbatim, then rolled up into who each engine names. Cross-engine means at least three of the engines actually answered, never a single-engine artifact.
Dated snapshots
Each capture carries the month it was taken. Standings are point-in-time. Monthly re-capture builds the temporal record, so drift, a model quietly changing its recommendation, is itself a measured signal.
Reproducible data
The raw answers are retained as JSONL and the roll-up is deterministic. Any published standing can be reproduced from the recorded file. The consensus statistics, pairwise agreement and Fleiss' kappa, are computed from that same data.
Minimum sample floor
A category is only published when the recorded sample is deep enough to be defensible, and a tool only earns a standing when named by at least three of the engines queried. Below the floor, we stay silent rather than fabricate.
The honesty floor, load-bearing
Frame, never claim. A standing reads "named by N of the engines in our recorded sample," a checkable frame, never "the #1 tool," an unfalsifiable claim.
True recorded standing only. Every figure is generated from the recorded data, so it cannot overstate. Where a cohort has no data, no standing is minted.
Minimum-engine threshold. A tool must be named by at least three engines to appear. Thin niches are skipped, not padded.
Paid never edits earned. A sponsored featured spot is labelled and structurally separate. Buying visibility never changes a recorded standing.
Full methodology and the underlying dataset are packaged as a citable research object: DOI 10.5281/zenodo.20767877, published under ORCID 0009-0005-6869-308X with byline Vincent Wesley Couey, licensed CC-BY-4.0.
05 The Verified badge · For the AI Disagreement Index
If the engines recommend you, prove it, honestly.
Scoped to the featured AI Disagreement Index. The badge is a feature of the AI recommendation dataset, where a vendor can earn a recorded standing, not of the physics releases.
Any vendor that clears the floor for a category earns a free Verified by Lattice badge stating its true recorded standing. The badge text is generated from the recorded data, so it can state exactly what the engines said and nothing more.
Vendors embed it on their own site. The badge links back here, to the category it was earned in, which is how the standing stays checkable: click through and see the receipts. The earned badge is a gift, not a purchase.
Want a labelled featured spot alongside the index instead? You can claim or defend your featured spot. A featured spot is always marked as sponsored and is structurally separate from the earned ranking. Earned is not paid, and paid never edits earned.
Illustrative badge
L
Verified by Lattice Named by [N] of [M] AI engines [category] · recorded [month] · data.deepsynthesis.org
Placeholder copy. Live badges carry the exact recorded count for the vendor and category, generated from the retained JSONL.