Skip to content
AI News HubLIVE
In-site rewrite5 min read

A structurally chunked, pre-embedded SQLite corpus of the EU AI Act

Summary

This dataset provides a single-file, pre-embedded SQLite corpus of the EU AI Act (Regulation (EU) 2024/1689), chunked by legal structure with BGE-M3 dense embeddings, metadata, risk tier labels, and more. It is designed for local query and RAG research, with verified completeness and transparent derivation rules.

SourceHacker News AIAuthor: olopadef
A structurally chunked, pre-embedded SQLite corpus of the EU AI Act
Report an error

The correction channel is not available yet. You can copy the article reference below for later.

Correction instructions
Read article

...\n options=options,\n )\n"," File \"/usr/local/lib/python3.14/site-packages/jwt/api_jwt.py\", line 368, in decode\n decoded = self.decode_complete(\n jwt,\n ......\n leeway=leeway,\n )\n"," File \"/usr/local/lib/python3.14/site-packages/jwt/api_jwt.py\", line 265, in decode_complete\n decoded = self._jws.decode_complete(\n jwt,\n ......\n detached_payload=detached_payload,\n )\n"," File \"/usr/local/lib/python3.14/site-packages/jwt/api_jws.py\", line 270, in decode_complete\n self._verify_signature(\n ~~~~~~~~~~~~~~~~~~~~~~^\n signing_input,\n ^^^^^^^^^^^^^^\n ......\n options=merged_options,\n ^^^^^^^^^^^^^^^^^^^^^^^\n )\n ^\n"," File \"/usr/local/lib/python3.14/site-packages/jwt/api_jws.py\", line 417, in _verify_signature\n raise InvalidSignatureError(\"Signature verification failed\")\n","jwt.exceptions.InvalidSignatureError: Signature verification failed\n"],"error_code":"JWTInvalidSignature"},"partial":false,"jwt":"eyJhbGciOiJFZERTQSIsImtpZCI6IjVHZDBvd0g5MTM2eDZjc1FvbE1zcktNWUZoRVFoUm5PVVVybEpjOUhaUEEifQ.eyJyZWFkIjp0cnVlLCJwZXJtaXNzaW9ucyI6eyJyZXBvLmNvbnRlbnQucmVhZCI6dHJ1ZX0sImlhdCI6MTc4NDI3NzU1NywianRpIjoiZDNiNzE5YWMtNWY3Yy00MmJhLWI3YTAtMmVlODI1YmViNzk5Iiwic3ViIjoiL2RhdGFzZXRzL2ZhaXRob2xvcGFkZS9haWFjdC1vcGVucmFnIiwiZXhwIjoxNzg0MjgxMTU3LCJpc3MiOiJodHRwczovL2h1Z2dpbmdmYWNlLmNvIn0.G9IB3kGgpHj-oFq89mMqMerAEemH2diBst966jRGgLFUWSz8Zqc-KtfXfREG9pZ_B9gZC5XHMblX6KNOQotJDQ"},"dataset":"faitholopade/aiact-openrag","isGated":false,"isPrivate":false,"hasParquetFormat":false,"isTracesDataset":false,"author":{"_id":"68c0102e4c4e9b23941f711b","avatarUrl":"/avatars/3c1177c9ff5b880e7b7cc87b38abda95.svg","fullname":"Faith Olopade","name":"faitholopade","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false},"compact":true,"isLoggedIn":false}">

Dataset Viewer

The dataset viewer is not available for this dataset.

The JWT signature verification failed. Check the signing key and the algorithm.

Error code: JWTInvalidSignature Exception: InvalidSignatureError Message: Signature verification failed Traceback: Traceback (most recent call last): File "/src/libs/libapi/src/libapi/jwt_token.py", line 286, in validate_jwt decoded = jwt.decode( jwt=token, ...... options=options, ) File "/usr/local/lib/python3.14/site-packages/jwt/api_jwt.py", line 368, in decode decoded = self.decode_complete( jwt, ...... leeway=leeway, ) File "/usr/local/lib/python3.14/site-packages/jwt/api_jwt.py", line 265, in decode_complete decoded = self._jws.decode_complete( jwt, ...... detached_payload=detached_payload, ) File "/usr/local/lib/python3.14/site-packages/jwt/api_jws.py", line 270, in decode_complete self._verify_signature( ~~~~~~~~~~~~~~~~~~~~~~^ signing_input, ^^^^^^^^^^^^^^ ...... options=merged_options, ^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "/usr/local/lib/python3.14/site-packages/jwt/api_jws.py", line 417, in _verify_signature raise InvalidSignatureError("Signature verification failed") jwt.exceptions.InvalidSignatureError: Signature verification failed

Need help to make the dataset viewer work? Make sure to review how to configure the dataset viewer, and open a discussion for direct support.

EmbeddingContent","children":[],"isValid":true,"title":"Schema — table EmbeddingContent"},{"id":"decoding-the-embeddings","label":"Decoding the embeddings","children":[],"isValid":true,"title":"Decoding the embeddings"},{"id":"getting-started--semantic-search-in-15-lines","label":"Getting started — semantic search in ~15 lines","children":[],"isValid":true,"title":"Getting started — semantic search in ~15 lines"},{"id":"evaluation","label":"Evaluation","children":[],"isValid":true,"title":"Evaluation"},{"id":"known-limitations","label":"Known limitations","children":[],"isValid":true,"title":"Known limitations"},{"id":"attribution","label":"Attribution","children":[],"isValid":true,"title":"Attribution"},{"id":"citation","label":"Citation","children":[],"isValid":true,"title":"Citation"}],"classNames":"top-6"}">

EU AI Act — structural RAG corpus

A single-file, pre-embedded SQLite corpus of the EU AI Act — Regulation (EU) 2024/1689 — chunked on legal structure only: one chunk per article paragraph, per recital, per annex point, per Article 3 definition. Chapter, section, and article metadata live in columns, not smeared into the text. The artifact is the corpus, not a pipeline — a downloadable database file you query locally, not a hosted service and not an authoritative legal interpretation.

Release

Release v1.0.0 (prepared 2026-07-16)

Primary artifact aiact_openrag.db (SQLite, embeddings included)

Schema version 2 (recorded in the meta table)

Chunks 933

Source text EUR-Lex consolidated version, CELEX 02024R1689-20240712

Source commit f94a363

Label audit commit 03e7aff (docs/label-audit.md)

Licensing is mixed — no single licence applies indiscriminately to every file. The EU legal text is reused under Commission Decision 2011/833/EU and is not relicensed by this dataset; the original corpus structure, metadata, embeddings and documentation are CC BY 4.0; adapted benchmark data is CC BY 4.0; software associated with the source repository is Apache-2.0. See LICENSE for the path-level division (LICENSES.md is the same text under the repository's canonical filename) and NOTICE for the attribution notice.

This is not legal advice. This corpus is a retrieval artifact for research and engineering. The chunking, tier labels, and metadata are operator-derived with documented rules — they are not authoritative interpretations. In particular, risk_tier is a labelling convention derived from the Act's structure (see docs/derivation.md), not a legal determination. Consult the Official Journal text and qualified counsel for compliance decisions.

Contents

933 chunks: 180 recitals, 522 article-paragraph chunks, 68 Article 3 definitions, 163 annex points

BGE-M3 dense embeddings (1024-dim float32, L2-normalized) for every chunk

risk_tier / obligation_on labels derived only where the text is unambiguous — every rule cites its provision in docs/derivation.md; NULL rather than guess

ELI deep links to the exact provision on EUR-Lex

Staged application dates per Article 113

Source: EUR-Lex consolidated version 02024R1689-20240712 (English). The English consolidated text is character-identical to the OJ text (32024R1689); article numbering is identical in both, and both CELEX ids are recorded. The build verifies, mechanically and fatally: every article 1–113 appears exactly once; every text node of the normative text is consumed by exactly one chunk and survives verbatim into the output (zero silent drops); no empty chunks; Annex III point counts match an independent extraction.

Schema — table EmbeddingContent

column type notes

chunk_id INTEGER PRIMARY KEY

celex_id TEXT 02024R1689-20240712

content_type TEXT recital | article | annex | definition

chapter TEXT e.g. CHAPTER III — HIGH-RISK AI SYSTEMS

section TEXT e.g. SECTION 2 — Requirements for high-risk AI systems

article_num TEXT 6, 6(2), 3(34), recital 44, Annex III.4(a)

heading TEXT e.g. Article 6 — Classification rules for high-risk AI systems

heading_level INTEGER 1 recital, 3 article/annex intro, 4 paragraph/point, 5 sub-point

chunk_text TEXT the provision text, nothing else

risk_tier TEXT direct textual classification only: prohibited (Art 5(1)) | high (Arts 6(1)/6(2) + Annex III, rebuttable per Art 6(3)) | NULL. limited/minimal are reserved, never assigned — operator-derived, see disclaimer

regime_bucket TEXT regime association: prohibited-practices | high-risk | transparency | gpai | voluntary-codes | NULL

obligation_on TEXT chunk-level operative subject: provider | deployer | importer | distributor | NULL

eli_url TEXT deep link to the exact provision

date_in_force TEXT EARLIEST date the provision applies, per Art 113 staging (Annex I: NULL) — an application date, not the Regulation's entry-into-force date (1 August 2024)

embedding BLOB 1024 × float32, raw bytes, L2-normalized (BGE-M3)

continued INTEGER 0 normally; 1,2,… for parts of a paragraph split at ~1000 tokens

A meta table records the source URL, both CELEX ids, embedding model, source-text licence notice, schema_version = 2, and a *_semantics note for each of the four derived columns (risk_tier_semantics, regime_bucket_semantics, obligation_on_semantics, date_in_force_semantics) stating exactly what each column does and does not claim.

The four derived columns answer four different questions:

risk_tier — does this chunk's own operative text classify? Narrow and deliberately sparse: only direct textual classifications ("shall be prohibited", "shall be considered to be high-risk").

regime_bucket — which regulatory regime does this provision belong to? Structural association (chapter/annex membership and the Commission's conventional tier names), independent of whether the text classifies.

obligation_on — which enum role is the chunk's operative obligated subject? Judged per chunk from its own sentences; NULL for authority-, Commission-, or AI-Office-bound and descriptive chunks.

date_in_force — earliest application date per Article 113; not the entry-into-force date, and NULL where no single date is unambiguous.

Value counts (933 chunks):

column population

risk_tier prohibited 2 · high 35 · NULL 896

regime_bucket prohibited-practices 10 · high-risk 332 · transparency 7 · gpai 51 · voluntary-codes 4 · NULL 529

obligation_on provider 56 · deployer 17 · importer 7 · distributor 6 · NULL 847

Decoding the embeddings

import sqlite3, numpy as np

con = sqlite3.connect("aiact_openrag.db") con.row_factory = sqlite3.Row row = con.execute("SELECT * FROM EmbeddingContent WHERE article_num = '5(1)'").fetchone() vec = np.frombuffer(row["embedding"], dtype=np.float32) # shape (1024,)

Getting started — semantic search in ~15 lines

Encoding new queries needs the embedding model (pip install sentence-transformers); everything else is the standard library plus NumPy.

import sqlite3, numpy as np from sentence_transformers import SentenceTransformer

con = sqlite3.connect("aiact_openrag.db") rows = con.execute("SELECT chunk_id, article_num, heading, chunk_text, embedding " "FROM EmbeddingContent WHERE embedding IS NOT NULL").fetchall() mat = np.vstack([np.frombuffer(r[4], dtype=np.float32) for r in rows])

model = SentenceTransformer("BAAI/bge-m3") q = model.encode(["is emotion recognition at work prohibited?"], normalize_embeddings=True)[0] for i in np.argsort(-(mat @ q))[:5]: r = rows[i] print(f"{r[1]:<16} {r[2]}\n {r[3][:150]}…\n")

Filter by metadata instead of (or as well as) similarity — that is the point of structural chunking:

-- what the text itself classifies (direct, defensible per provision) SELECT article_num, chunk_text FROM EmbeddingContent WHERE risk_tier = 'prohibited';

-- everything belonging to a regime (the broader, structural grouping) SELECT article_num, heading FROM EmbeddingContent WHERE regime_bucket = 'high-risk' AND content_type = 'article';

Evaluation

The corpus was evaluated against the AI Act Evaluation Benchmark (Davvetas et al.) on retrieval, QA and risk-classification tasks; full numbers, methodology and provenance are in eval/RESULTS.md.

One finding deserves a plain-language caveat here. A three-family model panel (mistral-nemo, qwen2.5:14b, claude-fable-5) rating all 339 benchmark scenarios showed inter-model agreement collapsing exactly on the two tiers where classification F1 is lowest: per-tier Fleiss κ was 0.238 (limited) and 0.449 (minimal) versus 0.784 (prohibited) and 0.738 (high-risk). This is consistent with ambiguity or noise at the limited/minimal label boundary, and it means limited/minimal per-tier scores on this benchmark should not be read as pure retrieval-quality signals — but it is an exploratory finding with substantial caveats, not proof: the raters are themselves LLMs, so shared model difficulty on a genuinely hard boundary cannot be separated from label n

[truncated for AI cost control]

Key points and analysis

Article intelligence

EngineersAdvanced

Key points

  • 933 chunks: 180 recitals, 522 article paragraphs, 68 Article 3 definitions, 163 annex points
  • BGE-M3 dense embeddings (1024-dim, L2-normalized) for semantic search
  • Risk tier and obligation labels assigned only where text is unambiguous
  • Sourced from EUR-Lex consolidated version, mechanically verified for completeness

Highlights and analysis are generated automatically and may contain errors. Check the original source.