Democr.ai: Self-hosted Agentic AI Runtime with Audit and RBAC

Notifications You must be signed in to change notification settings

Fork 0

Star 1

BranchesTags

Open more actions menu

Folders and files

NameName

Last commit message

Last commit date

Latest commit

History

65 Commits

65 Commits

clients

clients

democrai

democrai

docker

docker

docs_site

docs_site

engines

engines

examples

examples

extractors

extractors

modules

modules

scripts

scripts

tests

tests

.gitignore

.gitignore

CONTRIBUTING.md

CONTRIBUTING.md

LICENSE

LICENSE

README.md

README.md

init.py

init.py

constraints.txt

constraints.txt

main.py

main.py

pyproject.toml

pyproject.toml

requirements.txt

requirements.txt

run.bat

run.bat

run.py

run.py

run.sh

run.sh

setup_venv.bat

setup_venv.bat

setup_venv.py

setup_venv.py

setup_venv.sh

setup_venv.sh

Repository files navigation

Website: democr.ai

Documentation: democr.ai/docs/

Overview

Democr.ai is a complete runtime framework for AI applications. It provides:

Server-driven UI for agentic contexts, implementing and extending Google's A2UI protocol

Multi-client rendering of the same UI definition across web and desktop clients

Native multi-tenancy enforced at write time

Cross-platform OS-level sandboxing for engine and extension processes

Triple-layer audit through SQLAlchemy hooks, with sensitive field redaction

RBAC with declarative module manifests

Pluggable AI engine orchestration across local, remote, and distributed runtimes

Engine quotas for request and token governance by engine and subject scope

Knowledge subsystem with vector and graph backends

A strict modular architecture where everything — including authentication — is a module built against the public SDK

The framework is designed around a clear boundary between core runtime and extensions. The SDK is the only supported way to build on top.

What makes it different

Most AI application stacks combine a chat UI, an LLM client, a workflow library, and separate infrastructure for audit, permissions, sandboxing, and observability. Democr.ai puts those concerns in one runtime contract:

UI is declared server-side and rendered by independent clients.

Modules, engines, and extractors are installable extensions with a public SDK boundary.

Model calls flow through an orchestrator that supports local providers, remote providers, process isolation, quotas, and multi-node execution.

Security and audit are runtime primitives, not application conventions.

Knowledge ingestion, retrieval, media handling, tools, MCP, and agent skills share the same request and observability context.

That combination is intentionally broad. The project is still beta, but the architecture is already aimed at production-grade constraints rather than demos or single-chatbot prototypes.

A2UI

Let agents build UI components that render as native application surfaces, not as screenshots or markup fragments. Democr.ai modules describe UI declaratively on the server, then stream structured updates as the agent works.

The short demo shows the important part: one backend-declared YAML surface reflected through client renderers, while the backend keeps ownership of behavior, actions, and runtime effects.

Watch the A2UI demo video

Who it's for

Democr.ai targets applications where:

The deployment environment has compliance, regulatory, or operational constraints

Observability and reproducibility are required from day one, not added later

Avoiding vendor lock-in is a hard constraint: the framework is fully self-hostable, model-agnostic, and ships no proprietary cloud services

Auditability across user actions, data mutations, and AI calls is required at the deployment level

The team prefers an integrated runtime over assembling separate pieces (LLM library + UI framework + observability stack + audit + RBAC + workflow engine)

It is not a thin wrapper, a model router, or a chat UI library. It is a runtime where modules, engines, extractors, and clients compose into a single coherent system.

Core principles

Everything is a module

There are no privileged components. Authentication, system administration, and the demo applications shipped in this repository are all built as modules, using the same SDK that third-party developers use. There are no reserved APIs, no hidden hooks, and no internal-only services.

The modules included in this repository (auth, system, components) are primarily examples of how the framework works and how applications integrate with it. They provide a working baseline, but deployments are not required to use them as-is: teams can replace them, extend them, or build their own domain-specific implementations.

No vendor lock-in

The runtime is fully self-hostable.

AI engines are pluggable: local inference (vLLM, llama.cpp) and cloud providers (OpenAI, Anthropic, Google, and others), or custom integrations.

Storage backends are pluggable for vector (sqlite-vec, pgvector, and others), knowledge graph, and core persistence (SQLite, PostgreSQL).

No proprietary serialization formats. State is portable across deployments.

Observability and reproducibility as development guides

Observability is not added as later instrumentation — it is part of how the framework expects developers to build. Every action, AI call, and module operation flows through a correlation chain that allows step-by-step reconstruction of any request. Pipelines are observable. The audit subsystem captures mutations at three layers (request, ORM, AI invocation) automatically, with sensitive field redaction.

Reproducibility is achieved through deterministic configuration, structured event logs with stable IDs, and an immutable audit trail.

Security as a primitive, not a feature

The framework provides security as foundational primitives, not as opt-in features. See Threat model for the precise boundary between framework guarantees and deployer responsibilities.

Extending Google's A2UI protocol

The UI layer is an extended implementation of Google's A2UI protocol for server-driven UI in agentic contexts. The same UI definition renders consistently across multiple clients with no client-side framework code required from module authors. New clients can be added without changes to existing modules.

Architecture overview

Democr.ai is composed of distinct subsystems:

Core runtime — bootstrap, lifecycle, request handling, dependency graph

Network — WebSocket, IPC, HTTP under a unified protocol layer

AI engine orchestrator — job-based scheduling, batching, gRPC process isolation, HITL support

Knowledge — ingestion queue, projection workers, vector + graph retrieval

Sandbox — OS-level isolation (Landlock + seccomp + iptables + helper process | seatbelt | Windows Low Integrity + WFP)

Multi-tenancy — write-time enforcement with materialized scope filters

RBAC — declarative access policies in module manifests

Modules — framework extension surface, via SDK

Engines — pluggable AI inference providers

Extractors — pluggable knowledge extraction providers

Clients — independent rendering surfaces for web, desktop, and mobile-oriented clients

Platform — agents, skills, tools, MCP, UI builder

Detailed subsystem documentation is available at democr.ai/docs/.

Feature matrix

Capability Status Notes

Server-driven UI (extended A2UI) Implemented websocket and ipc transport

Multi-client rendering Implemented Web, Qt desktop, Tauri, React variants

Multi-tenancy Implemented Write-time enforcement

RBAC Implemented Declarative manifests

AI engine orchestration Implemented Job-based, gRPC isolated

Knowledge subsystem (vector + KG) Implemented Hybrid retrieval

Audit (triple layer) Implemented Sensitive field redaction

OS sandbox (Landlock + seccomp + iptables) Implemented

OS sandbox on macOS Implemented Seatbelt

OS sandbox on Windows Implemented Low Integrity + WFP; egress enforcement uses an elevated helper

Local AI inference Implemented vLLM, llama.cpp, ONNX, Ollama, and others

Cloud AI providers Implemented OpenAI, Anthropic, Google, and others

Multi-node engine orchestration Implemented Shared queue, node registry, Redis response streams

mTLS for internal gRPC services Roadmap Server-side TLS + service JWT first

Context budget guard Implemented Heuristic guard for oversized prompt messages and tool outputs, scaled by model context settings

Multi-node tuning Roadmap Distributed fine-tuning workflows

Mobile clients Experimental React Native client exists but is not certified

Engine quotas Implemented Request and token limits by engine and subject scope

Request observability Implemented Correlation chain across layers

Stress benchmark snapshot

The following numbers are local stress-test snapshots for the authenticated request path: login, session handling, and server-side rendering of the server-driven UI route /components/_effects/yaml. They document observed behavior under concurrent load on one machine; they are not universal performance claims or comparisons with other frameworks.

Environment:

CPU: Intel Core i9-14900K

RAM: 64 GB

GPU: NVIDIA RTX 3090 Suprim, not used by this benchmark path

OS: Ubuntu 24

Python: 3.12.3

Providers: remote providers

Instances Workers / instance Clients / worker Duration Requests Errors Throughput Avg latency p95 latency

4 4 4 303.69s 468,840 0 1,543.83 req/s 33.28ms 47.05ms

8 4 4 302.39s 718,050 0 2,374.55 req/s 43.30ms 47.94ms

Command shape:

PYTHONPATH=. .venv/bin/python scripts/multi_core_benchmark.py \ --instances 4 \ --workers 4 \ --clients 4 \ --duration-seconds 300 \ --profile custom \ --profile-log-summary

For the 4-instance run, aggregate resource usage was:

Metric Value

Avg total CPU 1326.9%

Peak RSS 7092.1 MB

Peak USS 5936.8 MB

Peak PSS 5986.8 MB

RSS can overstate standalone memory use because shared pages are counted more than once.

Threat model

The framework offers security as foundational primitives: OS-level sandboxing, immutable audit, RBAC, scoped data access, and at-rest encryption of secrets stored in the database. Modules operate within these primitives according to the policies they declare (sandbox allow-lists, RBAC permissions, access scope).

The threat model covered by the framework is accidental supply-chain compromise — a third-party module with a bug or compromised dependency must not be able to harm the system beyond its declared scope.

The threat model not covered is a deliberately malicious module installed consciously by the deployer. The selection of modules to install and the configuration of security policies are deployer responsibilities.

The framework's config.yaml is plaintext and is generated locally by the setup wizard. It is not designed to be deployed. Secrets are encrypted at rest only when stored in the database.

Quick start

Requirements

Python >=3.12, -m "desc" --autogenerate Create a new module data migration

python main.py rollback data --steps 1 Roll back one data migration step

python main.py validate-config Validate current configuration

python main.py setup examples/setup.example.yaml Run initial setup from YAML

python main.py install-engines examples/engine-install.example.yaml Install and activate engines/models from YAML

python main.py install-extractors examples/extractor-install.example.yaml Install and activate extractors from YAML

python main.py reset-install Reset the installation state

python main.py module-status Show module discovery and load status

python main.py knowledge-rebuild --all --dry-run Rebuild knowledge projections (dry run)

Engine installation from YAML

The core CLI can install and activate engine instances from a YAML file:

python main.py install-engines path/to/engines.yaml

The command starts the application runtime, uses the core SDK context, and follows the same engine lifecycle used by the runti

[truncated for AI cost control]