Notifications You must be signed in to change notification settings
Fork 0
Star 1
BranchesTags
Open more actions menu
Folders and files
NameName
Last commit message
Last commit date
Latest commit
History
42 Commits
42 Commits
.github/workflows
.github/workflows
config
config
crates
crates
deploy
deploy
docs
docs
evals
evals
prototypes/typescript
prototypes/typescript
scripts
scripts
.gitignore
.gitignore
.gitleaks.toml
.gitleaks.toml
Cargo.lock
Cargo.lock
Cargo.toml
Cargo.toml
Dockerfile
Dockerfile
LICENSE
LICENSE
README.md
README.md
deny.toml
deny.toml
Repository files navigation
Miser is an open-source, Rust-based AI gateway that routes OpenAI-compatible requests to the cheapest capable model through OpenRouter.
Documentation
Documentation index
High-Level Design
Low-Level Design
Security Model
Operations Runbook
Evaluation Methodology
Architecture
OpenCode / Codex / Aider / SDK | v Miser Gateway :8787 | override -> structural -> heuristics | local LLM (optional) | cloud LLM (optional) | tier policy -> OpenRouter model
The gateway is stateless, preserves unknown OpenAI request fields, forwards streaming responses, and exposes routing metadata through x-miser-* headers.
Classifier modes
Configure classifier.mode in config/miser.toml:
heuristic: zero-cost, local structural and regex classification
local_llm: OpenAI-compatible Ollama or local endpoint
cloud_llm: OpenAI-compatible cloud classifier
hybrid: heuristics first, then bounded local/cloud fallback
The default hybrid mode is conservative: the low-latency heuristic result is accepted when confident; optional model calls are attempted only for ambiguous requests and have independent deadlines.
Run locally
cp config/miser.env.example .env export OPENROUTER_API_KEY=sk-or-... cargo run -p miser-gateway -- --config config/miser.toml
Configure OpenCode:
{ "$schema": "https://opencode.ai/config.json", "provider": { "miser": { "npm": "@ai-sdk/openai-compatible", "name": "Miser Gateway", "options": { "baseURL": "http://127.0.0.1:8787/v1", "apiKey": "local" }, "models": { "auto": { "name": "Miser Auto" } } } }, "model": "miser/auto" }
Endpoints
POST /v1/chat/completions
GET /v1/models
GET /health/live
GET /health/ready
Evaluation
The versioned corpus is evals/cases.jsonl.
cargo run -p miser-evals -- --mode heuristic
The evaluator reports exact and adjacent-tier accuracy plus a confusion matrix. Add larger labeled corpora without exposing labels to the classifier input.
VPS benchmark
The Rust gateway was evaluated on the deployed VPS on 2026-08-09:
Strategy Hardware Cases Exact Adjacent Under-route Failures p50 latency p95 latency
Rust heuristics 2 vCPU, 7.8 GiB RAM, no GPU 25 92.0% 92.0% 0.0% 0
Create a user API key:
curl -X POST https://miser.rajeev.me/admin/keys \ -H "Authorization: Bearer miser_admin_" \ -H "Content-Type: application/json" \ -d '{"owner": "your-name"}'
List keys:
curl https://miser.rajeev.me/admin/keys \ -H "Authorization: Bearer miser_admin_"
Delete a key:
curl -X DELETE https://miser.rajeev.me/admin/keys/{key_id} \ -H "Authorization: Bearer miser_admin_"
Using API keys
{ "provider": { "miser": { "npm": "@ai-sdk/openai-compatible", "name": "Miser Gateway", "options": { "baseURL": "https://miser.rajeev.me/v1", "apiKey": "miser_" }, "models": { "auto": { "name": "Miser Auto" } } } }, "model": "miser/auto" }
Keys are validated on every request using constant-time hash comparison. The raw key is returned only once at creation time.
Deployment
The included Dockerfile creates a non-root image. deploy/miser.service provides a hardened systemd unit. Copy config/miser.toml and a mode-600 environment file containing OPENROUTER_API_KEY to the server.
Prototype
The original Bun/TypeScript prototype is preserved under prototypes/typescript for comparison and migration reference.
Development
cargo fmt --all cargo check --workspace cargo test --workspace cargo clippy --workspace --all-targets -- -D warnings
License
MIT
miser
MIT license
Security policy
Security policy
Activity
Stars
1 star
Watchers
0 watching
Forks
0 forks
Report repository