Uh oh!
There was an error while loading. Please reload this page.
Notifications You must be signed in to change notification settings
Fork 4
Star 19
BranchesTags
Open more actions menu
Folders and files
NameName
Last commit message
Last commit date
Latest commit
History
5 Commits
5 Commits
.github/workflows
.github/workflows
apps/aurora
apps/aurora
configs
configs
configuration
configuration
dashboard-ui
dashboard-ui
docs-assets
docs-assets
docs
docs
heartbeat-worker
heartbeat-worker
helm
helm
internal
internal
monitoring
monitoring
.dockerignore
.dockerignore
.goreleaser.yaml
.goreleaser.yaml
CODE_OF_CONDUCT.md
CODE_OF_CONDUCT.md
Dockerfile
Dockerfile
LICENSE
LICENSE
Makefile
Makefile
README.md
README.md
SECURITY.md
SECURITY.md
artifacthub-repo.yml
artifacthub-repo.yml
docker-compose.yml
docker-compose.yml
go.mod
go.mod
go.sum
go.sum
package.json
package.json
Repository files navigation
One API for every AI provider. Self-hosted. No vendor lock-in.
14 provider types • OpenAI & Anthropic compatible • Go • Apache 2.0 • Built for raw speed
What Aurora Does
Aurora sits between your app and LLM providers. Your app sends requests using the standard OpenAI or Anthropic SDK — Aurora routes them to whichever provider you've configured. One format handles everything — you dont need to worry about provider-specific formats.
Before: hardcoded provider
client = OpenAI(base_url="https://api.openai.com/v1", api_key="sk-...")
After: Aurora Gateway
client = OpenAI(base_url="http://localhost:8080/v1", api_key="your-aurora-key")
No SDK changes. No format changes. Just swap the base_url.
Features
Routing & Providers
14 provider types — OpenAI, Anthropic, Gemini, Groq, DeepSeek, OpenRouter, xAI, Z.ai, MiniMax, Azure OpenAI, Oracle, Ollama, vLLM, Jina
Auto-discovery — set an API key as an env var, restart, provider + all its models appear automatically
Provider pools — group multiple keys/endpoints, load-balance with round-robin or weighted distribution, health-aware failover
Model aliases — rename/remap any model to a custom identifier across the entire gateway
Model overrides — enable or disable specific models per user path, persisted via dashboard or user_pricing.yaml
Fallback — automatic failover on 5xx/429, or manual rules (from config or external JSON) mapping failed provider+model to backups
Resilience — exponential backoff with jitter, circuit breaker per provider (closed → open → half-open), per-provider override of global retry/circuit-breaker settings
Multiple instances — run OPENAI_EAST_API_KEY and OPENAI_WEST_API_KEY as separate providers
Custom base URLs — override any provider's endpoint (corporate proxies, regional endpoints)
Passthrough — /p/{provider}/* for full upstream API access (not just chat completions); filter which provider types get passthrough routes
Config-driven workflows — per-request routing, caching, guardrail, audit, usage, budget, and fallback behavior controlled by persisted workflow documents
API Surface
OpenAI-compatible — /v1/chat/completions, /v1/embeddings, /v1/rerank, /v1/models, /v1/files, /v1/batches
Responses API — /v1/responses with full CRUD, cancel, input items, compact
Anthropic-compatible — /v1/messages, /v1/messages/count_tokens (native Anthropic wire format); optional dedicated ingress at /v1/messages
Streaming — SSE streaming for all endpoints, preserved end-to-end
Keep-only-aliases mode — hide raw provider models from /v1/models and expose only aliased names
Configured provider models mode — fallback (add listed models to auto-discovered) or allowlist (only serve explicitly listed models)
Caching
Exact cache — SHA-256 hash match on request, Redis-backed, async writes
Semantic cache — vector similarity with configurable threshold, supports Qdrant, pgvector, Pinecone, Weaviate
Prompt cache — forwards cache_control to Anthropic/OpenAI/Gemini native prompt caching; configurable modes (auto, manual, off), component toggles, and minimum token threshold
Model registry cache — local filesystem + Redis, offline-safe; supports vendored JSON snapshots with per-field user pricing overrides
Security & Guardrails
Master key — top-level gateway auth
Managed API keys — scoped, rate-limited, per-key model authorization, usage stats
Rate limiting — per-key rate limiting backed by in-memory or Redis
PII redaction — email, phone, SSN, credit card detection and masking
Prompt injection blocking — detects and blocks injection attempts
System prompt protection — inject, override, or decorate system prompts
Regex blocking — custom pattern matching with block or sanitize actions
Length limits — character/token count enforcement on requests
LLM-based altering — guardrail that rewrites message content via an auxiliary LLM call (anonymization, custom prompts)
Guardrail direction & ordering — run before provider dispatch (input), after response (output), or both; same-order guardrails run in parallel
Batch guardrails — apply configured guardrails to inline items in /v1/batches requests
Observability
Audit logging — full request/response capture, buffered writes, configurable retention (body/header logging, buffer size, flush interval), live SSE stream
Usage analytics — per-model token counting, cost tracking, daily aggregation by model/user-path, pricing recalculation action
Prometheus metrics — aurora_requests_total, aurora_request_duration_seconds, aurora_requests_in_flight, plus gateway phase timing
Admin dashboard — React SPA built into the Go binary: providers, pools, models, aliases, guardrails, cache, usage, audit, auth keys, workflows, console, playground
pprof endpoints — Go runtime profiling at /debug/pprof/* (heap, goroutine, mutex, block, threadcreate)
Structured logging — configurable format (JSON/text), level (debug/info/warn/error), source info, service metadata
Cost Control
Token saver — policy-driven output compression (profiles: concise, caveman, ultra, wenyan); scoped to specific models/providers via include/exclude filters; configurable on-error behavior (allow/block)
Pricing management — per-model pricing overrides, recalculation, import/export
Usage budgets — per-key usage tracking and limits, per-request budget enforcement via workflow feature flags
Developer Experience
Single binary — npm install -g iaurora or docker pull aurorahq/aurora
CLI — aurora init, aurora models sync/diff/show, aurora update, aurora uninstall
CLI tools API — admin REST endpoints for CLI configuration sync, gated separately
Swagger docs — /swagger/index.html (build-tag gated)
Config profiles — pre-built configs for local, local-power, and team deployments
3-layer config — code defaults → config.yaml → env vars (env vars win)
Helm chart — deploy on Kubernetes with pre-built Helm chart
Docker Compose — full infrastructure stack: Redis, PostgreSQL, Qdrant, Prometheus, Grafana
Grafana dashboard — pre-configured panels for request rate, errors, latency, in-flight requests, per-model breakdown
Quick Start
Start routing AI traffic in 60 seconds.
Option A — CLI (npm)
npm install -g iaurora mkdir my-gateway && cd my-gateway aurora init # creates config.yaml, .env, data/
Set your provider keys in .env:
── REQUIRED ──────────────────────────────────────────────
AURORA_MASTER_KEY="your-secure-key"
── PROVIDER API KEYS (at least one) ─────────────────────
OPENAI_API_KEY="sk-..." ANTHROPIC_API_KEY="sk-ant-..." GEMINI_API_KEY="..." GROQ_API_KEY="gsk_..." DEEPSEEK_API_KEY="..." OPENROUTER_API_KEY="..." XAI_API_KEY="..." ZAI_API_KEY="..." MINIMAX_API_KEY="..." AZURE_API_KEY="..." ORACLE_API_KEY="..." OLLAMA_API_KEY="..." VLLM_API_KEY="..." JINA_API_KEY="..."
── OPTIONAL FEATURE TOGGLES (set true to enable) ────────
LOGGING_ENABLED=true # Audit logging to storage METRICS_ENABLED=true # Prometheus /metrics endpoint GUARDRAILS_ENABLED=true # Content safety filters TOKEN_SAVER_ENABLED=true # Output compression to cut token use
── PRODUCTION STORAGE ───────────────────────────────────
STORAGE_TYPE=postgresql
POSTGRES_URL=postgres://user:pass@localhost:5432/aurora
── REDIS CACHE (model cache + response cache) ──────────
REDIS_URL=redis://localhost:6379
RESPONSE_CACHE_SIMPLE_ENABLED=true
aurora
Option B — inline env vars (no .env needed)
Linux / macOS
AURORA_MASTER_KEY=your-secure-key \ OPENAI_API_KEY=sk-... \ ANTHROPIC_API_KEY=sk-ant-... \ GEMINI_API_KEY=... \ GROQ_API_KEY=gsk_... \ DEEPSEEK_API_KEY=... \ OPENROUTER_API_KEY=... \ XAI_API_KEY=... \ ZAI_API_KEY=... \ MINIMAX_API_KEY=... \ AZURE_API_KEY=... \ ORACLE_API_KEY=... \ OLLAMA_API_KEY=... \ VLLM_API_KEY=... \ JINA_API_KEY=... \ LOGGING_ENABLED=true \ METRICS_ENABLED=true \ GUARDRAILS_ENABLED=true \ TOKEN_SAVER_ENABLED=true \ aurora
Windows PowerShell
$env:AURORA_MASTER_KEY="your-secure-key"; ` $env:OPENAI_API_KEY="sk-..."; ` $env:ANTHROPIC_API_KEY="sk-ant-..."; ` $env:GEMINI_API_KEY="..."; ` $env:GROQ_API_KEY="gsk_..."; ` $env:DEEPSEEK_API_KEY="..."; ` $env:OPENROUTER_API_KEY="..."; ` $env:XAI_API_KEY="..."; ` $env:ZAI_API_KEY="..."; ` $env:MINIMAX_API_KEY="..."; ` $env:AZURE_API_KEY="..."; ` $env:ORACLE_API_KEY="..."; ` $env:OLLAMA_API_KEY="..."; ` $env:VLLM_API_KEY="..."; ` $env:JINA_API_KEY="..."; ` $env:LOGGING_ENABLED="true"; ` $env:METRICS_ENABLED="true"; ` $env:GUARDRAILS_ENABLED="true"; ` $env:TOKEN_SAVER_ENABLED="true"; ` aurora
Windows CMD
set AURORA_MASTER_KEY=your-secure-key ^ && set OPENAI_API_KEY=sk-... ^ && set ANTHROPIC_API_KEY=sk-ant-... ^ && set GEMINI_API_KEY=... ^ && set GROQ_API_KEY=gsk_... ^ && set DEEPSEEK_API_KEY=... ^ && set OPENROUTER_API_KEY=... ^ && set XAI_API_KEY=... ^ && set ZAI_API_KEY=... ^ && set MINIMAX_API_KEY=... ^ && set AZURE_API_KEY=... ^ && set ORACLE_API_KEY=... ^ && set OLLAMA_API_KEY=... ^ && set VLLM_API_KEY=... ^ && set JINA_API_KEY=... ^ && set LOGGING_ENABLED=true ^ && set METRICS_ENABLED=true ^ && set GUARDRAILS_ENABLED=true ^ && set TOKEN_SAVER_ENABLED=true ^ && aurora
Option C — Docker
docker run -d --name aurora -p 8080:8080 \ -e AURORA_MASTER_KEY="your-secure-key" \ -e OPENAI_API_KEY="sk-..." \ -e ANTHROPIC_API_KEY="sk-ant-..." \ -e GEMINI_API_KEY="..." \ -e GROQ_API_KEY="gsk_..." \ -e DEEPSEEK_API_KEY="..." \ -e OPENROUTER_API_KEY="..." \ -e XAI_API_KEY="..." \ -e ZAI_API_KEY="..." \ -e MINIMAX_API_KEY="..." \ -e AZURE_API_KEY="..." \ -e ORACLE_API_KEY="..." \ -e OLLAMA_API_KEY="..." \ -e VLLM_API_KEY="..." \ -e JINA_API_KEY="..." \ -e LOGGING_ENABLED=true \ -e METRICS_ENABLED=true \ -e GUARDRAILS_ENABLED=true \ -e TOKEN_SAVER_ENABLED=true \ aurorahq/aurora
Option D — Kubernetes (Helm)
Quick dev — Groq, no Redis, no auth
helm install aurora ./helm \ --namespace aurora --create-namespace \ --set image.repository=aurorahq/aurora \ --set image.tag=latest \ --set providers.groq.apiKey="gsk_your_key_here" \ --set providers.groq.enabled=true \ --set redis.enabled=false \ --set auth.masterKey=""
Production — multiple providers, auth, Redis
helm upgrade --install aurora ./helm \ --namespace aurora --create-namespace \ --set image.repository=aurorahq/aurora \ --set image.tag=latest \ --set auth.masterKey="your-secure-key" \ --set providers.openai.apiKey="sk-..." \ --set providers.openai.enabled=true \ --set providers.anthropic.apiKey="sk-ant-..." \ --set providers.anthropic.enabled=true \ --set providers.gemini.apiKey="..." \ --set providers.gemini.enabled=true \ --set providers.groq.apiKey="gsk_..." \ --set providers.groq.enabled=true \ --set providers.deepseek.apiKey="..." \ --set providers.deepseek.enabled=true \ --set redis.enabled=true
Full Helm docs: helm/README.md
Test your gateway
OpenAI format
curl http://localhost:8080/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer your-master-key" \ -d
[truncated for AI cost control]