Ved AI Voice Assistant: Fully Offline Local AI Agent

Notifications You must be signed in to change notification settings

Fork 0

Star 0

BranchesTags

Open more actions menu

Folders and files

NameName

Last commit message

Last commit date

Latest commit

History

38 Commits

38 Commits

api

api

data

data

graph

graph

tests

tests

ui

ui

voice

voice

.env.example

.env.example

.gitignore

.gitignore

Modelfile.coder

Modelfile.coder

Modelfile.executor

Modelfile.executor

Modelfile.standard

Modelfile.standard

Modelfile.turbo

Modelfile.turbo

README.md

README.md

init.py

init.py

main.py

main.py

auth.py

auth.py

chatbot.py

chatbot.py

command_processor.py

command_processor.py

conftest.py

conftest.py

model_adapter.py

model_adapter.py

requirements-dev.txt

requirements-dev.txt

requirements.txt

requirements.txt

telemetry.py

telemetry.py

todo.txt

todo.txt

voice_tuner.py

voice_tuner.py

Repository files navigation

A resource-constrained AI agent for Windows/Linux/macOS. Runs fully offline with local models on 12 GB VRAM / 32 GB RAM; falls back to 4 GB VRAM / 16 GB RAM when using OpenRouter/cloud API. Includes a Tkinter UI, FastAPI server, offline voice pipeline, per-thread RAG, and a planner-executor coding lane.

⚠️ Telemetry. Ved records active-user sessions in data/telemetry.json (local only, nothing leaves your network). Sessions are registered on GUI open or /auth/login, refreshed by heartbeats, and expire after 5 minutes of inactivity. To disable it entirely, add VED_TELEMETRY_DISABLED=true to .env. See Telemetry below.

Table of contents

What is Ved?

Features

Project layout

Quick start

First-time setup

Environment setup

Local model setup (Ollama)

Cloud model setup (OpenRouter)

Audio files

RAG vector-DB index

Running Ved

HTTP API

Voice pipeline

Threads, RAG, and file uploads

Script execution

Modes

Telemetry

Slash commands

Testing

Troubleshooting

What is Ved?

Ved is a single-user (or small-team) AI agent that runs entirely on your machine.

LangGraph orchestrator (chatbot.py) with three lanes:

standard — 3B CPU casual chat, no tools.

turbo / coder — planner + executor with tools (8B / 7B planner + 3B executor).

Tkinter desktop UI (ui/) with sticky mode chips, thread tabs, RAG attachment chips, and human-in-the-loop approval.

FastAPI HTTP server (api/) exposing the same chatbot via Server-Sent Events.

Offline voice (voice/) — wake-word → faster-whisper → LLM → Piper TTS, with mid-sentence barge-in.

Per-thread RAG with FIFO eviction.

Script executor (api/runner.py) for uploaded .py files.

Telemetry (telemetry.py) — local-only active-user counter.

Everything is offline by default. Flip USE_CLOUD_API=true to route calls through OpenRouter.

Features

100% offline by default; optional OpenRouter fallback.

Deterministic model switching: standard, turbo, coder, hibernate.

Per-thread RAG file uploads with FIFO eviction.

Human-in-the-loop approval for filesystem/code/app tool calls.

Interruptible offline voice pipeline.

HTTP API + Tkinter GUI.

Local telemetry (opt-out via .env).

Project layout

ved/ ├── main.py # Bootstrapper: python main.py ├── chatbot.py # Core LangGraph orchestrator ├── model_adapter.py # ChatOllama / ChatOpenAI factory ├── command_processor.py # Slash-command dispatcher ├── telemetry.py # Active-user tracking ├── Modelfile.standard # 3B casual-chat model ├── Modelfile.turbo # 8B planner model ├── Modelfile.coder # 7B coder planner model ├── Modelfile.executor # 3B executor model ├── Modelfile.hibernate # Empty profile (sleep mode) ├── data/ # Persistent state ├── graph/ # LangGraph nodes + state ├── ui/ # Tkinter UI ├── voice/ # Offline voice pipeline ├── api/ # FastAPI HTTP layer └── tests/ # Pytest suite

Quick start

python -m venv .venv

Windows: .venv\Scripts\pip install -r requirements.txt

Linux/macOS: .venv/bin/pip install -r requirements.txt

cp .env.example .env

Edit .env with your model paths / API keys.

.venv/bin/python main.py # Launch GUI

or

.venv/bin/python -m uvicorn api.server:app --port 8000 # Launch API

For full first-time setup, see below.

First-time setup

Install dependencies. See Quick start.

Copy .env.example → .env and fill in anything you need.

Download the Piper voice (~60 MB). Pick a voice at piper-voices, download both .onnx + .onnx.json, and place them in voice/assets/. Update voice_file / voice_json in .env.

Pull Ollama models (see Local model setup).

(Optional) Set DB_PATH for the RAG index (defaults to data/vectordb/index.bin).

(Optional) Set VED_TELEMETRY_DISABLED=true to disable telemetry.

(Optional) Set USE_CLOUD_API=true + API_KEY for OpenRouter.

Launch. python main.py.

Environment setup

Copy .env.example → .env and uncomment what you need.

Key Purpose

USE_CLOUD_API Route LLM calls to OpenRouter (true / unset).

OPENROUTER_MODEL OpenRouter model slug (default: qwen/qwen-2.5-coder-7b-instruct).

API_KEY OpenRouter API key.

voice_file / voice_json Piper TTS model + config paths.

wake_sound .wav played when switching to turbo.

DB_PATH RAG vector-DB index file.

VED_TELEMETRY_DISABLED Set true to disable telemetry.

VED_TELEMETRY_TIMEOUT Idle seconds before a session drops (default 300).

Local model setup (Ollama)

Install Ollama from https://ollama.com/download, then pull the models referenced by the Modelfiles:

Executor (used by turbo + coder)

ollama pull qwen2.5:3b-instruct

Standard / turbo planner

ollama pull qwen2.5:7b-instruct-q4_K_M

Coder planner

ollama pull qwen2.5-coder:7b-instruct-q4_K_M

To swap a model, edit the FROM line in the corresponding Modelfile.{mode} and re-pull.

Cloud model setup (OpenRouter)

For machines that can't host local models:

USE_CLOUD_API=true OPENROUTER_MODEL=qwen/qwen-2.5-coder-7b-instruct API_KEY=sk-or-v1-your-key-here

Get a free key at https://openrouter.ai/keys.

Audio files

Ved needs three audio assets (all configured in .env):

Asset Default path Provided?

Piper voice model voice/assets/*.onnx No — download from piper-voices.

Piper voice config voice/assets/*.onnx.json No — must match the .onnx.

Wake sound voice/assets/turbo_engine_short.wav Yes.

The wake-word model (alexa) is downloaded automatically by openwakeword on first launch.

RAG vector-DB index

Ved stores embeddings in a single binary pickle file. Set DB_PATH in .env (default: data/vectordb/index.bin). The file is created automatically on the first upload. To reset it, delete the file and restart.

Running Ved

GUI:

.venv/bin/python main.py

HTTP API:

.venv/bin/python -m uvicorn api.server:app --port 8000

API health check: curl http://127.0.0.1:8000/health

HTTP API

Key endpoints:

Method Path Purpose

GET /health Liveness check.

GET/POST /threads List / create threads.

POST /chat Send a prompt, returns SSE.

POST /chat/approval Resolve a pending tool approval.

POST /mode Set mode (standard, turbo, coder, hibernate).

GET/POST /files/thread List / upload to active thread.

GET/POST /files/global List / upload to global store.

POST /run Upload and execute a .py file.

GET/POST /telemetry/active Active users.

Full schema is in api/schemas.py.

Voice pipeline

States: WAKE_WORD → RECORDING → CONFIRMATION → PLAYING → reset.

Wake word: alexa (configurable in voice/voice_module.py).

STT: faster-whisper base on CPU.

TTS: Piper, interruptible.

Ambient calibration on first launch.

Threads, RAG, and file uploads

Each thread has its own message history and RAG store.

Upload .txt, .md, .pdf, .docx, code files, etc. to the active thread or global store.

Oldest uploads are evicted FIFO when the per-thread chunk quota is exceeded.

Use slash commands like /new, /threads, /switch, /list, /upload-global.

Script execution

POST /run accepts a .py file and executes it in a fresh tempdir:

Default timeout: 30 s (max 120 s via ?timeout_seconds=).

Output capped at 16 KiB per stream.

CLI args via args=foo bar.

Modes

Mode Planner/Chat Executor Tools Use for

standard 3B CPU — none Casual chat, CPU-only.

turbo 8B GPU 3B read/search/run/open-app Quick Q&A, summaries, light automation.

coder 7B GPU 3B full incl. edit/overwrite/propose_tool Coding, refactoring, file edits.

hibernate none none none Sleep; flushes all models from VRAM.

Switch via the UI chips or POST /mode.

Telemetry

Ved records active sessions in data/telemetry.json (local file only — nothing is sent over the network).

A session is registered when the GUI opens or /auth/login succeeds.

Heartbeats refresh activity while the user is active.

Sessions expire after VED_TELEMETRY_TIMEOUT seconds of inactivity (default 300).

Distinct users are de-duplicated by username.

To disable: add VED_TELEMETRY_DISABLED=true to .env.

Query from Python (telemetry singleton) or HTTP:

curl http://127.0.0.1:8000/telemetry/active curl -X POST http://127.0.0.1:8000/telemetry/heartbeat \ -H "Content-Type: application/json" \ -d '{"username": "api-user"}'

Slash commands

Command Action

/new [title] Create thread.

/threads List threads.

/switch Switch thread.

/rename Rename thread.

/delete Delete thread.

/clear Clear visible chat log.

/mode Switch mode.

/sleep / /hibernate Hibernate.

/wake / /resume Wake from hibernate.

/upload-global Upload to global RAG.

/run Run a .py script.

/pin / /unpin / /unpin_all Pin/unpin memories.

/list List thread attachments.

/memories Show pinned memories.

Testing

.venv/bin/python -m pytest tests/ -q

The Linux test venv can be created with requirements-dev.txt (lighter deps, no TensorFlow/Piper voice stack).

Troubleshooting

Problem Fix

ModuleNotFoundError: langchain_ollama Run pip install -r requirements.txt.

Can't reach Ollama Start ollama serve and pull the Modelfile models, or enable USE_CLOUD_API.

Voice doesn't hear wake word Delete data/telemetry.json and voice/pycache/ and relaunch.

/run times out Increase timeout_seconds (max 120).

Telemetry shows 0 users Heartbeats may have expired; lower VED_TELEMETRY_TIMEOUT or check data/telemetry.json.

License

See repository root for license details.

About

No description, website, or topics provided.

Resources

Readme

Uh oh!

There was an error while loading. Please reload this page.

Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks

Report repository

Releases

No releases published

Packages 0

Uh oh!

There was an error while loading. Please reload this page.

Contributors

Uh oh!

There was an error while loading. Please reload this page.

Languages

Python 100.0%