Notifications You must be signed in to change notification settings
Fork 0
Star 15
BranchesTags
Open more actions menu
Folders and files
NameName
Last commit message
Last commit date
Latest commit
History
41 Commits
41 Commits
.agents/skills/talkthrough
.agents/skills/talkthrough
.claude-plugin
.claude-plugin
.github
.github
assets
assets
docs
docs
examples
examples
integrations
integrations
scripts
scripts
src/talkthrough_mcp
src/talkthrough_mcp
tests
tests
.gitignore
.gitignore
.mcp.json
.mcp.json
.python-version
.python-version
AGENTS.md
AGENTS.md
CHANGELOG.md
CHANGELOG.md
CLAUDE.md
CLAUDE.md
CODE_OF_CONDUCT.md
CODE_OF_CONDUCT.md
CONTRIBUTING.md
CONTRIBUTING.md
LICENSE
LICENSE
README.md
README.md
SECURITY.md
SECURITY.md
llms-install.md
llms-install.md
llms.txt
llms.txt
pyproject.toml
pyproject.toml
server.json
server.json
uv.lock
uv.lock
Repository files navigation
Quickstart · Tools · FAQ · Troubleshooting · Changelog · Contributing
Feedback ingestion for AI agents. Record your screen and talk; your agent does the rest — files the bugs, writes the spec, builds the backlog.
talkthrough-mcp is a local-first MCP server that turns a narrated screen recording (or any video/audio file) into agent-ready structured data: timestamped transcript segments, optional who-said-what speaker labels, scene-change keyframes, OCR'd on-screen text, and wall-clock anchoring. Everything is served through lazy retrieval tools, so a 30-minute recording never floods the model context — the agent pulls exactly the transcript slice, moment bundle, or frame it needs.
There is no LLM inside the server and no cloud anywhere in the path: ffmpeg, faster-whisper, and RapidOCR run on your machine, and the calling agent brings the intelligence. What makes it different from screen-recorder SaaS and video-analyzer MCPs: it works on arbitrary local files, it ships the agent workflows (server prompts + example agents), and it anchors every timestamp to wall-clock time — so "the moment I said the checkout hung" maps straight to the right window of your server logs.
Quickstart
One command, no system dependencies: ffmpeg falls back to a bundled build, OCR is pip-only, and whisper models download themselves on first use. The only prerequisite is uv (brew install uv or curl -LsSf https://astral.sh/uv/install.sh | sh).
Claude Code
Two install paths — pick one, not both (the plugin already includes the server; installing both would register it twice):
Server only — the 7 tools + 5 prompts, and nothing else on your system. Choose this for a minimal setup, or when you manage MCP servers yourself across several clients:
claude mcp add -s user talkthrough -- uvx "talkthrough-mcp[diarization]"
Full plugin — the same server, plus native slash commands (/talkthrough:triage-recording, …) that handle the ceremony for you, a ready-made triage subagent, and an agent skill that teaches Claude the workflow. Choose this for the best out-of-the-box experience:
/plugin marketplace add korovin-aa97/talkthrough-mcp /plugin install talkthrough@talkthrough
Every other MCP client
Claude Desktop
claude_desktop_config.json:
{ "mcpServers": { "talkthrough": { "command": "uvx", "args": [ "talkthrough-mcp[diarization]" ] } } }
More: integrations/claude-desktop/
Cursor
~/.cursor/mcp.json (or project .cursor/mcp.json):
{ "mcpServers": { "talkthrough": { "command": "uvx", "args": [ "talkthrough-mcp[diarization]" ] } } }
More: integrations/cursor/
OpenAI Codex CLI
~/.codex/config.toml (or project-scoped .codex/config.toml in trusted projects):
[mcp_servers.talkthrough] command = "uvx" args = ["talkthrough-mcp[diarization]"]
More: integrations/codex/
Gemini CLI
~/.gemini/settings.json:
{ "mcpServers": { "talkthrough": { "command": "uvx", "args": [ "talkthrough-mcp[diarization]" ] } } }
More: integrations/gemini-cli/
Cline / Roo Code
cline_mcp_settings.json (via MCP Servers UI):
{ "mcpServers": { "talkthrough": { "command": "uvx", "args": [ "talkthrough-mcp[diarization]" ] } } }
More: integrations/cline/
OpenClaw
~/.openclaw/openclaw.json:
{ "mcp": { "servers": { "talkthrough": { "command": "uvx", "args": [ "talkthrough-mcp[diarization]" ] } } } }
More: integrations/openclaw/
OpenCode
opencode.json (project) or ~/.config/opencode/opencode.json:
{ "mcp": { "talkthrough": { "type": "local", "command": [ "uvx", "talkthrough-mcp[diarization]" ], "enabled": true } } }
More: integrations/opencode/
Goose
~/.config/goose/config.yaml:
extensions: talkthrough: enabled: true type: stdio cmd: uvx args: ["talkthrough-mcp[diarization]"]
More: integrations/goose/
GitHub Copilot CLI
~/.copilot/mcp-config.json:
{ "mcpServers": { "talkthrough": { "command": "uvx", "args": [ "talkthrough-mcp[diarization]" ] } } }
More: integrations/copilot-cli/
Windsurf
~/.codeium/windsurf/mcp_config.json:
{ "mcpServers": { "talkthrough": { "command": "uvx", "args": [ "talkthrough-mcp[diarization]" ] } } }
More: integrations/windsurf/
Zed
settings.json (Zed):
{ "context_servers": { "talkthrough": { "source": "custom", "command": { "path": "uvx", "args": [ "talkthrough-mcp[diarization]" ] } } } }
More: integrations/zed/
Any other MCP stdio client uses the same server command: uvx "talkthrough-mcp[diarization]". Per-engine folders with exactly these snippets plus verification steps live in integrations/; agents can self-install via llms-install.md.
Who said what (speaker diarization) — included in the configs above
Multi-person recordings (meetings, interviews, panels) can carry S1/S2/… speaker labels. Every install button, snippet, and the plugin above already ship the [diarization] engine, so asking your agent "who said what" just works — diarization itself still runs only when requested per call (process_media(path=..., diarize=true, num_speakers=)), and its models download once on first use.
Prefer the minimal server without the diarization engine? Use uvx talkthrough-mcp as the command instead (the MCP registry entry also resolves to this lean form) — an explicit diarize=true will then answer with the one-line install fix. Details in Speakers.
Local checkout (development)
git clone https://github.com/korovin-aa97/talkthrough-mcp claude mcp add talkthrough -- uv run --directory /path/to/talkthrough-mcp talkthrough-mcp
Then, in your agent:
Process ~/Desktop/recording.mov and triage it — or just invoke the triage-recording server prompt.
Tools
Tool What it does
process_media(path, recorded_at?, vocabulary?, language?, model?, diarize?, num_speakers?, force?) Ingest a video/audio file: local STT, keyframes, OCR, wall-clock, opt-in speaker labels. Returns a compact summary. Idempotent by content hash — re-calls are instant; diarize=true on a processed job adds speakers without re-transcribing.
get_transcript(job_id, start_ms?, end_ms?, format?) Paginated transcript as segments, text, or srt (speaker-prefixed when diarized, plus a roster header); truncation returns next_start_ms.
get_frames(job_id, at_ms? | start_ms?+end_ms?, max_frames?, include_duplicates?) Keyframe images nearest a timestamp or evenly thinned across a range (unique frames by default, max 6/call); each frame names its absolute path.
get_moment(job_id, start_ms, end_ms) The "one remark" bundle: transcript slice + up to 3 frames + their OCR text + wall-clock range (+ speakers_in_range when diarized).
search(job_id, query) Substring search over the transcript AND on-screen OCR text; hits carry t_ms/t_wall, frame refs, and the speaker when diarized.
extract_frame(job_id, at_ms, crop?) Exact-timestamp full-resolution re-extract from the source video (optional crop) when keyframes miss the instant; returns the file's absolute path.
list_jobs() Recent processed recordings with durations, wall-clock starts, counts, and speaker counts when diarized.
Every tool description ships 10+ usage examples, so agents pick the right tool without extra prompting.
Server prompts (slash commands in MCP clients)
Prompt Workflow
triage-recording Narrated screencast → precise findings JSON (bug/feature/question routing, frame evidence)
spec-from-workshop Recorded workshop → structured spec with quoted decisions and open questions
backlog-from-demo Product demo → prioritized backlog with timestamped evidence
meeting-actions Meeting audio → action items, decisions, open questions
correlate-with-logs Recording remarks ↔ system logs via wall-clock windows
The same prompts live as plain files in examples/prompts/ if your client doesn't surface MCP prompts. The findings contract used by triage-recording is examples/output-contract.schema.json.
Works as a skill too (no MCP required)
The same workflow ships as a cross-engine Agent Skill at .agents/skills/talkthrough/ — Claude Code, Codex CLI ($talkthrough), Cursor, Copilot, Gemini CLI, Goose and other SKILL.md-compatible tools read it. Agents without MCP wiring can drive the CLI directly: talkthrough-mcp process recording.mov --json prints the same summary the MCP tool returns, and the job store is shared either way.
Wall-clock anchoring
Every timestamped result carries both t_ms (video-relative) and t_wall (ISO 8601 real time) once the recording start is known. Resolution ladder:
recorded_at parameter (agent/user override) → confidence exact
QuickTime com.apple.quicktime.creationdate tag, carries the local timezone (QuickTime Player recordings; ⌘⇧5 wrote it before macOS 26) → high
Container creation_time tag (UTC) → medium — macOS 26+ ⌘⇧5/ReplayKit screen recordings land here (no creationdate tag anymore); pass recorded_at= when local-tz t_wall matters
File mtime minus duration (recorders finalize files at recording END) → low
Nothing → tools still work with relative t_ms only
Why it matters: "the upload spinner froze here" becomes a ±30 s grep window in your server logs.
Speakers (optional diarization)
With the [diarization] extra installed (included in every generated config — see Quickstart), process_media(diarize=true) labels who said what — locally, like everything else here (sherpa-onnx runtime, no torch, no accounts, no GPU):
Speakers become S1, S2, … in order of first appearance; every transcript segment gets a speaker, and the tools surface it everywhere — roster with talk time in get_transcript, speakers_in_range in get_moment, speaker on search hits, S1: prefixes in the text/SRT formats, a speaker count in list_jobs.
Know the headcount? Pass num_speakers. Clustering to an exact k removes the main failure mode of unknown-count mode (similar voices merging or one voice splitting). Agents are instructed to do this via the tool guidance; do the same in your own calls.
Already processed a recording? Calling process_media(diarize=true) on it re-runs only diarization — whisper is not re-run, and labels appear in the existing job in seconds. Same for changing num_speakers.
Labels are anonymous by design; mapping S1 → "Alice" is the calling agent's job (self-introductions, vocatives, the attendees argument of the meeting-actions prompt). The server never guesses names.
Models download once (~47 MB total) from pinned, checksum-verified URLs into ~/.talkthrough/models/; warm runs are zero-network like the rest of the pipeline. Speed on an M-series CPU (4 threads): a 26-minute meeting diarizes in about 2 minutes (RTF ≈ 0.08), on top of the transcription time. Memory: expect on the order of 1–1.5 GB peak RSS while an hour-plus meeting is being diarized (measured on a real 73-minute recording); it is released when the stage completes.
Role Model Download Weights license
Segmentation pyannote segmentation-3.0 (ONNX export by k2-fsa) 7 MB MIT
Embedding (default) NeMo en_titanet_small 40 MB Apache-2.0 (per its NGC model card, the NeMo Toolkit lice
[truncated for AI cost control]