BenchLocal – AI-powered local business search

Uh oh!

There was an error while loading. Please reload this page.

Notifications You must be signed in to change notification settings

Fork 36

Star 366

BranchesTags

Open more actions menu

Folders and files

NameName

Last commit message

Last commit date

Latest commit

History

122 Commits

122 Commits

app

app

docs

docs

packages

packages

scripts

scripts

themes

themes

.env.release.example

.env.release.example

.gitignore

.gitignore

ARCHITECTURE.md

ARCHITECTURE.md

BENCHLOCAL_REGISTRY_V1.md

BENCHLOCAL_REGISTRY_V1.md

BENCH_PACK_AUTHORING.md

BENCH_PACK_AUTHORING.md

BENCH_PACK_TEMPLATE.ts

BENCH_PACK_TEMPLATE.ts

BENCH_PROTOCOL_V1.md

BENCH_PROTOCOL_V1.md

CONFIG_SCHEMA_V1.md

CONFIG_SCHEMA_V1.md

LICENSE

LICENSE

README.md

README.md

RELEASING.md

RELEASING.md

package-lock.json

package-lock.json

package.json

package.json

screenshot.png

screenshot.png

tsconfig.base.json

tsconfig.base.json

Repository files navigation

Test LLMs on real tasks. Compare models side-by-side.

Website · Download · Watch demo · Build a Bench Pack

BenchLocal is a local-first desktop app for running, comparing, and managing installable LLM Bench Packs against local or remote models.

Official Bench Packs today:

ToolCall-15

BugFind-15

DataExtract-15

InstructFollow-15

PromptAuthority-15

ReasonMath-15

StructOutput-15

CLI-40

HermesAgent-20

BenchLocal owns the shared desktop runtime:

provider configuration

model registry

Bench Pack install and update flow

per-tab sampling overrides

run execution and result history

verifier lifecycle management

persisted desktop UI state

Agent access

BenchLocal can expose a local agent surface so AI agents and automation tools can control benchmark workflows while the desktop UI stays live.

Enable it from Settings > Agent Access. The app will show:

a bearer token

the local Agent Guide URL

the OpenAPI URL

the MCP Streamable HTTP URL

The HTTP API uses JSON commands for actions such as listing Bench Packs, managing providers and models, creating tabs, selecting models, refreshing availability, starting runs, resuming runs, retrying results, and stopping active runs. Live progress is available through Server-Sent Events at /v1/events.

MCP-capable agents can connect to /mcp with the same bearer token and use standard benchlocal_* tools plus BenchLocal state resources. This is the preferred integration path for agents that support tool calls.

See docs/agent-control-api.md for endpoint details, MCP tools/resources, safety rules, and the extension checklist for adding future UI features to the agent surface.

Each Bench Pack owns its benchmark behavior:

scenario definitions

benchmark-specific prompts

scoring logic

verifier contracts where required

benchmark-specific traces and summaries

Repo layout

app/ Electron app shell, desktop UI, main process, preload, renderer

packages/benchlocal-core shared protocol, config, workspace, and theme types

packages/benchlocal-sdk authoring helpers for Bench Pack repos

packages/benchpack-host host-side install, inspection, verifier, and run orchestration logic

themes/ built-in desktop themes

scripts/ local macOS release helpers

docs/ packaging and release docs

Developer references

ARCHITECTURE.md

BENCH_PACK_AUTHORING.md

BENCH_PROTOCOL_V1.md

CONFIG_SCHEMA_V1.md

BENCHLOCAL_REGISTRY_V1.md

docs/agent-control-api.md

docs/macos-release.md

docs/windows-release.md

docs/linux-release.md

Build commands

npm run build compile the app and workspace packages for development

npm run pack compile and package the production desktop app, including DMG and ZIP artifacts

npm run build:dir compile and produce an unpacked local app bundle

npm run build:win compile and package unsigned Windows NSIS and ZIP artifacts

npm run build:linux compile and package Linux AppImage and tar.gz artifacts

npm run release:all build the signed macOS release plus Windows and Linux desktop artifacts in one command

License

MIT

About

Test LLMs on real tasks. Compare models side-by-side.

Resources

Readme

License

MIT license

Uh oh!

There was an error while loading. Please reload this page.

Activity

Custom properties

Stars

366 stars

Watchers

4 watching

Forks

36 forks

Report repository

Releases 13

v0.3.0

Latest

Jun 1, 2026

+ 12 releases

Packages 0

Uh oh!

There was an error while loading. Please reload this page.

Contributors

Uh oh!

There was an error while loading. Please reload this page.

Languages

TypeScript 90.6%

CSS 8.0%

JavaScript 1.3%

HTML 0.1%