待翻译：Show HN: Winuse – Cross-platform desktop GUI automation for AI agents

AI 服务暂时不可用，以下为来源正文，待恢复后补全翻译。

Notifications You must be signed in to change notification settings Fork 0 Star 0 BranchesTags Open more actions menu Folders and files NameName Last commit message Last commit date Latest commit History 35 Commits 35 Commits .cargo .cargo crates crates docs docs examples examples python/winuse python/winuse skills/winuse skills/winuse tests tests .gitignore .gitignore AGENTS.md AGENTS.md Cargo.lock Cargo.lock Cargo.toml Cargo.toml Makefile Makefile README.md README.md pyproject.toml pyproject.toml Repository files navigation Native GUI automation with a Rust core, a Python API, and an agent-oriented CLI. winuse supports macOS Accessibility and Windows UI Automation behind the same Backend trait. Development On macOS, winuse requires macOS 15.2 or later. Grant Accessibility permission to the terminal or Python host that runs winuse. Screenshots additionally require Screen Recording permission. On Windows 10 or 11, UI Automation and visible-region screenshots need no separate privacy permission. Windows integrity isolation (UIPI) can block input to an elevated application, so run winuse at the same integrity level as the target. Secure desktop windows are not supported. python3 -m venv .venv .venv/bin/pip install maturin pytest .venv/bin/maturin develop .venv/bin/pytest Python API Move text through the system clipboard (pair with copy/paste shortcuts): desktop.clipboard_write("Hello") # then press cmd+v to paste clip = desktop.clipboard_read() print(clip.kind, clip.text, clip.files) # text | files | image | empty desktop.clipboard_read(save_image="shot.png") # export an image clipboard Control windows by id: window = desktop.window(app="TextEdit") window.activate(); window.minimize(); window.restore(); window.maximize(); window.close() desktop.windows() # all real windows; each has .minimized Wait for an async UI state instead of polling snapshots yourself: send = window.find(role="button", name="Send") if send.wait_until("enabled", timeout=10): send.click() window.find(role="static-text", name="Status").wait_until("text", text="Done", timeout=30) from winuse import Desktop desktop = Desktop() if not desktop.is_authorized(): raise RuntimeError("The platform accessibility backend is unavailable") window = desktop.window(app="TextEdit") editor = window.find(role="text-area") editor.fill("Hello from winuse") editor.press("enter") editor.fill("query", submit=True) # fill then press Enter (search boxes) editor.type_text("inserted at the cursor") # type vs fill: inserts, does not replace editor.press("s", modifiers=["cmd"]) # shortcut; cmd is Ctrl on Windows editor.click(double=True) # double-click editor.hover() # hover without clicking window.find(role="list-item", name="File").drag_to(name="Folder") # drag onto a target subtree = window.find(role="group", name="Sidebar").snapshot(max_depth=5) print(subtree.text()) Watch a subtree for semantic additions and removals: messages = window.find(role="list", name="Messages") with messages.watch(interval=0.25) as changes: while change := changes.next(timeout=30): print(change.kind, change.element) Inspect every accessibility attribute exposed by an element: inspection = window.find(role="static-text", name="Message").inspect() print(inspection.text()) Full native-attribute inspection works on both platforms: macOS reports the element's AX attributes, Windows reports the UIA properties the element supports (name, automation id, class name, value, patterns, …). Capture the visible screen region occupied by an element: button = window.find(role="button", name="Send") png = button.screenshot("send-button.png", padding=8) The method always returns the PNG bytes. Passing a path also writes the same bytes to that file. The captured region reflects what is currently visible on screen, including any windows covering the element. Elements store a selector, not a long-lived native handle. Each action resolves the element against a fresh accessibility snapshot. CLI Build the CLI: cargo build --release -p winuse-cli Install the agent skill (SKILL.md + references + host binary) to ~/.agents/skills/winuse, or package a distributable zip with all three platform binaries: make install # DEST=... to override the location make package # → target/dist/winuse--skill.zip Commands accept ordinary flags and always return a JSON envelope: target/release/winuse doctor target/release/winuse windows target/release/winuse snapshot --window 123 target/release/winuse click --window 123 --role button --name Send --exact target/release/winuse fill --window 123 --role text-area --text "Hello" target/release/winuse press --window 123 --key enter target/release/winuse scroll --window 123 --direction up --amount 500 The repository also contains a distributable agent skill at skills/winuse. Its bundled executable is built from winuse-cli. Acknowledgments winuse was inspired by and references agent-browser (Vercel Labs) — its approach of driving a UI semantically for agents, with stable selectors and JSON tool responses, shaped winuse's design for native desktop apps. Activity Stars 0 stars Watchers 0 watching Forks 0 forks Report repository