待翻譯：Show HN: I kept finding robot datasets with skewed clocks, so I built a checker

AI 服務暫時不可用，以下為來源正文，待恢復後補全翻譯。

Notifications You must be signed in to change notification settings Fork 2 Star 2 BranchesTags Open more actions menu Latest commit History 6 Commits 6 Commits Folders and files NameName Last commit message Last commit date .githooks .githooks .github/workflows .github/workflows configs configs data/fixtures data/fixtures docs docs embodied_sync embodied_sync examples examples scripts scripts tests tests .gitignore .gitignore ARCHITECTURE.md ARCHITECTURE.md LICENSE LICENSE README.md README.md TESTING_STRATEGY.md TESTING_STRATEGY.md pyproject.toml pyproject.toml Repository files navigation Know whether your sensors agree before your policy pays the price. Often, robot-learning sensor streams or datasets need to be sychronized. Cameras run at one rate, robot state at another, packets arrive late, and a device reconnect can silently reset its time offset. When you are trying to finish an experiment, the last thing you need is to discover after training that the observations were paired differently on the robot than they were in the dataset. embodied-sync gives you one place to align, replay, inspect, and validate multimodal timing. It works with both live sensor streams and recordings, and it fits around the tools you already use: UMI, LeRobot, ROS 2/rosbag2 + MCAP, LSL/XDF, Rerun, and SurgSync-style datasets. Project status: alpha. The live and recorded workflows work today and have test coverage. Adapter support varies by format; Current scope spells out what each one can do and which tests need local data. Start where your data is You do not need to reorganize your workflow around the library. If you have... Why use embodied-sync? Start here A recording on disk Check skew, missing observations, and alignment policy before you spend compute on training. embsync align or the Python API Sensors running now Build causal policy observations, surface stale or unmapped streams immediately, and record the session for later review. SyncSession A result you want to inspect visually Open a portable report instead of parsing logs. You can send the same file to a collaborator. Browser GUI An alignment that is expensive to get wrong Ask for a second, independent review before you accept or publish the run. Verifier API Use the CLI for shell and batch workflows, the browser report and inspector for visual review, or the Python API inside experiments and services. They all work with the same timestamp model and alignment metadata. Want to see the full workflow first? The sync_quality_demo.ipynb notebook walks through corruption, recorded alignment, live alignment, reports, and a real LeRobot import. For a version with plain text output, use sync_quality_demo_plain.ipynb. Install The package is on PyPI - the easiest way to start using is the latest official pip wheel: pip install embodied-sync In case you want to customize or install from source: pip install -e . # core types, alignment, calibration, and live sessions pip install -e ".[dev]" # add the development and test tools pip install -e ".[full]" # add format adapters and inspection tools If you only need one ecosystem, install just that adapter: pip install -e ".[mcap]" pip install -e ".[lerobot]" pip install -e ".[lsl]" pip install -e ".[surg_sync]" pip install -e ".[umi]" pip install -e ".[rerun]" The base install is designed to be deliberately small: numpy and pyyaml only. Align a recorded run Why you need it. A dataset can look plausible frame by frame and still be wrong for learning. A nearest camera frame may come from the future, a fast state stream may be held too long, or dropped samples may disappear inside a clean-looking tensor. Run the aligner before training. It records the choice, skew, confidence, and missing status for every policy frame. How to use it. Start with a deterministic example, inject known timing problems, then generate an aligned episode and report: embsync synth --out runs/clean --seed 0 --duration-s 10 embsync corrupt runs/clean \ --profile configs/corrupt_kitchen_sink.yaml \ --out runs/bad embsync align runs/bad \ --out episodes/bad_10hz \ --target-rate-hz 10 \ --check-ground-truth embsync report episodes/bad_10hz \ --out reports/bad.html \ --json-summary reports/bad.json Use nearest_neighbor, zoh, or linear_interp globally, or set a policy per stream. The right choice depends on what the signal means. The decision table in choosing_alignment_policy.md will help you choose instead of relying on the default. The sync-quality notebook runs this same workflow in memory and plots where latency, jitter, drift, drops, and stalls appear. Synchronize a live robot Why you need it. A running policy cannot look ahead. It can only use samples that have actually arrived, which means an offline-clean dataset can still produce stale observations on the robot. SyncSession makes that causal boundary visible. You can keep your existing SDK callbacks and control loop. How to use it. Attach callbacks for push-based sensors, push polled values directly, and ask for a synchronized bundle at each policy tick: import embodied_sync as embsync with embsync.init( run_dir="runs/experiment_001", streams={ "camera": embsync.StreamConfig(rate_hz=30, tolerance_ms=20.0), "robot": embsync.StreamConfig(rate_hz=250, tolerance_ms=4.0), }, primary="camera", ) as sync: camera_sdk.on_frame( sync.attach("camera", timestamp=lambda frame: frame.device_ts_ns) ) while running: sync.push("robot", robot_sdk.read_state()) bundle = sync.get() if bundle.ok: act(bundle["camera"], bundle["robot"]) Each bundle includes per-stream skew and quality metadata. The session raises a typed SyncViolation for stale holds, missing streams, clock resets, and unmapped clock domains. It does not quietly treat them as valid observations. The session also records to the same run format used by the recorded-data tools. You can stop the robot, run embsync report runs/experiment_001, and inspect exactly what the policy saw. The online section of the sync_quality_demo.ipynb compares live observation staleness with recorded alignment of the same data. For the complete session API, see sync_session_api.md. Recover a shared clock Why you need it. Two precise timestamps are not comparable just because they are both measured in nanoseconds. Device clocks can start at different origins, drift at different rates, or reset after a reconnect. Do not hard-code an offset in preprocessing and hope it still holds next week. Measure the mapping instead. How to use it. Record a physical event that both devices can observe, such as a clap, and fit the mapping: embsync calibrate clap \ --audio recording.wav \ --events visual_events.json \ --source-domain microphone \ --target-domain camera \ --out calibration.json The calibration package also supports matched event trains and visual timestamps. Start with timestamps_clock_domains.md for the mental model and practical limits of each method. Bring your existing dataset Why you need it. You probably did not come to the lab to rewrite a dataset loader. The import tools preserve source timestamps and put each supported format into the same sample model. Your alignment and report code can then stay the same across datasets. How to use it. If you do not recognize the layout of a local directory, profile it first. The importer will suggest how to read it: embsync inspect-dataset /data/recording --out profile.json embsync infer-import /data/recording --out inference.json embsync import-auto /data/recording \ --plan inference.json \ --out runs/recording The tool scores the possible formats and clock mappings. It imports the data only when one option wins by a clear margin. If the answer is ambiguous, it stops and shows you the evidence. It only runs built-in import code; it never writes code from a guess. See automatic_dataset_import.md for the full workflow. LeRobot v3.0 has a direct path: embsync import-lerobot data/external/lerobot/pusht \ --out runs/lerobot_pusht embsync report runs/lerobot_pusht embsync align runs/lerobot_pusht --out episodes/pusht embsync export-lerobot episodes/pusht --out out/pusht_lerobot You can also export numeric aligned episodes as UMI/diffusion-policy Zarr replay buffers: embsync export-umi episodes/pusht --out out/pusht_umi.zarr The LeRobot section of the sync_quality_demo_plain.ipynb shows the equivalent Python workflow and explains what happens to timestamp precision, episode boundaries, and video references. Review results in the browser Why you need it. Timing bugs are easier to discuss when everyone can see the same missing rates, skew, confidence, and alignment policy. The browser view is one self-contained HTML file. Attach it to an experiment record, serve it from a lab machine, or send it to a collaborator. You do not need a dashboard server. How to use it. Point report at either an aligned episode or an imported run, then open the generated file in any browser: embsync report runs/recording \ --out reports/recording.html \ --json-summary reports/recording.json \ --title "grasping rig - camera replacement" For event-train calibration, the inspector shows the selected match next to the neighboring matches that the aligner rejected. You can then see whether the events really match. The public API lives in embodied_sync.inspect. Its provider interface accepts your own video, audio, force, or device-specific media reader. See interpreting_sync_reports.md for a field guide to every report column. Use the Python API Why you need it. CLI commands are convenient for one run, but experiments, dataset gates, and CI checks often need typed results. The Python API returns the same alignment metadata and report objects directly. You do not need to parse files or subprocess output. How to use it. Load a normalized run, align it, inspect the result, and write the same HTML shown above: from embodied_sync.align import align_run from embodied_sync.datasets.io import load_run from embodied_sync.reports import build_report, save_report_html run = load_run("runs/recording") aligned = align_run(run, target_rate_hz=10.0, method="zoh") report = build_report(aligned) for stream in report.streams: print(stream.name, stream.missing_rate, stream.median_skew_ns) save_report_html(aligned, "reports/recording.html") The notebooks are the most complete executable API examples: visual version and plain version. Get a second opinion with the Verifier API Why you need it. A classical alignment fit can be internally consistent and still pair the wrong events. That risk matters most on long collections, high-value demonstrations, and datasets that will be shared across a team. The Verifier API is the premium review path. It checks the proposed offset independently. If it disagrees by enough, it marks the result for inspection. It never overwrites the classical fit or its evidence. How to use it. Connect the client to your verifier endpoint, then send the reference and candidate URIs with the proposed offset: export EMBODIED_SYNC_VERIFY_URL=https://verifier.example.com export EMBODIED_SYNC_VERIFY_TOKEN='your-token' embsync verify \ file:///data/video.mp4 \ file:///data/audio.wav \ --offset-ms 20 \ --search-radius-ms 400 \ --tolerance-ms 200 \ --metadata scene=pick_001 \ --out verification.json The client sends URIs and alignment metadata, not the media bytes themselves. Your robots and CI machines do not need the verification models installed. One controlled service can review runs from every rig. The response includes the verifier identity, proposed offset, confidence, and whether a person should inspect the result. The public Python adapter adds this review to the HTML inspector. See the Verifier API guide for the Python client, inspector integration, authentication, and v1 wire contract. Test timing fa [truncated for AI cost control]