Skip to content
AI News HubLIVE
Original source2 min read

From Pixels to Newtons: Predicting In Vivo Joint Contact Forces from Monocular Video

Summary

A physics-free pipeline predicts instantaneous 3D hip and knee contact forces from uncalibrated monocular video, matching subject-specific musculoskeletal simulations and generalizing zero-shot to an independent instrumented cohort.

SourcearXiv Computer VisionAuthor: Jessy Lauer
From Pixels to Newtons: Predicting In Vivo Joint Contact Forces from Monocular Video
Report an error

The correction channel is not available yet. You can copy the article reference below for later.

Correction instructions
Read article

[2606.06631] From Pixels to Newtons: Predicting In Vivo Joint Contact Forces from Monocular Video

[Submitted on 4 Jun 2026]

Title:From Pixels to Newtons: Predicting In Vivo Joint Contact Forces from Monocular Video

View a PDF of the paper titled From Pixels to Newtons: Predicting In Vivo Joint Contact Forces from Monocular Video, by Jessy Lauer

View PDF HTML (experimental)

Abstract:Joint contact forces govern implant longevity, cartilage health, and rehabilitation outcomes, shaping who develops osteoarthritis, who recovers well from joint replacement, and who benefits from biomechanical interventions. Yet they remain measurable only invasively, in a few dozen patients with instrumented implants. I present a physics-free pipeline to predict instantaneous 3D hip and knee contact forces from an uncalibrated monocular video: no markers, force plates, electromyography, subject-specific imaging, or musculoskeletal model. Parametric body meshes are recovered per frame, encoded as kinematic features, and decoded into forces by a transformer whose pose stream is adaptively modulated at every layer by body shape, joint, side, activity text, and self-supervised video tokens (V-JEPA 2), unifying hip and knee in a single model. Under leave-one-subject-out cross-validation across 26 patients and 25 activity categories from the in vivo OrthoLoad database, the pipeline matches the accuracy of subject-specific musculoskeletal simulations ($0.32 \pm 0.08$ BW RMSE for hip; $0.23 \pm 0.03$ BW for knee) and resolves peak force changes smaller than those reported for gait retraining and osteoarthritis progression. Applied zero-shot to an independent instrumented cohort, it rivals or outperforms prior published methods. Even without curated activity labels, video features alone preserve accuracy and enable end-to-end inference on raw footage. Driven by the predictor, a generative motion prior produces biomechanically plausible variants with reduced peak loading, rediscovering strategies from the predictive simulation literature. This pipeline establishes uncalibrated monocular video as a viable modality for estimating joint loading, opening a path toward retrospective analysis of archived clinical recordings, primary-care screening, and at-home rehabilitation tracking.

Subjects:

Computer Vision and Pattern Recognition (cs.CV)

Cite as: arXiv:2606.06631 [cs.CV]

(or arXiv:2606.06631v1 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2606.06631

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Jessy Lauer [view email] [v1] Thu, 4 Jun 2026 18:31:00 UTC (2,831 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled From Pixels to Newtons: Predicting In Vivo Joint Contact Forces from Monocular Video, by Jessy Lauer

View PDF

HTML (experimental)

TeX Source

view license

Current browse context:

cs.CV

new | recent | 2026-06

Change to browse by:

cs

References & Citations

NASA ADS

Google Scholar

Semantic Scholar

Loading...

Data provided by:

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)

Related Papers

Recommenders and Search Tools

Link to Influence Flower

Influence Flower (What are Influence Flowers?)

Core recommender toggle

CORE Recommender (What is CORE?)

Author

Venue

Institution

Topic

About arXivLabs

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)

Key points and analysis

Article intelligence

EngineersAdvanced

Key points

  • Requires only uncalibrated monocular video, with no markers, force plates, EMG, subject-specific imaging, or musculoskeletal model.
  • Leave-one-subject-out validation across 26 patients and 25 activities yields about 0.32 BW RMSE for the hip and 0.23 BW for the knee.
  • Zero-shot application to an independent instrumented cohort rivals or outperforms prior published methods.
  • A generative motion prior driven by the predictor produces plausible variants with reduced peak loading.

Highlights and analysis are generated automatically and may contain errors. Check the original source.