Diagnosis of Human Object Interaction Detectors for Real World Educational Applications
Human-object interaction (HOI) recognition is critical for automatically analyzing student behavior in complex educational environments. Although state-of-the-art HOI detectors perform well on benchmark datasets, their performance often degrades in real-world training environments due to domain-specific objects, occlusions, and complex visual conditions. This paper introduces a diagnosis-driven framework that integrates a triplet-level HOI error taxonomy with error-factor attribution analysis for real-world educational video data, studied in the context of Critical Care Air Transport Team (CCATT) mixed-reality medical training. The approach improves the macro-F1 score of a pretrained CDN model from 48.6 to 90.2 on the CCATT dataset.
[2606.02789] Diagnosis of Human Object Interaction Detectors for Real World Educational Applications
[Submitted on 1 Jun 2026]
Title:Diagnosis of Human Object Interaction Detectors for Real World Educational Applications
View a PDF of the paper titled Diagnosis of Human Object Interaction Detectors for Real World Educational Applications, by Divya Mereddy and 3 other authors
View PDF
Abstract:Human-object interaction (HOI) recognition is critical for automatically analyzing student behavior in complex educational environments. Although state-of-the-art (SOTA) HOI detectors perform well on benchmark datasets, their performance often degrades when deployed in real-world training environments due to domain-specific objects, occlusions, and complex visual conditions. In this paper, we introduce a diagnosis-driven framework that integrates a triplet-level HOI error taxonomy with error-factor attribution analysis for real-world educational video data. We study this problem in the context of Critical Care Air Transport Team (CCATT) mixed-reality medical training. Based on an analysis of HOI failure modes and their causes, we develop a diagnosis-informed refinement strategy for adapting pretrained HOI models to the target domain. Experiments on the CCATT dataset show that this approach improves the macro-F1 score of a pretrained CDN model from 48.6 to 90.2 through targeted refinement guided by diagnosed error factors. These results highlight the value of detailed diagnostic analysis for informing targeted adaptation of HOI models in real-world educational environments.
Subjects:
Computer Vision and Pattern Recognition (cs.CV)
Cite as: arXiv:2606.02789 [cs.CV]
(or arXiv:2606.02789v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2606.02789
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: Divya Mereddy [view email] [v1] Mon, 1 Jun 2026 18:55:39 UTC (9,444 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled Diagnosis of Human Object Interaction Detectors for Real World Educational Applications, by Divya Mereddy and 3 other authors
View PDF
view license
Current browse context:
cs.CV
new | recent | 2026-06
Change to browse by:
cs
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
Loading...
Data provided by:
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)