Skip to content
AI News HubLIVE
Original source2 min read

Inhabitant: An Agent Framework for Long-Horizon Human-Scene Interaction with Spatial Control and Quantitative Feedback

Summary

The paper introduces Inhabitant, a training-free agent framework for long-horizon human-scene interaction in which a language-model agent specifies motions via positions, paths, and poses, measures execution deviations quantitatively, and revises its control specification accordingly. The authors also build Inhabit, a benchmark of 75 household tasks across 31 furniture layouts. Inhabitant completes 98.7% of tasks with Claude Opus 5.5, 93.3% with Claude Sonnet 5.5, and 82.7% with Gemini 4 Argon; ablations confirm the necessity of both spatial control and quantitative feedback.

SourcearXiv RoboticsAuthor: Yao Zhang, Zhuchenyang Liu, Yu Xiao
Inhabitant: An Agent Framework for Long-Horizon Human-Scene Interaction with Spatial Control and Quantitative Feedback
Report an error

The correction channel is not available yet. You can copy the article reference below for later.

Correction instructions
Read article

[Submitted on 18 Nov 2026]

Title:Inhabitant: An Agent Framework for Long-Horizon Human-Scene Interaction with Spatial Control and Quantitative Feedback

View a PDF of the paper titled Inhabitant: An Agent Framework for Long-Horizon Human-Scene Interaction with Spatial Control and Quantitative Feedback, by Yao Zhang and 2 other authors

View PDF HTML (experimental)

Abstract:Recent work on human-scene interaction (HSI) generation moves from single interactions to long-horizon interaction, where a language-model agent plans and executes a sequence of interactions in a 3D scene under one instruction. However, existing agents struggle in challenging everyday scenarios (e.g., picking up a toy and throwing it into a basket) because they lack spatial control over the motion and quantitative feedback on its execution, and they often fail to complete the interaction sequence. To address these gaps, we present Inhabitant, a training-free agent framework in which the agent specifies each motion through positions, paths and poses, measures how the executed interaction deviates from its intent, and revises the control specification accordingly. For an extensive evaluation in such scenarios, we build Inhabit, a benchmark of 75 household tasks in 31 furniture layouts of an apartment, in which every task chains one to three goals on movable objects and is carried out by the whole body, excluding fingers. On Inhabit, Inhabitant completes 98.7\% of the tasks with Claude Opus 5.5, and 93.3\% and 82.7\% with Claude Sonnet 5.5 and Gemini 4 Argon, respectively. Ablations confirm that without spatial control the agent fails most tasks, and without quantitative feedback its motion becomes less precise. Code is available on the project page.

Comments: this https URL

Subjects:

Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)

Cite as: arXiv:2611.25005 [cs.RO]

(or arXiv:2611.25005v1 [cs.RO] for this version)

https://doi.org/10.48550/arXiv.2611.25005

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Yao Zhang [view email] [v1] Wed, 18 Nov 2026 18:29:20 UTC (863 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled Inhabitant: An Agent Framework for Long-Horizon Human-Scene Interaction with Spatial Control and Quantitative Feedback, by Yao Zhang and 2 other authors

View PDF

HTML (experimental)

TeX Source

view license

Additional Features

Audio Summary

Current browse context:

cs.RO

new | recent | 2026-11

Change to browse by:

cs cs.CV

References & Citations

SciX

Google Scholar

Semantic Scholar

Loading...

Data provided by:

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)

Related Papers

Recommenders and Search Tools

Link to Influence Flower

Influence Flower (What are Influence Flowers?)

Core recommender toggle

CORE Recommender (What is CORE?)

Author

Venue

Institution

Topic

About arXivLabs

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)

Key points and analysis

Article intelligence

EngineersAdvanced

Key points

  • Inhabitant is training-free: the agent explicitly specifies each motion through positions, paths, and poses, then uses quantitative deviation feedback to iteratively revise its control specification.
  • The Inhabit benchmark contains 75 household tasks across 31 apartment furniture layouts, each chaining one to three goals on movable objects and requiring whole-body execution without fingers.
  • Task completion reaches 98.7% with Claude Opus 5.5, 93.3% with Claude Sonnet 5.5, and 82.7% with Gemini 4 Argon.
  • Ablations show that removing spatial control causes most tasks to fail, while removing quantitative feedback reduces motion precision.

Highlights and analysis are generated automatically and may contain errors. Check the original source.