3D Scene Graphs: Open Challenges and Future Directions
3D Scene Graphs (3DSGs) have emerged as a powerful representation for spatial AI, combining geometric grounding with semantic and relational abstractions. They are relevant to robotics and computer vision problems including manipulation, navigation, task planning, and scene understanding. However, the field is fragmented across different communities with distinct formulations, construction pipelines, and evaluation protocols, making it difficult to compare methods and assess challenges for real-world deployment. This survey provides a unified and critical review of 3DSGs, focusing on open challenges and future directions. It formalizes 3DSGs under a common definition, analyzes modeling choices (node/edge attributes, hierarchical structure, dynamic scenes, affordance-aware extensions), reviews construction from raw sensory data, and examines downstream applications and evaluation strategies. A dedicated website supplements the survey.
[2606.19383] 3D Scene Graphs: Open Challenges and Future Directions
[Submitted on 15 Jun 2026]
Title:3D Scene Graphs: Open Challenges and Future Directions
View a PDF of the paper titled 3D Scene Graphs: Open Challenges and Future Directions, by Dennis Rotondi and 12 other authors
View PDF
Abstract:3D Scene Graphs (3DSGs) have emerged as a powerful representation for spatial AI by combining geometric grounding with semantic and relational abstractions of the environment. Their expressiveness has made them relevant to a broad range of problems in robotics and computer vision, including manipulation, navigation, task planning, scene understanding, and many others. However, the field remains fragmented: different communities adopt distinct formulations, construction pipelines, and evaluation protocols, making it difficult to compare methods, identify common assumptions, and assess remaining challenges for robust real-world deployment. This survey provides a unified and critical review of 3DSGs, with particular emphasis on open challenges and future directions. We first formalize 3DSGs under a common definition and analyze the principal modeling choices that characterize existing formulations, including node and edge attributes, hierarchical structure, dynamic scene representations, and affordance-aware extensions. We then review how 3DSGs are built from raw sensory observations, discussing the most common terminologies, conventions, and techniques. Finally, we examine downstream applications and evaluation strategies, from intrinsic graph quality to task-level performance. To support the community, we also provide a dedicated website that organizes and extends the surveyed content, accessible at this https URL.
Comments: Invited article for the Annual Review of Control, Robotics, and Autonomous Systems Volume 10
Subjects:
Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
Cite as: arXiv:2606.19383 [cs.RO]
(or arXiv:2606.19383v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2606.19383
arXiv-issued DOI via DataCite
Submission history
From: Dennis Rotondi [view email] [v1] Mon, 15 Jun 2026 08:14:08 UTC (6,076 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled 3D Scene Graphs: Open Challenges and Future Directions, by Dennis Rotondi and 12 other authors
View PDF
TeX Source
view license
Current browse context:
cs.RO
new | recent | 2026-06
Change to browse by:
cs cs.CV
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
Loading...
Data provided by:
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)