Physical Atari: A Robust and Accessible Platform for Real-time Reinforcement Learning on Robots
Researchers built a robot called Robotroller that actuates an Atari CX40+ controller and a device called Atari Devbox that renders game frames and reward signals from the Arcade Learning Environment. Together with an off-the-shelf camera and desktop computer, the system forms Physical Atari, a robust (bearings for movement, high-frequency servo monitoring) and accessible (under $1,000, 3D-printed parts) platform for studying real-world reinforcement learning. Weeks of non-stop experiments validated that RL algorithms can learn directly on robots, while showing that small distribution shifts between training and deployment severely degrade policy performance, emphasizing the need for on-device adaptation.
[2606.19357] Physical Atari: A Robust and Accessible Platform for Real-time Reinforcement Learning on Robots
[Submitted on 29 May 2026]
Title:Physical Atari: A Robust and Accessible Platform for Real-time Reinforcement Learning on Robots
View a PDF of the paper titled Physical Atari: A Robust and Accessible Platform for Real-time Reinforcement Learning on Robots, by Khurram Javed and 4 other authors
View PDF HTML (experimental)
Abstract:We built a robot called the Robotroller that actuates an Atari CX40+ controller and a device called the Atari Devbox that renders the game frame and the reward signal from the Arcade Learning Environment on a screen. The Robotroller and the Atari Devbox, together with an off-the-shelf camera and a desktop computer, constitute a system that can be used to study reinforcement learning algorithms in the physical world. We call the full system Physical Atari. In this paper, we detail the key decisions that make Physical Atari a robust and accessible platform. To make the system robust, we designed the Robotroller so that all movement is done through bearings, which reduces wear. Additionally, we wrote software that monitors the state of the servos at a high frequency and intervenes to limit stress. To make the system accessible, we used affordable off-the-shelf components and parts that can be manufactured using consumer 3D printers. Physical Atari can be built for under $1,000 and has been used for weeks of non-stop reinforcement learning experiments without any mechanical failures. We used it to validate that reinforcement learning algorithms can learn directly on robots and show that even small distribution shifts between learning and deployment can significantly degrade the performance of policies. Our results underscore the importance of on-device adaptation for strong performance on robots.
Comments: To appear at RLC 2026
Subjects:
Robotics (cs.RO); Artificial Intelligence (cs.AI)
Cite as: arXiv:2606.19357 [cs.RO]
(or arXiv:2606.19357v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2606.19357
arXiv-issued DOI via DataCite
Submission history
From: Khurram Javed [view email] [v1] Fri, 29 May 2026 19:24:57 UTC (4,231 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled Physical Atari: A Robust and Accessible Platform for Real-time Reinforcement Learning on Robots, by Khurram Javed and 4 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.RO
new | recent | 2026-06
Change to browse by:
cs cs.AI
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
Loading...
Data provided by:
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)