跳到主要内容
AI News HubLIVE
来源内容 · 翻译待补全6 分钟阅读

待翻译:Dyna-2: General Purpose Physical Agents with Hour-Long Autonomy — Jason Ma, Dyna Robotics

文章摘要

AI 服务暂时不可用,以下为来源摘要,待恢复后补全翻译:Our first robotics pod with Dyna’s Chief Scientist on their big bet on World Action Models & the Next Frontier of Physical AI

待翻译:Dyna-2: General Purpose Physical Agents with Hour-Long Autonomy — Jason Ma, Dyna Robotics
报告错误

纠错通道尚未开通,可先复制下方文章信息留存。

查看更正说明
直接读正文

AI 服务暂时不可用,以下为来源正文,待恢复后补全翻译。

Watch along AIE NYC and join us at AIE CODE in SF next month! This episode includes a live demo of the Taku robot that is best enjoyed on YouTube. Last month, Dyna made waves by announcing Dyna-2, a world-action model pre-trained on one million hours of human video, claiming several new scaling laws on human data across four orders of magnitude. They have now followed up the model with a physical robot agent, Dyna-2.1, that has to be seen to be believed: In his talk at AIE this year, Jason talked about how the real challenge for robotics in the real world is reliable, long-horizon autonomy, and how Dyna got its models to a 99.4% success rate folding restaurant napkins for 24 hours straight, including recovering after it pulled over the whole stack. From training a world-action model on over one million hours of human video to deploying robots that fold thousands of napkins daily at Din Tai Fung, Dyna Robotics is pushing physical AI beyond impressive demos and into real-world production. In this episode, Dyna Robotics co-founder and Chief Scientist Jason Ma joins swyx on-site to unpack how robots can learn new tasks from as little as 45 minutes of demonstrations, why predicting the future of the physical world may be the key to general-purpose robotics, and what it actually takes to build robots that work reliably for real customers. We go deep on Dyna-2, Dyna’s million-hour world-action model that combines future video prediction with robot actions, the scaling laws behind human-to-robot transfer, diffusion transformers and flow matching, and why synthetic environments and self-generated training data could unlock the next wave of robotics progress. Jason also explains why Dyna builds its own hardware instead of relying on off-the-shelf humanoids, how agentic harnesses enable robots to execute hour-long workflows, why dexterous hands and missing sensory data remain major bottlenecks, and what separates real commercial deployments from the graveyard of robotics startups. We also cover world-action models versus vision-language-action models, GPT-6 and coding agents in robotics, and Jason’s journey from AlphaGo and Penn to NVIDIA, Google DeepMind, and founding Dyna. We discuss: Why Dyna-2 was trained on over one million hours of human video World-action models: predicting future video to guide physical robot actions Diffusion transformers, flow matching, and real-time robotic control How robots learn new tasks with as little as 45 minutes of demonstrations Why robotics has no equivalent of the internet and data remains its biggest scaling challenge Human egocentric video, robot demonstrations, and cross-embodiment transfer Synthetic environments, simulation, and models that generate their own training data Why robots need on-device inference, model distillation, and custom hardware Why Dyna chose wheeled semi-humanoids instead of traditional bipedal robots The hardware bottleneck: why dexterous robot hands still overheat Best Western, Din Tai Fung, and the difference between robotics pilots and scaled deployments The Bitter Lesson of Robotics: scaling data instead of engineering specialized solutions Agentic harnesses and robots executing multi-step, hour-long physical workflows GPT-6, coding agents, and AI-generated simulation environments for robotics research World-action models vs. VLAs, and why vision alone may not be enough Jason’s journey through AlphaGo, Penn, NVIDIA, Google DeepMind, and the Eureka project Dyna’s research culture, deployment-first philosophy, and vision for millions of robots Jason Ma LinkedIn: https://www.linkedin.com/in/jason-ma-742224a2/ X: https://x.com/JasonMa2020 Dyna Robotics Website: https://www.dyna.co/ X: https://x.com/DynaRobotics Timestamps 00:00:00 Introduction and Dyna-2’s World-Action Model 00:03:08 Diffusion Transformers, Flow Matching, and Generalist Robots 00:08:19 The Robotics Data Bottleneck 00:13:50 Synthetic Data and Self-Improving World Models 00:16:51 Real-Time Inference and Model Distillation 00:19:45 Full-Stack Robotics: Hardware and Supply Chains 00:25:02 Why Dyna Built a Wheeled Semi-Humanoid 00:29:35 Teaching Robots New Tasks and Evaluating Performance 00:31:59 Autonomous Workflows and Hardware Limitations 00:36:06 Real-World Deployments at Best Western and Din Tai Fung 00:41:07 The Bitter Lesson of Robotics 00:46:13 Agentic Harnesses for Physical AI 00:49:07 GPT-6, Coding Agents, and AI-Native Robotics 00:52:02 World-Action Models vs. VLAs 00:59:17 From AlphaGo to Founding Dyna Robotics 01:04:20 Dyna’s Research Culture and the Future of Robotics Transcript Introduction: Jason Ma and Dyna Robotics Swyx [00:00:00]: All right, we’re here on site with Jason Ma, the co-founder, chief scientist of Dyna Robotics. Thanks for having us in your office. Jason Ma [00:00:06]: Of course. Thank you for coming. Vibhu [00:00:07]: It’s been a blast. We got to see—you’ll see plenty of B-roll—drinks being pulled, a lot of evals, robots with knives for the first time. So under the hood of all this you have a world action model. What is that? Dyna-2: A World Action Model That Predicts the Future Jason Ma [00:00:22]: So last month, we released our world action model called Dyna-2. So it’s the first world action model trained on more than one million hours of data. So going back to your question first, world action model is a model that’s able to predict the future video, the future state of the world, so hence the name world model. But in addition, you can also use that future generation to also produce actions robots can execute in the physical world to control the robot. So why do we do that? It’s a very powerful architecture, because if you think about how humans act in the physical world, in our brain, we are implicitly thinking about how the world will evolve based on our actions. So now we have a model architecture that’s able to do that to guide robot actions. So what we find is that by doing so, the model is much more dexterous, much more robust, and much more capable of generalizing to environments, to scenarios it’s never seen before. World Action Models: Inputs, Outputs, and Visual Reasoning Vibhu [00:01:17]: I wanna prompt this question a bit. I think a lot of people hear world model, and they think you’re outputting video or world or 3D state. You’re also doing action. I think most people don’t understand what models are under the hood of robotics. So what are the inputs and outputs? What’s the training data? What is the architecture? Give us all the details. Jason Ma [00:01:39]: So, for inputs, the most important input is video, right? You can think about videos that the robot is capturing with its cameras, right? So video or images of the past many seconds, that’s the input. But for robots, what’s really interesting is that robot also has a lot of actuators and joint encoders on itself. So the robot also knows, the angles of its joints. So those can also be inputs to this world model. That’s something that’s very different from just a pure video model that’s only looking at videos, right? So that’s the input. And additionally, for robots, you want to be able to talk to it, to instruct it to do different tasks, right? So you can also feed it language instruction. So those are what I consider the basic set of inputs that go into any robotics models, including world action models. And once we have that the output, at the basic level, it needs to output actions the robots can execute on the hardware. But in the case of world action model, what’s really interesting is that in addition to the actions, it can also output future video, right? So basically, you can almost output the future video or some representation of it first, and then use that to guide robot actions. So the way I think about it is this is kind of analogous to chain of thought, you know. And basically, the models are reasoning in the visual, the physics space, and then outputting actions. So that’s the key difference between world action models outputs versus other kind of models, robot models outputs. Diffusion Transformers, Flow Matching, and Smooth Actions Vibhu [00:03:08]: You put out a good blog post around your model. I think a lot of people understand language models or transformers, next-token predictors. In this case, you shared quite a bit. There’s diffusion, flow matching, you’re predicting next world state plus action. One thing you say is a million hours. How do you determine the architecture? Like, what are the choices there? What is the architecture? I feel like in language models, there’s a lot of take something off the shelf- Take trillions of tokens, do prediction, SFT, RL, and you have a model. What about this? How do you go from nothing to your model? Is there vanilla architecture off the shelf? Do you determine all this and what’s, what is the current state? Jason Ma [00:03:49]: Yeah, I think, yeah. So the mental framework to think about how do we build models for robotics is that there are essentially a couple things to look at. One is data, and the second is how do we make a model learn as much as possible from that data? So let’s first talk about the model actually, right? So the blog post we released is our Dyna-2 world action model. So under the hood is a diffusion transformer. The model uses an encoder-only transformer architecture. Vibhu [00:04:20]: Right, yeah. Jason Ma [00:04:20]: But what’s really interesting is that in visual domains, right, if you think about generating images, generating videos, and in this case, robots looking at images to generate actions, people have found the diffusion kind of architecture to work really well. So what is a diffusion model? A diffusion model is a model that takes some random noise and through iterative computation can go from noise to something that’s really high-dimensional, that’s something really highly defined and have very fine-grained feature. If you think about generating images, right, you want sharp edges, sharp corners, color contrast. And for robotics, even though you’re not generating image, you’re generating actions that needs to be very precise, very smooth, right? So it has kind of the same kind of consideration and desiderata as generating image. So that’s why from first principles, the whole field, not just us, have been using this kind of diffusion transformer architecture to train robotics foundation models. You need the transformer aspect because transformer is the thing that can just ingest the most amount of data, can learn. Then diffusion helps you to refine the outcome to get to the smooth, precise, fine-grained motion you need to actually execute on the robot. So that’s the architecture that goes behind a lot of our recent breakthroughs. Swyx [00:05:37]: And we’ve covered flow matching before on the podcast. It is mostly an efficiency speed-up. Like, let’s say you normally would take forty to a hundred diffusion steps. You can reduce it to four. Jason Ma [00:05:48]: Yeah, that’s right. So with the diffusion model, one thing that’s very hard in practice is that you need to tune the scheduler, like by how much you’re tuning or converting the initial noise input into the final output. So flow matching is kind of like a special case of diffusion that makes the transformation much simpler. In practice, you don’t have to tune as much, and it works just as well. That is why I think about two to three years ago, most generative modeling for image and video has shifted from a diffusion model to a flow matching model. But the concept is the same, and flow matching is just a special, simpler to tune, variant of diffusion. Yeah. Swyx [00:06:26]: It actually makes me think that. So flow matching obviously has trade-offs in quality. But actually, it’s very suited for action diffusion and robotics because you get to reevaluate in real time. Whereas in images, you are— it’s basically one s [truncated for AI cost control]

展开要点与分析

文章情报

工程师中级

要点

  • AI 服务暂时不可用,系统已先保留来源内容与降级元数据。
  • Our first robotics pod with Dyna’s Chief Scientist on their big bet on World Action Models & the Next Frontier of Physical AI

要点与分析由自动化流程生成,可能有误,请结合原始来源核实。