Fei-Fei Li co-authors new embodied AI paper: Sim2Real is too costly to scale, while Real2Sim offers abundant data

Still talking about Sim2Real? What is hotter in the robotics world right now is Real2Sim!

Recently, NVIDIA GEAR, together with Fei-Fei Li's team, Georgia Institute of Technology and other institutions, jointly released a new Real2Sim system—

SimFoundry needs only a single real-world video to automatically generate a robotic simulation environment that can be interacted with, trained in, and evaluated.

SimFoundry can also automatically replace objects, adjust scene layouts, and even generate new manipulation tasks while keeping object functions and affordances unchanged. In other words, a single real-world video no longer yields only one simulation scene; it can automatically expand into an almost unlimited data-generation space.

As a result, SimFoundry can not only train robots in simulation but also reliably predict how different robot policies will perform in reality.