How Claude performs on robotics tasks

Skip to main contentNewsResearchPolicyCommitmentsCompanyTry Claude Chapter 01: Summary of findingsClaude plays robotics01Summary of findings02Simple settings03Direct control is bad, but improving04Tools bridge some of the gap05What’s the bottleneck?06Conclusion07Appendix08LatencyPublished

Shmuel Berman, Michael Ilie, Jia Deng, and Daniel Freeman

Do language models’ strengths transfer to robotics, a domain which requires the synthesis of logical skills and precise 3D understanding? Can a model perceive a scene, understand a particular robot’s state, and issue actions that reliably effect change in the physical world?

We begin by evaluating robot-relevant capabilities on a set of simple control tasks, including classic reinforcement learning (RL) problems such as balancing an inverse pendulum and controlling a hopper.

To approximate an upper bound on direct-control performance, we pause the simulator between LLM calls so real-time latency does not dominate the results. Without this, many direct tests would fail for a trivial reason: the models would simply react too slowly to control the environment. We expect inference speed to continue increasing, and this setup gives a clear view of best-case capability as it does.