PhyDEX: A Benchmark for Physical Reasoning in Vision-Language-Action Models via Dynamic Extrinsic Dexterity
Hao Vo, Taisei Hanyu, Yuki Ikebe, Pham Tri Quang, Sreevenkata Anjani Tishita Godavarthi, Nghi D. Q. Bui, Daniel Sonntag, Chase Rainwater, Duy Minh Ho Nguyen, Anh Nguyen, Ngan Le
Abstract
Vision-Language-Action (VLA) models have advanced rapidly in language-conditioned robotic manipulation, yet existing benchmarks mainly evaluate gripper--object contact and largely ignore how objects interact with the environment through gravity, friction, inertia, and mass distribution. As a result, it remains unclear whether current policies truly understand physics or merely interpolate from their training data. We address this gap through the lens of dynamic extrinsic dexterity and introduce PhyDEX, a throwing benchmark in which a robot must infer an object's mass, center of mass, and friction to send it to a destination beyond its kinematic workspace. PhyDEX contains 35K trajectories (143 hours) across nine tasks and 100 scenes, covering uniform, irregular, and dynamic-mass objects with static, multiple, and moving destinations. All data are generated by a scalable, fully automatic pipeline that combines grasp generation, motion planning, and closed-loop throw optimization under randomized physical properties. We evaluate three representative VLAs, GR00T N1.7, DynamicVLA, and ForceVLA, and find that all achieve success rates below 3\%. Our analysis reveals two key bottlenecks: limited robustness of pretrained models, which causes most failures before the throw, and physical blindness, as providing ground-truth physical properties roughly doubles success. Experiments on a humanoid robot and in the real world show the same failure patterns. We hope PhyDEX will drive the development of physically grounded robot policies.
Create a lesson
Related papers
AgentRoboCritic: Understanding the Capability of Frontier Agents on Direct Robot Control
Yiyang Du, Yuxuan Kuang, Qitao Zhao et al.
A Tutorial on Complementarity-Based Contact Models for Robotics
Onur Beker, A. René Geist, Michael Posa et al.
PGX: Physically Grounded Execution for Long-Horizon Humanoid Loco-Manipulation
Kai Lu, Chenyang Ma, Ioannis Havoutis
ACE-Brain-0.7: Learning Execution-Grounded Reasoning for Embodied Agents
Yinhe Wang, Shijun Shi, Feng Pan et al.
JARVIS: An Agent That Designs, Builds, and Tests Circuit Boards on Its Own
Shibo Zhou, Tianqi Hong, Zheng Yan et al.
An Open-Source ROS 2 Framework for Shared Control in Surgical Teleoperation: Integrating the Haply Inverse3 with the da Vinci Research Kit
Balint Varga, Sebastian Fritz, Gergely Kiss et al.