Tsinghua-led team releases HandsOn benchmark for dexterous manipulation

On October 26, a research team led by Ding Ning at Tsinghua University, together with Beijing University of Posts and Telecommunications, Peking University, and Nature Will researchers, released HandsOn, a composable simulation platform and benchmark for cross-embodiment dexterous manipulation. The paper, code, and data are now public.

HandsOn includes 66 human-verified tasks and 18,000 human teleoperation demonstrations, covering tool use, fine contact, bimanual coordination, and task transfer. The platform supports five robot arms, six dexterous hands, and about 200 rigid, articulated, and deformable object assets that can be combined in randomized scenes. Data collection and quantitative evaluation in this release use the ROKAE arm, first-generation Wuji Hand, and PhysX configuration.

Tasks are split into Core-50 and Transfer-16: Core-50 provides demonstrations for training and measures execution on seen tasks, while Transfer-16 withholds target-task demonstrations to evaluate skill transfer and composition. On Core-50, the best multi-task policy among nine trained methods reached an average success rate of 29.1%. On Transfer-16, all eight multi-task trained policies averaged below 1%. In zero-shot tests, GPT-6 used the move_eef interface to control the palm and fingers, achieving 5.2% on Core-50 and 1.9% on Transfer-16 across 660 trials.

The team also tracked stages such as grasping, contact, bimanual coordination, and goal-state maintenance, finding that robots often complete the early part of an action but fail at critical later steps. The paper is available on arXiv, code on GitHub, and the dataset on Hugging Face.