15 Institutions Release TouchScale, a 500-Hour Vision-Tactile Dataset for Robot Learning

On October 9, a consortium of 15 institutions, including Texas A&M University, Google DeepMind, Carnegie Mellon University, Stanford University, Microsoft, and NVIDIA, released TouchScale, a 500-hour human vision-tactile dataset built under unified sensor and collection protocols. The dataset comprises approximately 87,000 interaction recordings, over 1,500 objects, and about 2,000 task descriptions, collected by around 20 participants across nine environment categories.

Unlike previous efforts that simply merged smaller datasets, TouchScale fixed the hardware from the collection stage, using a unified wearable system to synchronously record head-mounted RGB-D, dual-wrist RGB, and full-hand tactile signals. Each tactile glove contains 880 taxels covering the fingers and palm, and a VLM-assisted automatic quality control pipeline was introduced to ensure data quality at scale.

The research team applied TouchScale for mid-training without using human action labels or mapping human hand trajectories to robots. On an xArm6 arm and BrainCo Revo 2 dexterous hand, they evaluated four contact-intensive tasks: soft/hard object sorting, bottle cap removal, test tube transfer, and whiteboard wiping. The average success rate improved from 22.5% to 57.5%, with gains across all four tasks. Additionally, scaling the training data from 10% to 100% raised cross-sensor tactile prediction cIoU from 0.311 to 0.383, demonstrating clear scaling benefits.

The study suggests that tactile data can be collected and learned at scale, much like egocentric video, offering a new path for robots to acquire transferable physical interaction experience.