NVIDIA Launches Cosmos3-H-Dreams, a Generative Surgical Simulator Trained on Clinical Cholecystectomy Video

On October 21, 2026, NVIDIA released Cosmos3-H-Dreams, a generative surgical simulator it describes as the first trained on clinical cholecystectomy video and capable of real-time kinematic control. Powered by Cosmos 3 and NVIDIA's FlashDreams accelerated streaming-inference library, the system runs interactively on a single NVIDIA RTX PRO 6000 GPU.

Unlike the earlier Cosmos-H-Dreams, which was built on Cosmos-Predict2.5 for dVRK tabletop suturing, Cosmos3-H-Dreams learns visual dynamics directly from synchronized video and robot kinematics recorded during clinical procedures on the CMR Surgical Versius platform, rather than explicitly modeling geometry, tissue deformation, or physical interactions. The model is adapted from Cosmos3-H-Embodied-Nano, a 16-billion-parameter model pretrained on Open-H-Embodiment and fine-tuned on curated cholecystectomy video and kinematics. It uses 28 action dimensions, with training data at 832x480 resolution and 10 Hz action alignment.

For long interactive rollouts, NVIDIA uses a teacher-to-student distillation pipeline that produces four-step and two-step denoising checkpoints. The integration with FlashDreams accelerates streaming inference, while an adapted FadeMem module consolidates the growing key/value cache during long generations. NVIDIA's demonstrations show the model generating different continuations of the same surgical scene from a shared initial frame, conditioned on different instrument actions recorded with a VR headset.

NVIDIA has released the model, code, and a recipe for adapting the simulator to other datasets and controllers. The company notes that Cosmos3-H-Dreams is a research simulator and not a validated system for patient care.