Scaling Robotic Manipulation via Structured World Models and Tactile Sensing

Explore advanced approaches to scaling robotic manipulation through structured world models and tactile sensing in this comprehensive conference talk. Discover how particle- and graph-based neural dynamics models, inspired by human intuitive physics, enable model-based planning and control across diverse object types including rigid, deformable, articulated, and granular materials. Learn about the development of neural and physics-informed digital twins that facilitate scalable data generation, policy iteration, and evaluation for long-horizon, contact-rich manipulation tasks. Examine cutting-edge research in tactile sensing, from understanding human grasping principles using dense tactile gloves to creating flexible, low-cost tactile arrays and portable visuo-tactile grippers. Understand how these tactile systems, combined with simulation and large-scale real-world data collection, enable robust learning and improved sim-to-real transfer for challenging scenarios involving visual occlusion, fragile objects, and complex physical interactions. Gain insights into the key ingredients necessary for scaling robotic manipulation toward greater generality, robustness, and physical competence, and learn how these advances contribute to the development of physically grounded foundational robotic models.