# Sundae Robotics 11: Beyond the Gripper — Teaching Robots Dexterity Through Touch, Vision, and Play · Sun, Nov 22, 2026 · San Francisco · Free Source: https://events.trymimetic.com/events/sf/sundae-robotics-11-beyond-the-gripper-teaching-robots-dexterity-through-touch High Signal Event # Sundae Robotics 11: Beyond the Gripper — Teaching Robots Dexterity Through Touch, Vision, and Play 📅 November 22, 2026 🕐 2:00 PM PT 📍 San Francisco 👥 64 attending 🎟️ Free · Approval required ## What we're seeing - **Approval required · venue shared on acceptance** ## About This Event 🤖🍨 Grab a sundae and join Sundae Robotics, a private, invite-only Sunday series bringing together robotics researchers, founders, and builders working at the frontier of physical intelligence. **Sundae Robotics 12** Dexterous Manipulation, Tactile Perception & Robot Learning **Featured Talk: Beyond the Gripper — Teaching Robots Dexterity Through Touch, Vision, and Play** **Beyond the Gripper: Teaching Robots Dexterity Through Touch, Vision, and Play** Keynote: **[Akash Sharma](https://akashsharma.me/?utm_source=luma)** Research Scientist, Amazon Frontier AI & Robotics · CMU Robotics Institute PhD Robots that impress in demos still mostly use parallel-jaw grippers, and true in-hand dexterity remains out of reach. The two standard routes each face fundamental limitations: demonstrations often force humans to act like robots, while simulation cannot faithfully model touch or real-world vision. In this talk, Akash will present four ingredients that aim to close those gaps. First, he will introduce **PTLD**, which learns tactile policies without ever simulating a tactile sensor by distilling a privileged simulation-trained teacher on real hardware. He will then discuss **INVAR**, which learns visual invariance once from clean and randomized simulation pairs, allowing policies trained on clean simulation to transfer zero-shot. Next, Akash will present **CARD**, which retargets human video, glove, and motion-capture demonstrations according to what the hand touches rather than simply how the hand moves. Finally, he will discuss **DSP**, which turns minutes of human play into goal-conditioned dexterity through autonomous play, world models, and distillation. Together, these works point toward a common recipe for scaling dexterous manipulation: let humans be humans, use a privileged teacher, scale with the robot's own experience, and distill what is learned into a deployable policy. Akash Sharma is a **Research Scientist at Amazon Frontier AI & Robotics (FAR)**, where he works with **Jitendra Malik** on robot hands and dexterous manipulation. He received his PhD from the **Carnegie Mellon University Robotics Institute**, where he was advised by Prof. **Michael Kaess**. During his PhD, he was also a visiting researcher with the **FAIR Robotics team at Meta**, where he worked closely with **Mustafa Mukadam** and **Francois Hogan**. Akash's research focuses on robot perception using varied sensing modalities, particularly **vision and touch**, to enable dexterous manipulation. He is also broadly interested in state estimation and reconstruction in dynamic environments. His recent work explores how tactile sensing, simulation, human demonstrations, and large-scale robot experience can be combined to overcome limitations of conventional dexterous learning pipelines. His work on **PTLD** develops a sim-to-real approach for tactile dexterity that avoids tactile simulation by using privileged real-world sensing to distill information into deployable tactile policies. Akash has also worked extensively on general-purpose tactile representations. **Sparsh**, **Sparsh-skin**, and **Sparsh-X** explore self-supervised representations across vision-based tactile sensors, distributed tactile skin, and multimodal touch, while **DexterityGen** investigates combining reinforcement-learned dexterous primitives with human guidance for complex manipulation. His broader research spans multi-fingered manipulation, human-to-robot skill transfer, tactile perception, sim-to-real learning, object-centric manipulation, state estimation, and scalable data generation for robot learning. Before his PhD, Akash received a Master's in Robotics from **Carnegie Mellon University**, where he worked on object-based SLAM for robot navigation. He received his bachelor's degree in Electronics and Communication Engineering from **Sri Jayachamarajendra College of Engineering**. **Topics** • Moving beyond parallel-jaw grippers toward true in-hand dexterity • Vision and touch for multi-fingered robot manipulation • Why conventional human demonstrations struggle to scale to dexterous hands • Why tactile simulation remains a bottleneck for sim-to-real learning • PTLD and privileged tactile latent distillation • Learning tactile policies without simulating tactile sensors • Using privileged sensing on real hardware to supervise deployable policies • Sim-to-real transfer for tactile manipulation • INVAR and learning visual invariance for zero-shot transfer • Separating visual robustness learning from downstream policy learning • Training policies in clean simulation while remaining robust to real-world vision • CARD and contact-aware human-to-robot retargeting • Retargeting demonstrations based on interaction rather than pose matching • Learning from human video, gloves, and motion capture • Preserving task-relevant contacts across human and robot embodiments • DSP and learning dexterity from human play • Autonomous robot play as a scalable source of experience • World models for goal-conditioned dexterous manipulation • Distilling large-scale robot experience into deployable policies • Combining human priors, privileged teachers, and autonomous learning **Open Discussion + Q&A** • Why has multi-fingered dexterity remained so much harder to scale than parallel-jaw manipulation? • Which aspects of touch are most important for in-hand manipulation: contact location, force, slip, pressure, or something else? • Can we obtain the benefits of tactile simulation without actually building an accurate tactile simulator? • What information should a privileged teacher have access to that the deployed robot should not? • How much real-world tactile data is needed to successfully distill privileged information? • Should visual domain randomization happen during every policy training run, or can invariance be learned once and reused? • What kinds of visual invariances transfer across different manipulation tasks? • Why is matching human and robot poses often the wrong objective for retargeting dexterous demonstrations? • Is preserving object contact more important than preserving human hand trajectories? • How can demonstrations from video, gloves, and motion capture be unified despite providing very different kinds of information? • What can humans demonstrate naturally that is difficult to express through robot teleoperation? • How far can a few minutes of human play be amplified through autonomous robot experience? • What role should world models play in scaling goal-conditioned dexterous behavior? • When is distillation preferable to deploying a larger teacher policy directly? • Could autonomous play eventually generate most of the data needed for dexterous robot learning? • What combination of vision, touch, demonstrations, simulation, and autonomous experience is most likely to unlock general-purpose robot dexterity? ## Hosted by [Edmond](https://luma.com/user/Edmond_) [James (Jingxi) Xu](https://luma.com/user/jamesxu) [haseab 🌵](https://luma.com/user/haseab) [Founders, Inc.](https://luma.com/user/finc) ## Location 📍 San Francisco, CA AI Founders Talk Research [Register for This Event](https://lu.ma/nsqliiuv?utm_source=trymimetic&utm_medium=calendar&utm_campaign=ai_events) [7 more SF Robotics events this week](/events/robotics/sf) Free Growth Analysis ## Get a free growth analysis for your company See how your website, messaging, and go-to-market strategy stack up, in minutes. [Get My Free Analysis](/?utm_source=events&utm_medium=referral&utm_campaign=robotics&utm_content=event_detail_cta) ## More SF events like this ### [Robotics Founders Evening: Founders on Scale, Hardware and Shenzhen](/events/sf/sf-to-shenzhen-robotics-founders-evening-mission-robotics-nov-2026) Tomorrow · 5:00 PM PT Mission Robotics, 3001 19th St, San Francisco, CA 94110, USA ### [Build with Flux: Design Electronics and Enclosures with AI](/events/sf/build-night-build-hardware-with-flux-nov-2026) Tomorrow · 6:30 PM PT San Francisco, CA 125 attending ### [Tech Makers Goes West @Mission Robotics: Chargebotic + Pitch Night](/events/sf/tech-makers-goes-west-mission-robotics-chargebotic-pitch-night-nov-2026) Fri, Nov 13 · 5:30 PM PT Mission Robotics, 3001 19th St, San Francisco, CA 94110, USA ### [Robotics & World Models Reading Club 39: SF 11/14.](/events/sf/robotics-world-models-reading-club-39-sf-1114-nov-2026) Sat, Nov 14 · 2:00 PM PT San Francisco, CA [← Back to San Francisco Events](/events/robotics/sf)