Robotics & World Models Reading Club 33: Perceiving, Predicting, and Planning for Physical Interaction + AdaJEPA. SF 10/10

Name: Robotics & World Models Reading Club 33: Perceiving, Predicting, and Planning for Physical Interaction + AdaJEPA. SF 10/10
Start: 2026-10-10T14:00:00-07:00
End: 2026-10-10T17:00:00-07:00
Location: San Francisco, CA

📅 October 10, 2026

🕐 2:00 PM PT

📍 San Francisco

👥 81 attending

🎟️ $32.8 · Approval required

What we're seeing

Junfan Zhu tends to sell out · 3 of their last 5 events did
Approval required

About This Event

Robotics & World Models Reading Club 33: Perceiving, Predicting, and Planning for Physical Interaction + AdaJEPA. SF 10/10

A high-signal reading group for AI researchers & builders pushing the frontiers of robotic world models, WAMs, and embodied intelligence. In our previous sessions, we brought together researchers and engineers from Boston Dynamics, Google DeepMind, NVIDIA, Stanford, UC Berkeley, Physical Intelligence, Tesla, Generalist, Rhoda AI, and leading Bay Area robotics startups.

Hosted by Junfan Zhu & Aurora Feng.

Reading Club 33's Core Theme: Perceiving, Predicting, and Planning for Physical Interaction

Hongyu Li, Robotics Researcher at NVIDIA

Robots that manipulate the real world must perceive contact, predict physical change, and plan under uncertainty. This talk presents my work on multimodal perception and model-based planning for generalizable robot manipulation. I first present NovaFlow, which enables zero-shot manipulation by distilling actionable 3D flow from generated video, and NovaPlan, which closes the loop on long-horizon tasks through video-language planning. I then introduce Deform360, a large-scale multi-view visuotactile dataset for deformable world models, and use it to examine the trade-offs between 2D video models and explicit 3D particle dynamics. Finally, I present Hydra-0, a generalist world model conditioned on action flow, which represents robot actions as pixel motion. This shared visual interface lets the model learn action consequences across embodiments, tasks, and environments, and supports both policy evaluation and control. Together, this work argues for physically grounded robot intelligence that unites touch, vision, world models, and planning.

Hongyu Li is a robotics researcher at NVIDIA. His research focuses on co-designing multimodal perception and planning systems for robot manipulation, with an emphasis on learning models for environmental and object interaction. His work spans visuo-haptic perception, tactile sensing, world models, with applications to deformable object manipulation and zero-shot robot planning. During his Ph.D., he worked with Prof. George Konidaris at Brown University and Prof. Yunzhu Li at Columbia University. His research has appeared in venues including RSS, CoRL, ICRA, ECCV, IROS, and RA-L. Prior to joining NVIDIA, he interned at the Robotics and AI Institute, Amazon Robotics, and Honda Research Institute.

Keynote 2: AdaJEPA

Ying Wang, NYU

JEPA world models enable planning by predicting the consequences of actions in a learned representation space. We explore how to shape this space for effective planning and how to adapt the model when its predictions diverge from reality.

📐What is a good latent space for world modeling and planning? Inspired by the perceptual straightening hypothesis in human vision, we introduce temporal straightening to improve representation learning for latent planning. Temporal straightening encourages locally straight latent trajectories, making Euclidean distance of representations a better proxy for geodesic distance and gradient-based planning more stable and effective.

🤖How can world models adapt when their predictions become inaccurate, especially under test-time distribution shift? We introduce AdaJEPA, an adaptive WM that plans, acts, and adapts in a closed loop. Every action leads to a new observation, and every transition refines the latent representation and prediction.

Ying Wang is a PhD student at NYU, advised by Prof. Yann LeCun and Prof. Mengye Ren. Her research focuses on world models. Prior to starting her PhD, she earned an MS in Data Science at NYU and a BS in Computer Science, Statistics, and Finance at McGill University.

Keynote 3: Ben Zhou (Memories.ai)

We present a visual memory system for physical AI, built from two components: the Large Visual Memory Model (LVMM) and a video data lake. Together they enable robotic agents to self-improve through a data-centric loop.

LVMM is an omni perception model built specifically for human-centric video, covering both egocentric (first-person) and exocentric (third-person) footage. It takes video and text as input and produces a unified set of outputs from a single backbone: dense embeddings for indexing and retrieval, natural-language captions, head and hand pose, and 3D understanding of the scene and the person's interaction with it. LVMM also produces decision outputs, such as judging whether a person is performing a task correctly, which moves the model from describing activity to evaluating it.

The video data lake is infrastructure optimized end to end for video workloads, covering streaming ingestion, processing, storage, and retrieval. LVMM continuously indexes what flows into the lake, turning raw footage into structured, searchable, training-ready memory.

On top of this system, physical agents can improve themselves in a data-centric way. They can retrieve relevant human demonstrations, detect their own failures, and find the data that targets those failures. That data then feeds back into training, so each deployment cycle makes the agent better. This shortens the path from collecting video to improving robot behavior, and positions visual memory as core infrastructure for scaling embodied intelligence..

Lightning Talk 4: GPT-6 CUA → Robotics: The Next Real2Sim2Real?

Tonghe Zhang (CMU)

How CUA training in GPT-6 could directly reshape robot perception, planning, and decision-making—and unlock new possibilities for agentic Real2Sim2Real.

But agentic robotics has a catch: where does language-model agency actually help, and where does it break in the physical world? 👀

Location

San Francisco

Date & Time

Saturday, October 10, 2026 | 2:00 PM – 5:00 PM

Join Discord Community

https://discord.gg/WH7DrTHRXK

Follow Saturday Robotics on X

/saturdayrobotic

Agenda

2:00 PM – 2:30 PM Door Opens & Social

Food 😋, beverages🧋 and UNLIMITED strawberries 🍓 (our official reading club fruits ☺️😄).

2:30 PM – 3:30 PM Keynote by

YouTube Recording: TBD (We are looking for recording volunteers)

3:30 PM – 5:00 PM Q&A, open-floor roundtable (10–20 min per topic) on spotlight papers or any paper you’d like to highlight. Feel free to share why the paper matters and its technical details.

Past events

#iros-reading-club-31-0928: 🍾 IROS 2026 x Saturday Robotics — Robotics Research Night | Reading Club 30. Pittsburgh 9/28

Session 31 Luma: https://luma.com/tzbw7n61

#reading-club-29-0919: AgiBot × ManiFormer: Building the Next Generation of Physical AI. SF 9/19

Session 29 Luma: https://luma.com/sk70aihm

#reading-club-28-0912: Booster T2: The Next Frontier of Open Humanoid Robotics. SF 9/12

Session 28 Luma: https://luma.com/00z3oxw6

#reading-club-26-0905: Rethinking Robot Development: Co-Designing Morphology, Sensing, and Learning

Chenyang Ma (Applied Intuition, UNC)

Session 26 Luma: https://luma.com/jm9jceqr

#reading-club-25-0829: Sunnyvale 8/29

Session 25 Luma: https://luma.com/enowqo5v

#reading-club-24-0822: Contact-Rich Robot Learning from Human Videos and Tactile. SF 8/22

Kelin Yu (Maryland, Amazon FAR)

Session 24 Luma: https://luma.com/yl76re1b

#reading-club-23-0815: Engineering Robotic Simulators for Evaluation and Beyond. SF 8/15

Kaifeng Zhang (Columbia, World Labs)

Session 23 Luma: https://luma.com/non72bev

#reading-club-22-0808: ODEWorld: A Continuous Predictive Architecture via Physical-Time Flow. SF 8/8

Haoyi Niu (UC Berkeley)

Session 22 Luma: https://luma.com/7nnl4a5r
Reading Club 22 Review: https://x.com/junfanzhu98/status/2086482247344079100?s=20
Reading Club 22 Recap: https://x.com/junfanzhu98/status/2086484540013560280?s=20

#reading-club-21-0801: Vision-Language-Kinematics Supervision for Perception-Based Humanoid Loco-Manipulation — SF 8/1

Yen-Jen Wang (UC Berkeley, Amazon FAR)

Session 21 Luma: https://luma.com/4xkibxbh
Reading Club 21 Review: https://x.com/junfanzhu98/status/2083948325112533456?s=20
Reading Club 21 Recap: https://x.com/junfanzhu98/status/2083950393734975839?s=20

#reading-club-20-0725: Agentic Robotics Models (ENPIRE and Cap-X), Mountain View 07/25

Haoru Xue (UC Berkeley)

Kris Hauser (Samsung Research America, Robot Intelligence Lab)

Session 20 Luma: https://luma.com/5ltk12w5
Reading Club 20 Review: https://x.com/junfanzhu98/status/2081420671180284190?s=20
Reading Club 20 Recap: https://x.com/junfanzhu98/status/2081423890988044794?s=20

#SIGGRAPH-reading-club-19-0722: SIGGRAPH x Saturday Robotics — World Models for Robotics: Bridging Graphics, Simulation & Physical Intelligence | Reading Club 19, LA 07/22

Session 19 Luma: https://luma.com/yh2212ac
Reading Club 19 Review: https://x.com/junfanzhu98/status/2081923681627079055?s=20
Reading Club 19 Recap: https://x.com/junfanzhu98/status/2081924824709124174?s=20

#reading-club-18-0718: Causal World Models For Real-World Intelligence. SF 07/18

Guanming Wang & Bill (General Instinct, YC P26)

Feng Fan (UCSD, Aether AI)

Session 18 Luma: https://luma.com/f74eguvr
Reading Club 18 Review: https://x.com/junfanzhu98/status/2078738476602884502?s=20
Reading Club 18 Recap: https://x.com/junfanzhu98/status/2078747463058694625?s=20
Autonomous Panel Moderation: Rebuilding the Factory: Physical AI on the Production Line: https://x.com/junfanzhu98/status/2078385948887490814?s=20

#reading-club-17-0711: Soft Tactile-Centric Multimodal Intelligence Toward Safe and Dexterous Manipulation. SF 07/11

Quan Luu, Purdue.

Session 17 Luma: https://luma.com/e53zawq2
Reading Club 17 Review: https://x.com/junfanzhu98/status/2076205508646850819?s=20
Reading Club 17 Recap: https://x.com/junfanzhu98/status/2076207394426917187?s=20

#private-lunch-icml-0709: Saturday Robotics x ICML Private Lunch (Seoul)

Private Lunch: https://luma.com/khqg11gt

#reading-club-16-0704: The Embodied AI Hardware Stack — Supply Chain, Sensors, and the Data Flywheel — SF 07/04

Jerry Huang, Robotics Center of Silicon Valley.

Session 16 Luma: https://luma.com/cgzyfpeb
Reading Club 16 Review: https://x.com/junfanzhu98/status/2073666442512547957?s=20
Reading Club 16 Recap: https://x.com/junfanzhu98/status/2073667174678589620?s=20

#reading-club-15-0627: Scaling Touch: Flexible Tactile Skin for Dexterous Manipulation

Binghao Huang, Columbia, Amazon FAR.

Session 15 Luma: https://luma.com/7lts5ppf
Reading Club 15 Review: https://x.com/junfanzhu98/status/2071129728984273105?s=20
Reading Club 15 Recap: https://x.com/junfanzhu98/status/2071132114226139489?s=20

#deep-tech-week-14-0625: Deep Tech Week Research Night

SPEAR: A Simulator for Photorealistic Embodied AI Research. (ECCV 2026 accepted) by Mike Roberts, Senior Research Scientist, Adobe Research.

Bogdan Cristei, Venture Partner at SHACK15 Ventures.

Simone Totaro, CTO at Saturn Dynamics.

Shumo Chu, CEO at General Intelligence Labs.

Margaret Zhang, CEO at ThirdBrain Labs.

Session 14 Luma: https://luma.com/l1g9c2l1
Deep Tech Week 14 Review: https://x.com/junfanzhu98/status/2070417784560066592?s=20
Deep Tech Week 14 Recap: https://x.com/junfanzhu98/status/2070420270964514923?s=20

#reading-club-13-0620: HumanEgo: Train Robot Policy from 30 min Egocentric Videos — SF 0620

Session 13 Luma: https://luma.com/6vkhxnum
Reading Club 13 Review 1: HumanEgo: Zero-Shot Robot Learning, Human Egocentric Video, Leo Wang, Amazon FAR & UMaryland https://x.com/junfanzhu98/status/2068603103138713824?s=20
Reading Club 13 Recap 1: https://x.com/junfanzhu98/status/2068605511549743127?s=20
Reading Club 13 Review 2: Causal World Models: Biwei Huang, Aether AI & UCSD
https://x.com/junfanzhu98/status/2068743219685560453?s=20
Reading Club 13 Recap 2: https://x.com/junfanzhu98/status/2068746229643936177?s=20

#reading-club-12-0613: Origami Robotics (YC W26) on Dexterity

Session 12 Luma: https://luma.com/5w7c1t2a
Reading Club 12 Review: https://x.com/junfanzhu98/status/2066275974988337178?s=20
Reading Club 12 Recap: https://x.com/junfanzhu98/status/2066278639554245097?s=20

#cvpr-denver-11-0606: 🤖🥘 Saturday Robotics x Manycore Tech x Neural Motion | CVPR 2026 Denver Research Night | Robotics & World Models Reading Club 11

Junfan Zhu & Aurora Feng, Founders of Saturday Robotics

Anthony Zhao, Head of North America at Manycore Tech SpacialVerse

Aurora Feng, Founder at Neural Motion. NM-GenET.

Max Zhaoshuo Li, Robotics and World Model Tech Lead at NVIDIA Cosmos. Cosmos 3.

Xiaofan Li, World Model Tech Lead at X Square Robot. WALL-WM.

Zesen Zhao, University of Michigan. Test-Time Scaling for World Action Models via Zero-Shot Geometric Verification.

Pengyi Liao. VGGT-Ω: From 3D Reconstruction to Scalable Spatial Representation.

Jie Wang, University of Pennsylvania, GRASP Lab. Toward a Robotics MMLU: Lessons from Sim & Real Evaluations of Generalist Policies.

Gordon Qian, Senior AI Researcher at Snap. Diffusion-DRF: Free, Rich, and Differentiable Reward for Video Diffusion Fine-Tuning.

YouTube livestream: https://www.youtube.com/live/P_3gSC-5cYM?si=530zqf2NscCq643O
CVPR Denver Research Night Luma: https://luma.com/zamm9g2g
CVPR Denver Research Night Lightning Talks Review: https://x.com/junfanzhu98/status/2065102150418788581?s=20
CVPR Denver Research Night Lightning Talks Recap: https://x.com/junfanzhu98/status/2065104616497524843?s=20
CVPR Hot Takes Review: https://x.com/junfanzhu98/status/2065234892653547904?s=20
CVPR Hot Takes Recap: https://x.com/junfanzhu98/status/2065236892988416166?s=20

#reading-club-10-0530: Bringing Robots to Life — Learning Humanoid Instincts from the Body Up | San Francisco 0530

Haochen Shi (Stanford, co-advised by Karen Liu & Shuran Song)

Session 10 Luma: https://luma.com/czz76qe1
Reading Club 10 Recap: https://x.com/junfanzhu98/status/2061145697693683878?s=20

#private-dinner-01-0529: Robo Plov x Saturday Robotics

Private Dinner 01 Luma: https://luma.com/3rzqwond
Private Dinner 01 Review: https://www.linkedin.com/posts/junfan-zhu_saturday-robotics-was-excited-to-host-its-activity-7466384182190182400-Zgnv?utm_source=share&utm_medium=member_desktop&rcm=ACoAABxP-p0BpUNGDf347aKh_1uJAPzG4er0As8

#reading-club-09-0523: CVPR Warm-up & Founders Spotlight — DeltaWorld + VisuoTactile Dexterous Hands

Tommie Kerssies (Amazon Frontier AI & Robotics)

Arjun Subramaniam (Factory Intelligence)

Session 09 Luma: https://luma.com/wooiz0bf
Reading Club 09 Review: Part 1: Robotics & World Model Reading Club 9.1: CVPR Warm-up— A Frame is Worth 1 Token: DeltaToken. https://x.com/junfanzhu98/status/2058449627184267621?s=20
Reading Club 09 Review: Part 2: Robotics & World Model Reading Club 9.2: Tactile Sensor & Reliable Manipulation in Production. https://x.com/junfanzhu98/status/2058461947637694948?s=20

#reading-club-08-0516: Embodied Human Data as the “Internet of Motion and Behavior”

Ryan Punamiya (NVIDIA Gear, Georgia Tech)