World Models Reading Group

The World Models Reading Group meets biweekly at Mila to discuss recent work on world models.

When
Thursday, 3:00–4:00 pm (ET)
Where
Mila, Auditorium 2 — and on Zoom
Updates
Join the mailing list or subscribe by email (no Google account needed) At Mila? Follow #world-models on Slack

Schedule

Upcoming

  • Thu, Nov 12, 2026 3:00 pm ET Next
    RoboJEPA: Scaling Robotic Latent World Models Artem Zholus

    Abstract

    Latent world models have shown a remarkable ability to predict future states and to plan in the real world. In practice, however, we lack a principled way to estimate how their capabilities scale with model size, data, and compute, an open problem that slows progress in the field. In this work we present RoboJEPA, a world model based on the Joint Embedding Predictive Architecture (JEPA) and trained on a large-scale dataset spanning 12 robotic embodiments. We show that RoboJEPA’s imagination error, the error of its latent rollouts, follows a second-order power law in compute, allowing us to predict model quality well beyond the scale at which the law is fit. We further show that downstream robotic planning performance improves predictably with compute, and that imagination error is strongly correlated with it, making it a reliable proxy for real-robot evaluation. Finally, we demonstrate that latent world models can be deployed zero-shot as robotic agents, planning toward a single goal image to solve tasks requiring long-horizon planning on real hardware. We release all model checkpoints together with our training and robot deployment code. To our knowledge, this is the first work to establish scaling laws for multi-embodiment robotic world models trained on real robot data, and RoboJEPA, at 8B parameters, is the largest JEPA predictor model trained to date.

    Bio

    Artem Zholus is a last-year PhD student at Mila supervised by Prof. Sarath Chandar and a Member of Technical staff at AMI Labs. His research covers various aspects of world models such as memory, zero-shot real world generalization and multiembodiment capabilities.
  • Thu, Nov 26, 2026 3:00 pm ET
    Title TBD Sherry Yang

Past

  • Thu, Oct 29, 2026 3:00 pm ET
    When Does LeJEPA Learn a World Model? David Klindt

    Abstract

    A representation that scrambles the true degrees of freedom of the world cannot support reliable planning or compositional generalization. We prove that LeJEPA (alignment plus Gaussian regularization) linearly recovers the world’s latent variables from nonlinear observations, a property known as linear identifiability, in a broad class of worlds where latents evolve under stationary, additive-noise transitions. Our main result is that among all such worlds, the Gaussian is the unique latent distribution for which this guarantee holds. The forward direction rests on a spectral decomposition in which each degree of nonlinearity is strictly penalized by alignment, making the linear map the optimum; the converse rules out every non-Gaussian alternative. We further prove an approximate identifiability result where the guarantee degrades gracefully, and show that linear, orthogonal identifiability enables optimal latent-space planning. We validate the theory with experiments ranging from 2D examples to 1024-dimensional latents, including distributional ablations and pixel-based robotic control. Our theory turns an empirically successful recipe into a mathematical guarantee, providing the foundation for building World Models that provably recover the structure of the world.

    Bio

    David Klindt is an Assistant Professor at Cold Spring Harbor Laboratory whose lab studies intelligence in brains and machines. A central idea in his work is treating artificial neural networks as a new model organism: systems that can be probed, perturbed, and fully instrumented in ways biological brains don’t allow, to develop and test theories of representation and learning that transfer back to neuroscience. His research spans four connected threads: identifiability theory (when learned representations reflect real, recoverable structure rather than arbitrary training artifacts), world models (how systems build internal models of their environment), mechanistic interpretability (understanding what is actually encoded inside trained networks), and computational neuroscience (particularly how the visual cortex represents and processes sensory information).

    Recording

  • Thu, Oct 15, 2026 3:00 pm ET
    Hallucination in World Models is Predictable and Preventable Nicklas Hansen

    Abstract

    Generative world models now render strikingly realistic, action-controllable futures, yet they frequently hallucinate: rollouts remain visually fluent while drifting from the true dynamics. This raises two questions: can we tell when a world model is hallucinating, and what can we do about it? In this talk, I will address both, and argue that hallucination is, first and foremost, a data coverage problem. I will first introduce MMBench2, a 427-hour, 210-task dataset for visual world modeling with ground-truth actions, rewards, and live simulators, along with a 350M-parameter world model trained on this dataset. Using this testbed, I will show that hallucination is systematic, that each type of hallucination ties to specific model components, and that simple uncertainty metrics computed from the model’s own predictions reveal where it will fail at runtime. I will then show that the same metrics can be used for mitigation: coverage-aware sampling during training, and curiosity-driven data collection. I will conclude with open questions, including how these findings carry over to real-world data.

    Bio

    Nicklas Hansen is a postdoctoral researcher at NYU Courant working with Lerrel Pinto and Yann LeCun. Nick’s research focuses on developing generalist AI agents that learn from interaction. They recently completed their PhD at UC San Diego advised by Xiaolong Wang and Hao Su, and have also spent time at NVIDIA Research, Meta AI (FAIR), and UC Berkeley.

    Recording

  • Thu, Oct 1, 2026 3:00 pm ET
    Towards World Models via Joint-Embedding Predictive Architectures Michael Rabbat

    Abstract

    This talk provides an overview and motivation for why world models are a necessary component of future AI systems. The term “world model” is overloaded in the broader discourse. For the purposes of this talk I’ll argue that a world model should: (a) construct an agent’s internal representation of the world through perception, and (b) should be able to predict the evolution of this representation through time, including in response to the agent’s actions. Such a world model enables planning. Two main research questions are then: how to learn such a world model from data? and how to use it for planning and control? I’ll give an overview of our work on Joint-Embedding Predictive Architectures (JEPAs) and will discuss interesting directions for future work.

    Bio

    Michael Rabbat is Co-founder and VP World Models at Advanced Machine Intelligence (AMI Labs), where he is building a new breed of AI systems focused on understanding and reasoning about the world. He is also an Associate Industry Member at Mila. Previously, Michael was a Research Director at Meta, where he was a founding member of FAIR Montréal, contributing to the company’s fundamental AI research efforts. Prior to that, he spent over a decade as a professor at McGill University in Electrical and Computer Engineering. He earned his PhD from the University of Wisconsin, Madison. Throughout his career he has worked on a variety of topics across signal processing, learning, and optimization, including distributed optimization and federated learning. His current research is focused on methods for training world models and using them for planning and control.

    Recording(Mila login required)

Organizers

Sponsors

Get in touch

Join the mailing list to get an email about each upcoming talk. To propose a paper or sign up to present, get in touch.

Join the mailing list Get in touch

No Google account? Send any email to mila-world-models-rg+subscribe@googlegroups.com, then reply to the confirmation email you receive (its "Join" button needs a Google account; replying doesn't).