To Retain or to Adapt? Generalizing Continual Learning

Giulia Lanzillotta*, Mandana Samiei*, Doina Precup, Razvan Pascanu^†, and Claire Vernade^†

Continual learning has traditionally focused on mitigating catastrophic forgetting by approximating the joint-task learning solution. We challenge this retention-centered perspective, showing that preserving past knowledge is not always optimal in non-stationary environments. We introduce a theoretical framework based on Average Lifelong Error and Transfer Efficiency, characterizing when historical knowledge helps adaptation and when it becomes a liability. This perspective leads to a broader family of Predictive Continual Learning algorithms that explicitly optimize future performance under changing task distributions.

Under review at TMLR

Human Adults and LLMs as Scientists: Who Benefits from Active Exploration

Mandana Samiei*, Eunice Yiu*, Anthony GX-Chen, Dongyan Lin, Jocelyn Shen, Blake A. Richards, Alison Gopnik, and Doina Precup

Active experimentation is a hallmark of scientific reasoning, yet it remains unclear when active exploration provides a measurable advantage over passive observation. We compare human adults and large language models on a causal discovery task, allowing both to actively design interventions or passively observe evidence.Our results show that active exploration substantially improves learning for conjunctive causal structures, where informative interventions are difficult to identify, while offering little benefit for simpler disjunctive rules. Although state-of-the-art LLMs can achieve human-level accuracy, they remain considerably less efficient in selecting informative experiments, highlighting important differences between successful reasoning and effective scientific exploration..

Accepted at the 48th Annual Conference of the Cognitive Science Society (CogSci 2026)

Learning Schemas in Reinforcement Learning: bottleneck structure discovery.

Mandana Samiei, Doina Precup and Blake A. Richards

In this work we show how schemas can be learned in RL by discovering the bottleneck structure of the task.

Under submission Nature communications.

The Schema Spectrum: Explicit, Implicit, and Emergent Structures in AI and the Brain.

Mandana Samiei, Doina Precup and Blake A. Richards

This perspective uses recent results in generative AI models to reconsider two standard assumptions in schema theory: (1) that schemas exist as explicit objects in the brain, and (2) that schemas are categorically distinct from episodic memories. This is based on the observation that large language models exhibit many phenomena reminiscent of schematic learning in the absence of any explicit engineering as such, suggesting that schemas may be an emergent property of distributed representations in neural networks.

Accepted at Neuron 2026.

Language Agents Mirror Human Causal Reasoning Biases. How Can We Help Them Think Like Scientists?

Language model (LM) agents exhibit human-like biases when causally exploring. We compare this to human data. We also develop a scalable test-time sampling algorithm to fix this, by sampling hypotheses as code and acting to eliminate them.

Anthony GX-Chen, Dongyan Lin*, Mandana Samiei*, Doina Precup, Blake Richards, Rob Fergus, Kenneth Marino

*equal contribution, ordered alphabetically. Presenters are shown with underline.

Presented at Conference on Language Modelling (COLM) 2025.

The Role of Schemas in Reinforcement Learning: Insights and Implications for Generalization

Mandana Samiei, Doina Precup and Blake A. Richards

In cognitive psychology, schemas are considered to be "building blocks" of cognition, shaping how people view the world and interact with it. The goal of this paper is to propose a method for learning schemas in RL. We argue that, by representing tasks through schemas, agents can more effectively generalize from past experiences and adapt to new, unseen environments with minimal data.

Presented at Reinforcement Learning and Decision Making - RLDM 2025 .

Towards Efficient Generalization in Continual RL using Episodic Memory

As part of a collaboration between Microsoft Research Research and Mila, work done with Ida Momennejad, Geoffrey J. Gordon, John Langford, Mehdi Fatemi, Blake A. Richards, and Guillaume Lajoie. Gave an invited talk on Towards Efficient Generalization in Continual RL using Episodic Memory at Microsoft Research Summit 2021. Slides.

Notes from CogSci 2026

During CogSci 2026 I started keeping detailed notes on talks, posters, and conversations that I found particularly interesting. Inspired by Dave Abel's habit of sharing conference notes, I decided to collect my own notes as a way to better remember ideas, connect themes across talks, and hopefully make them useful to others as well.