Related work

The foundational work on continual learning, 1959 to 2026: methods, theory, benchmarks, surveys, and the neuroscience of memory. Updated weekly; some carry our notes.

15 papers of 11,817Sort Recent · Most cited
  1. 2025
    Revisiting Replay and Gradient Alignment for Continual Pre-Training of Large Language ModelsIstabrak Abbes, G. Subbaraj, Matthew Riemer … Irina RisharXiv
    PDF ↗
  2. 2024
    Simple and Scalable Strategies to Continually Pre-train Large Language ModelsAdam Ibrahim, Benjamin Th'erien, Kshitij Gupta … Irina RishTrans. Mach. Learn. Res.
    PDF ↗
  3. 2022
    Broken Neural Scaling LawsEthan Caballero, Kshitij Gupta, Irina Rish, David KruegerICLR
    PDF ↗
  4. 2022
    Challenging Common Assumptions about Catastrophic Forgetting and Knowledge AccumulationTimothée Lesort, Оleksiy Ostapenko, Diganta Misra … Irina RishCoLLAs
    PDF ↗
  5. 2022
    Continual Learning with Foundation Models: An Empirical Study of Latent ReplayОleksiy Ostapenko, Timothée Lesort, Pau Rodríguez … Laurent CharlinCoLLAs
    PDF ↗
  6. 2022
    Foundational Models for Continual Learning: An Empirical Study of Latent ReplayOleksiy Ostapenko, Timothée Lesort, P. Rodríguez … Laurent CharlinarXiv
  7. 2021
    Sequoia: A Software Framework to Unify Continual Learning ResearchFabrice Normandin, Florian Golemo, Oleksiy Ostapenko … M. CacciaarXiv
    PDF ↗
  8. 2021
    Continual Learning in Deep Networks: an Analysis of the Last LayerTimothée Lesort, Thomas George, Irina RisharXiv
    PDF ↗
  9. 2021PDF ↗
  10. 2020
    Unified Models of Human Behavioral Agents in Bandits, Contextual Bandits and RLBaihan Lin, Guillermo Cecchi, Djallel Bouneffouf … Irina RisharXiv
    PDF ↗
  11. 2019
    A Story of Two Streams: Reinforcement Learning Models from Human Behavior and NeuropsychiatryBaihan Lin, Guillermo Cecchi, Djallel Bouneffouf … Irina RishAdaptive Agents and Multi-Agents Systems · Columbia University · IBM (United States) · +1
    PDF ↗
  12. 2020
    Towards Lifelong Self-Supervision For Unpaired Image-to-Image TranslationVictor Schmidt, Makesh Narsimhan Sreedhar, Mostafa ElAraby, Irina RisharXiv
    PDF ↗
  13. 2020
    Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual LearningM. Caccia, Pau Rodríguez, Оleksiy Ostapenko … Laurent CharlinNeurIPS
    PDF ↗
  14. 2019
    Continual Learning with Self-Organizing MapsPouya Bashivan, Martin Schrimpf, Robert Ajemian … Yuhai TuarXiv · Massachusetts Institute of Technology
    PDF ↗
  15. 2018
    Learning to Learn without Forgetting By Maximizing Transfer and Minimizing InterferenceMatthew Riemer, Ignacio Cases, Robert Ajemian … Gerald TesauroICLR · IBM (United States) · Stanford University · +2
    PDF ↗
About this index

We keep this list because we read the field and wanted one place to see it. It covers work on continual learning itself, in the core areas of machine learning, and leaves out papers that apply it inside another field, such as medical imaging or fault diagnosis. It is seeded from the community lists kept by ContinualAI and by Xialei Liu, then filled out from OpenAlex, and every week a script looks for new papers on OpenAlex and arXiv. A model reads each candidate and decides whether it belongs; a person reviews the additions before they go live. Authors and affiliations come from OpenAlex, so a recent preprint can lack its institutions for a week or two.

Missing something, or filed under the wrong venue? Write to hello@unify.ai with the arXiv id or DOI.