Related work

The foundational work on continual learning, 1959 to 2026: methods, theory, benchmarks, surveys, and the neuroscience of memory. Updated weekly; some carry our notes.

36 papers of 11,817Sort Recent · Most cited
  1. 2026
    To Retain or to Adapt? Generalizing Continual LearningGiulia Lanzillotta, Mandana Samiei, D. Precup … Claire VernadearXiv
    PDF ↗
  2. 2025
    What Can Grokking Teach Us About Learning Under Nonstationarity?Clare Lyle, Gharda Sokar, Razvan Pascanu, Andr'as GyorgyarXiv
    PDF ↗
  3. 2024
    Non-Stationary Learning of Neural Networks with Automatic Soft Parameter ResetAlexandre Galashov, Michalis K. Titsias, Andr'as Gyorgy … M. SahaniNeurIPS
    PDF ↗
  4. 2024
    Normalization and effective learning rates in reinforcement learningClare Lyle, Zeyu Zheng, Khimya Khetarpal … Will DabneyNeurIPS
    PDF ↗
  5. 2024
    No Representation, No Trust: Connecting Representation, Collapse, and Trust Issues in PPOSkander Moalla, A. Miele, Razvan Pascanu, Caglar GulcehreNeurIPS
    PDF ↗
  6. 2024
    Disentangling the Causes of Plasticity Loss in Neural NetworksClare Lyle, Zeyu Zheng, Khimya Khetarpal … Will DabneyCoLLAs
    PDF ↗
  7. 2023
    Continual Learning: Applications and the Road ForwardEli Verwimp, S. Ben-David, Matthias Bethge … Gido M. van de VenTrans. Mach. Learn. Res.
    PDF ↗
  8. 2023
    Power Norm Based Lifelong Learning for Paraphrase GenerationsThalaiyasingam Ajanthan, Puneet Kumar, J. Kirkpatrick … Andrei A. RusuSIGIR
  9. 2023
    Towards Robust and Efficient Continual Language LearningAdam Fisch, Amal Rannen-Triki, Razvan Pascanu … M. RanzatoarXiv
    PDF ↗
  10. 2023
    Learning to Modulate pre-trained Models in RLThomas Schmied, M. Hofmarcher, F. Paischer … Sepp HochreiterNeurIPS
    PDF ↗
  11. 2023
    Kalman Filter for Online Classification of Non-Stationary DataMichalis K. Titsias, Alexandre Galashov, Amal Rannen-Triki … J. BornscheinICLR
    PDF ↗
  12. 2023
    The Tunnel Effect: Building Data Representations in Deep Neural NetworksWojciech Masarczyk, M. Ostaszewski, Ehsan Imani … Tomasz Trzci'nskiNeurIPS
    PDF ↗
  13. 2023
    Deep Reinforcement Learning with Plasticity InjectionEvgenii Nikishin, Junhyuk Oh, Georg Ostrovski … André BarretoNeurIPS
    PDF ↗
  14. 2023
    Towards Compute-Optimal Transfer LearningMassimo Caccia, Alexandre Galashov, Arthur Douillard … Razvan PascanuarXiv
    PDF ↗
  15. 2023
    Understanding plasticity in neural networksClare Lyle, Zeyu Zheng, Evgenii Nikishin … Will DabneyICML
    PDF ↗
  16. 2023
    Continually learning representations at scaleAlexandre Galashov, Jovana Mitrovic, Dhruva Tirumala … Razvan PascanuCoLLAs
  17. 2022
    NEVIS'22: A Stream of 100 Tasks Sampled from 30 Years of Computer Vision ResearchJörg Bornschein, Alexandre Galashov, Ross Hemsley … Marc’Aurelio RanzatoarXiv
    PDF ↗
  18. 2022
    Disentangling Transfer in Continual Reinforcement LearningMaciej Wołczyk, Michał Zając, Razvan Pascanu … Piotr MiłośNeurIPS
    PDF ↗
  19. 2022
    Architecture Matters in Continual LearningSeyed Iman Mirzadeh, Arslan Chaudhry, Dong Yin … Mehrdad FarajtabararXiv
    PDF ↗
  20. 2021
    Wide Neural Networks Forget Less CatastrophicallySeyed Iman Mirzadeh, Arslan Chaudhry, Yin, Dong … Mehrdad FarajtabarICML · Google (United States)
    PDF ↗
  21. 2021
    Powerpropagation: A sparsity inducing weight reparameterisationJonathan Schwarz, Siddhant M. Jayakumar, Razvan Pascanu … Yee Whye TehNeurIPS · Google (United States)
    PDF ↗
  22. 2021
    Task-agnostic Continual Learning with Hybrid Probabilistic ModelsPolina Kirichenko, Mehrdad Farajtabar, Dushyant Rao … Razvan PascanuarXiv
    PDF ↗
  23. 2021
    A study on the plasticity of neural networksTudor Berariu, Wojciech Marian Czarnecki, Soham De … Claudia ClopatharXiv · Imperial College London
    PDF ↗
  24. 2021
    Continual World: A Robotic Benchmark For Continual Reinforcement LearningMaciej Wołczyk, Michał Zając, Razvan Pascanu … Piotr MiłośNeurIPS · Jagiellonian University · Google (United States)
    PDF ↗
  25. 2020
    Embracing Change: Continual Learning in Deep Neural Networks.Raia Hadsell, Dushyant Rao, Andrei A. Rusu, Razvan PascanuTrends in Cognitive Sciences · Google DeepMind (United Kingdom) · Google (United Kingdom)
  26. 2020
    Linear Mode Connectivity in Multitask and Continual LearningSeyed Iman Mirzadeh, Mehrdad Farajtabar, Dilan Görür … Hassan GhasemzadehICLR · Washington State University · Google (United States)
    PDF ↗
  27. 2020
    Understanding the Role of Training Regimes in Continual LearningSeyed Iman Mirzadeh, Mehrdad Farajtabar, Razvan Pascanu, Hassan GhasemzadehNeurIPS
    PDF ↗
  28. 2019
    Continual Unsupervised Representation LearningDushyant Rao, Francesco Visin, Andrei Rusu … Raia HadsellNeurIPS · Carnegie Mellon University · Google (United States) · +2
    PDF ↗
  29. 2019
    Scalable and Order-robust Continual Learning with Hierarchically Decomposed Networks.J. Kirkpatrick, Razvan Pascanu, Neil C. Rabinowitz … Guillaume DesjardinsPreprint
  30. 2019
    Task Agnostic Continual Learning via Meta LearningXu He, Jakub Sygnowski, Alexandre Galashov … Razvan PascanuarXiv · Google (United States)
    PDF ↗
  31. 2019
    Functional Regularisation for Continual Learning using Gaussian ProcessesMichalis K. Titsias, Jonathan Schwarz, Alexander Matthews … Yee Whye TehICLR · Google (United States)
    PDF ↗
  32. 2019
    Functional Regularisation for Continual LearningMichalis K. Titsias, Jonathan Schwarz, A. G. Matthews … Y. TehPreprint
  33. 2018
    Progress & Compress : A scalable framework for continual learningJonathan Schwarz, Jelena Luketina, Wojciech Marian Czarnecki … Raia HadsellICML
    PDF ↗
  34. 2018
    Memory-based Parameter AdaptationPablo Sprechmann, Siddhant M. Jayakumar, Jack W. Rae … Charles BlundellICLR · Google (United States)
    PDF ↗
  35. 2016
    Overcoming catastrophic forgetting in neural networksJames Kirkpatrick, Razvan Pascanu, Neil C. Rabinowitz … Raia HadsellPNAS · Google DeepMind (United Kingdom) · Imperial College London
    PDF ↗
  36. 2016
    Progressive Neural NetworksAndrei A. Rusu, Neil C. Rabinowitz, Guillaume Desjardins … Raia HadsellarXiv
    PDF ↗
About this index

We keep this list because we read the field and wanted one place to see it. It covers work on continual learning itself, in the core areas of machine learning, and leaves out papers that apply it inside another field, such as medical imaging or fault diagnosis. It is seeded from the community lists kept by ContinualAI and by Xialei Liu, then filled out from OpenAlex, and every week a script looks for new papers on OpenAlex and arXiv. A model reads each candidate and decides whether it belongs; a person reviews the additions before they go live. Authors and affiliations come from OpenAlex, so a recent preprint can lack its institutions for a week or two.

Missing something, or filed under the wrong venue? Write to hello@unify.ai with the arXiv id or DOI.