Related work

The foundational work on continual learning, 1959 to 2026: methods, theory, benchmarks, surveys, and the neuroscience of memory. Updated weekly; some carry our notes.

35 papers of 11,817Sort Recent · Most cited
  1. 2022
    TAME: Task Agnostic Continual Learning using Multiple ExpertsHaoran Zhu, Maryam Majzoubi, Arihant Jain, Anna ChoromanskaCVPR · New York University · Google (United States)
    PDF ↗
  2. 2021
    Learning to Prompt for Continual LearningZifeng Wang, Zizhao Zhang, Chen‐Yu Lee … Tomas PfisterCVPR · Universidad del Noreste · Google (United States)
    PDF ↗
  3. 2020
    CVPR 2020 Continual Learning in Computer Vision Competition: Approaches, Results, Current Challenges and Future DirectionsVincenzo Lomonaco, Lorenzo Pellegrini, Pau Rodríguez … Davide MaltoniArtificial Intelligence · University of Bologna · Mila - Quebec Artificial Intelligence Institute · +7
    PDF ↗
  4. 2021
    Wide Neural Networks Forget Less CatastrophicallySeyed Iman Mirzadeh, Arslan Chaudhry, Yin, Dong … Mehrdad FarajtabarICML · Google (United States)
    PDF ↗
  5. 2021
    Powerpropagation: A sparsity inducing weight reparameterisationJonathan Schwarz, Siddhant M. Jayakumar, Razvan Pascanu … Yee Whye TehNeurIPS · Google (United States)
    PDF ↗
  6. 2021
    Continual World: A Robotic Benchmark For Continual Reinforcement LearningMaciej Wołczyk, Michał Zając, Razvan Pascanu … Piotr MiłośNeurIPS · Jagiellonian University · Google (United States)
    PDF ↗
  7. 2021
    Continual Learning for Text Classification with Information Disentanglement Based RegularizationYufan Huang, Yanzhe Zhang, Jiaao Chen … Diyi YangNAACL · Georgia Institute of Technology · Google (United States) · +1
    PDF ↗
  8. 2021
    Towards Continual Learning for Multilingual Machine Translation via Vocabulary SubstitutionXavier García, Noah Constant, Ankur P. Parikh, Orhan FıratNAACL · Google (United States)
    PDF ↗
  9. 2020
    Reset-Free Lifelong Learning with Skill-Space PlanningKevin Lü, Aditya Grover, Pieter Abbeel, Igor MordatchICLR · University of California, Berkeley · Stanford University · +1
    PDF ↗
  10. 2020
    A Combinatorial Perspective on Transfer LearningJianan Wang, Eren Sezener, David Budden … Joel VenessNeurIPS · Google (United States)
    PDF ↗
  11. 2020
    Continual Learning for Neural Semantic ParsingVladislav Lialin, Rahul Goel, Andrey Simanovsky … Rushin ShaharXiv · University of Massachusetts Lowell · Google (United States)
    PDF ↗
  12. 2020
    Linear Mode Connectivity in Multitask and Continual LearningSeyed Iman Mirzadeh, Mehrdad Farajtabar, Dilan Görür … Hassan GhasemzadehICLR · Washington State University · Google (United States)
    PDF ↗
  13. 2020
    Routing Networks with Co-training for Continual LearningMark Collier, Efi Kokiopoulou, Andréa Gesmundo, Jesse BerentarXiv · Google (United States)
    PDF ↗
  14. 2020
    Anatomy of Catastrophic Forgetting: Hidden Representations and Task SemanticsVinay Ramasesh, Ethan Dyer, Maithra RaghuICLR · Google (United States) · Cornell University
    PDF ↗
  15. 2020
    Learning to Incentivize Other Learning AgentsJiachen Yang, Ang Li, Mehrdad Farajtabar … Hongyuan ZhaNeurIPS · Georgia Institute of Technology · Google (United States)
    PDF ↗
  16. 2020
    Gaussian Gated Linear NetworksDavid Budden, Adam Marblestone, Eren Sezener … Joel VenessNeurIPS · Google (United States)
    PDF ↗
  17. 2020
    Dropout as an Implicit Gating Mechanism For Continual LearningSeyed Iman Mirzadeh, Mehrdad Farajtabar, Hassan GhasemzadehCVPR · Washington State University · Google (United States)
    PDF ↗
  18. 2019
    Sequential Mastery of Multiple Visual Tasks: Networks Naturally Learn to Learn and Forget to ForgetGuy Davidson, Michael C. MozerCVPR · Supélec · University of Applied Sciences and Arts of Southern Switzerland · +2
    PDF ↗
  19. 2020
    BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong LearningYeming Wen, Dustin Tran, Jimmy BaICLR · University of Toronto · Google (United States)
    PDF ↗
  20. 2020
    What Happens To BERT Embeddings During Fine-tuning?Amil Merchant, Elahe Rahimtoroghi, Ellie Pavlick, Ian TenneyBlackboxNLP Workshop on Analyzing and Interpreting Neural… · Google (United States) · University of California, Santa Cruz · +1
    PDF ↗
  21. 2019
    Continual Unsupervised Representation LearningDushyant Rao, Francesco Visin, Andrei Rusu … Raia HadsellNeurIPS · Carnegie Mellon University · Google (United States) · +2
    PDF ↗
  22. 2019
    Orthogonal Gradient Descent for Continual LearningMehrdad Farajtabar, Navid Azizan, A. Mott, Ang LiAISTATS · Google (United States)
    PDF ↗
  23. 2019
    Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive ProcessesJames Requeima, Jonathan Gordon, John Bronskill … Richard E. TurnerNeurIPS · University of Cambridge · Google (United States)
    PDF ↗
  24. 2019
    Task Agnostic Continual Learning via Meta LearningXu He, Jakub Sygnowski, Alexandre Galashov … Razvan PascanuarXiv · Google (United States)
    PDF ↗
  25. 2019
    Online Meta-LearningChelsea Finn, Aravind Rajeswaran, Sham M. Kakade, Sergey LevineICML · Stanford University · University of Washington · +3
    PDF ↗
  26. 2019
    Learning and Evaluating General Linguistic IntelligenceDani Yogatama, Cyprien de Masson d’Autume, Jerome T. Connor … Phil BlunsomarXiv · Google (United States)
    PDF ↗
  27. 2019
    Functional Regularisation for Continual Learning using Gaussian ProcessesMichalis K. Titsias, Jonathan Schwarz, Alexander Matthews … Yee Whye TehICLR · Google (United States)
    PDF ↗
  28. 2019
    Incremental Reading for Question AnsweringSamira Abnar, Tania Bedrax-Weiss, Tom Kwiatkowski, William W. CohenarXiv · University of Amsterdam · Google (United States)
    PDF ↗
  29. 2019
    Non-Parametric Adaptation for Neural Machine TranslationAnkur Bapna, Orhan FıratNAACL · Google (United States)
    PDF ↗
  30. 2018
    Experience Replay for Continual LearningDavid Rolnick, Arun Ahuja, Jonathan Schwarz … Greg WayneNeurIPS · California University of Pennsylvania · University of Pennsylvania · +1
    PDF ↗
  31. 2018
    Memory-based Parameter AdaptationPablo Sprechmann, Siddhant M. Jayakumar, Jack W. Rae … Charles BlundellICLR · Google (United States)
    PDF ↗
  32. 2018
    Lifelong Machine Learning, Second EditionZhiyuan Chen, Bing LiuMachine Learning · Google (United States) · University of Illinois Chicago
  33. 2017
    Generative and Discriminative Text Classification with Recurrent Neural NetworksDani Yogatama, Chris Dyer, Ling Wang, Phil BlunsomarXiv · Baidu (China) · Carnegie Mellon University · +1
    PDF ↗
  34. 2016
    Lifelong Machine LearningZhiyuan Chen, Bing LiuMachine Learning · Google (United States) · University of Illinois Chicago
  35. 2016
    Net2Net: Accelerating Learning via Knowledge TransferTianqi Chen, Ian Goodfellow, Jonathon ShlensICLR · University of Washington · Google (United States)
    PDF ↗
About this index

We keep this list because we read the field and wanted one place to see it. It covers work on continual learning itself, in the core areas of machine learning, and leaves out papers that apply it inside another field, such as medical imaging or fault diagnosis. It is seeded from the community lists kept by ContinualAI and by Xialei Liu, then filled out from OpenAlex, and every week a script looks for new papers on OpenAlex and arXiv. A model reads each candidate and decides whether it belongs; a person reviews the additions before they go live. Authors and affiliations come from OpenAlex, so a recent preprint can lack its institutions for a week or two.

Missing something, or filed under the wrong venue? Write to hello@unify.ai with the arXiv id or DOI.