Related work

The foundational work on continual learning, 1959 to 2026: methods, theory, benchmarks, surveys, and the neuroscience of memory. Updated weekly; some carry our notes.

22 papers of 11,817Sort Recent · Most cited
  1. 2022
    Building a Subspace of Policies for Scalable Continual LearningJean-Baptiste Gaya, Thang Doan, Lucas Caccia … Roberta RăileanuICLR · Sorbonne Université · Institut Systèmes Intelligents et de Robotique · +2
    PDF ↗
  2. 2022
    Broken Neural Scaling LawsEthan Caballero, Kshitij Gupta, Irina Rish, David KruegerICLR
    PDF ↗
  3. 2022
    On the Soft-Subnetwork for Few-shot Class Incremental LearningHaeyong Kang, Jaehong Yoon, Sultan Rizky Hikmawan Madjid … Chang D. YooICLR
    PDF ↗
  4. 2022
    A Theoretical Framework for Inference and Learning in Predictive Coding NetworksBeren Millidge, Yuhang Song, Tommaso Salvatori … Rafał BogaczICLR
    PDF ↗
  5. 2022
    A Model or 603 Exemplars: Towards Memory-Efficient Class-Incremental LearningDa-Wei Zhou, Qiwei Wang, Han-Jia Ye, De‐Chuan ZhanICLR
    PDF ↗
  6. 2022
    Continual evaluation for lifelong learning: Identifying the stability gapMatthias De Lange, Gido M. van de Ven, Tinne TuytelaarsICLR
    PDF ↗
  7. 2022PDF ↗
  8. 2022
    Information-theoretic Online Memory Selection for Continual LearningShengyang Sun, Daniele Calandriello, Huiyi Hu … Michalis K. TitsiasICLR
    PDF ↗
  9. 2022PDF ↗
  10. 2022
    Resonance in Weight Space: Covariate Shift Can Drive Divergence of SGD with MomentumKirby Banman, Liam Peet-Pare, Nidhi Hegde … Martha WhiteICLR
    PDF ↗
  11. 2022
    Acceleration of Federated Learning with Alleviated Forgetting in Local TrainingChencheng Xu, Zhiwei Hong, Minlie Huang, Tao JiangICLR
    PDF ↗
  12. 2022
    Learning Fast and Slow for Online Time Series ForecastingQuang Pham, Chenghao Liu, Doyen Sahoo, Steven C. H. HoiICLR
    PDF ↗
  13. 2022
    Memory Replay with Data Compression for Continual LearningLiyuan Wang, Xingxing Zhang, Kuo Yang … Jun ZhuICLR
    PDF ↗
  14. 2022
    Incremental Learning of Structured Memory via Closed-Loop TranscriptionShengbang Tong, Xili Dai, Ziyang Wu … Yi MaICLR
    PDF ↗
  15. 2022
    TRGP: Trust Region Gradient Projection for Continual LearningSen Lin, Li Yang, Deliang Fan, Junshan ZhangICLR
    PDF ↗
  16. 2022PDF ↗
  17. 2022PDF ↗
  18. 2022
    Effect of scale on catastrophic forgetting in neural networksV. Ramasesh, Aitor Lewkowycz, Ethan DyerICLR
  19. 2022
    Pretrained Language Model in Continual Learning: A Comparative StudyTongtong Wu, Massimo Caccia, Zhuang Li … Gholamreza HaffariICLR
  20. 2022
    Looking Back on Learned Experiences For Class/task Incremental LearningMozhgan Pourkeshavarz, Guoying Zhao, M. SabokrouICLR
  21. 2022
    Continual Learning with Filter Atom SwappingZichen Miao, Ze Wang, Wei Chen, Qiang QiuICLR
  22. 2022
About this index

We keep this list because we read the field and wanted one place to see it. It covers work on continual learning itself, in the core areas of machine learning, and leaves out papers that apply it inside another field, such as medical imaging or fault diagnosis. It is seeded from the community lists kept by ContinualAI and by Xialei Liu, then filled out from OpenAlex, and every week a script looks for new papers on OpenAlex and arXiv. A model reads each candidate and decides whether it belongs; a person reviews the additions before they go live. Authors and affiliations come from OpenAlex, so a recent preprint can lack its institutions for a week or two.

Missing something, or filed under the wrong venue? Write to hello@unify.ai with the arXiv id or DOI.