Related work

The foundational work on continual learning, 1959 to 2026: methods, theory, benchmarks, surveys, and the neuroscience of memory. Updated weekly; some carry our notes.

45 papers of 11,817Sort Recent · Most cited
  1. 2025
    Efficient Prompting for Continual Adaptation to Missing ModalitiesZirun Guo, Shulei Wang, Wang Lin … Tao JinNAACL
    PDF ↗
  2. 2025
    Systematic Knowledge Injection into Large Language Models via Diverse Augmentation for Domain-Specific RAGKushagra Bhushan, Yatin Nandwani, Dinesh Khandelwal … Sachindra JoshiNAACL
    PDF ↗
  3. 2025
    MixLLM: Dynamic Routing in Mixed Large Language ModelsXinyuan Wang, Yanchi Liu, Wei Cheng … Haifeng ChenNAACL
    PDF ↗
  4. 2025
  5. 2025
    Multi-Stage LLM Fine-Tuning with a Continual Learning SettingChanghao Guan, Chao Huang, Hongliang Li … Jian LiuNAACL
  6. 2025
    Task-wrapped Continual Learning in Task-Oriented Dialogue SystemsMin Zeng, Haiqin Yang, Xi Chen, Yike GuoNAACL
  7. 2024
    On the Impact of Fine-Tuning on Chain-of-Thought ReasoningElita Lobo, Chirag Agarwal, Himabindu LakkarajuNAACL
    PDF ↗
  8. 2024PDF ↗
  9. 2024
    SLIM: Let LLM Learn More and Forget Less with Soft LoRA and Identity MixtureJiayi Han, Liang Du, Hongwei Du … Donghong HanNAACL
    PDF ↗
  10. 2024PDF ↗
  11. 2024PDF ↗
  12. 2024
    Q-Tuning: Queue-based Prompt Tuning for Lifelong Few-shot Language LearningYanhui Guo, Shaoyuan Xu, Jinmiao Fu … Bryan WangNAACL
    PDF ↗
  13. 2024PDF ↗
  14. 2024PDF ↗
  15. 2024
    Rehearsal-Free Modular and Compositional Continual Learning for Language ModelsMingyang Wang, Heike Adel, Lukas Lange … Hinrich SchützeNAACL
    PDF ↗
  16. 2024PDF ↗
  17. 2024PDF ↗
  18. 2024PDF ↗
  19. 2024
    UNDIAL: Self-Distillation with Adjusted Logits for Robust Unlearning in Large Language ModelsYijiang River Dong, Hongzhou Lin, Mikhail Belkin … Ivan Vuli'cNAACL
    PDF ↗
  20. 2024PDF ↗
  21. 2024
  22. 2024
  23. 2024
  24. 2023
    Carpe diem: On the Evaluation of World Knowledge in Lifelong Language ModelsYujin Kim, Jaehong Yoon, Seonghyeon Ye … Se-young YunNAACL
    PDF ↗
  25. 2023PDF ↗
  26. 2022
    ProQA: Structural Prompt-based Pre-training for Unified Question AnsweringWanjun Zhong, Yifan Gao, Ning Ding … Nan DuanNAACL
    PDF ↗
  27. 2022
    KALA: Knowledge-Augmented Language Model AdaptationMinki Kang, Jinheon Baek, Sung Ju HwangNAACL
    PDF ↗
  28. 2022
    Overcoming Catastrophic Forgetting During Domain Adaptation of Seq2seq Language GenerationDingcheng Li, Zheng Chen, Eunah Cho … Yang LiuNAACL · Amazon (United States)
  29. 2022
    Lifelong Pretraining: Continually Adapting Language Models to Emerging CorporaXisen Jin, Dejiao Zhang, Henghui Zhu … Xiang RenNAACL
    PDF ↗
  30. 2021
    Continual Learning for Text Classification with Information Disentanglement Based RegularizationYufan Huang, Yanzhe Zhang, Jiaao Chen … Diyi YangNAACL · Georgia Institute of Technology · Google (United States) · +1
    PDF ↗
  31. 2021
    Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification TasksZixuan Ke, Hu Xu, Bing LiuNAACL · University of Illinois Chicago · Meta (Israel)
    PDF ↗
  32. 2021
    AdaptSum: Towards Low-Resource Domain Adaptation for Abstractive SummarizationTiezheng Yu, Zihan Liu, Pascale FungNAACL · Hong Kong University of Science and Technology
    PDF ↗
  33. 2021
    Towards Continual Learning for Multilingual Machine Translation via Vocabulary SubstitutionXavier García, Noah Constant, Ankur P. Parikh, Orhan FıratNAACL · Google (United States)
    PDF ↗
  34. 2021
    Continual Learning for Neural Machine TranslationYue Cao, Haoran Wei, Boxing Chen, Xiaojun WanNAACL · Peking University · Alibaba Group (United States)
  35. 2021
    Pruning-then-Expanding Model for Domain Adaptation of Neural Machine TranslationShuhao Gu, Yang Feng, Wanying XieNAACL · Chinese Academy of Sciences · Institute of Computing Technology · +2
    PDF ↗
  36. 2021
    Hyperparameter-free Continuous Learning for Domain Classification in Natural Language UnderstandingTing Hua, Yilin Shen, Changsheng Zhao … Hongxia JinNAACL · Samsung (United States) · Research!America (United States)
    PDF ↗
  37. 2021
    Language Scaling for Universal Suggested Replies ModelQianlan Ying, Payal Bajaj, Budhaditya Deb … Daxin JiangNAACL · Microsoft (United States) · Microsoft Research Asia (China) · +3
    PDF ↗
  38. 2021
    Combining Weakly Supervised ML Techniques for Low-Resource NLUVíctor Soto, Konstantine ArkoudasNAACL · Amazon (United States)
  39. 2019
    Sentence Embedding Alignment for Lifelong Relation ExtractionHongfei Wang, Wenhan Xiong, Mo Yu … William Yang WangNAACL · University of California, Santa Barbara
    PDF ↗
  40. 2019
    Overcoming Catastrophic Forgetting During Domain Adaptation of Neural Machine TranslationBrian J. Thompson, Jeremy Gwinnup, Huda Khayrallah … Philipp KoehnNAACL · Johns Hopkins University
  41. 2019
    An Embarrassingly Simple Approach for Transfer Learning from Pretrained Language ModelsAlexandra Chronopoulou, Christos Baziotis, Alexandros PotamianosNAACL · National Technical University of Athens
    PDF ↗
  42. 2019
    Non-Parametric Adaptation for Neural Machine TranslationAnkur Bapna, Orhan FıratNAACL · Google (United States)
    PDF ↗
  43. 2019
    Continual Learning for Sentence Representations Using ConceptorsTianlin Liu, Lyle Ungar, João SedocNAACL · Constructor University · University of Pennsylvania
    PDF ↗
  44. 2019
    Continuous Learning for Large-scale Personalized Domain ClassificationHan Li, Jihwan Lee, Sidharth Mudgal … Young‐Bum KimNAACL · University of Wisconsin–Madison · Amazon (Germany)
    PDF ↗
  45. 2015
    Lifelong Machine Learning for Topic Modeling and BeyondZhiyuan ChenNAACL · University of Illinois Chicago
About this index

We keep this list because we read the field and wanted one place to see it. It covers work on continual learning itself, in the core areas of machine learning, and leaves out papers that apply it inside another field, such as medical imaging or fault diagnosis. It is seeded from the community lists kept by ContinualAI and by Xialei Liu, then filled out from OpenAlex, and every week a script looks for new papers on OpenAlex and arXiv. A model reads each candidate and decides whether it belongs; a person reviews the additions before they go live. Authors and affiliations come from OpenAlex, so a recent preprint can lack its institutions for a week or two.

Missing something, or filed under the wrong venue? Write to hello@unify.ai with the arXiv id or DOI.