Related work

The foundational work on continual learning, 1959 to 2026: methods, theory, benchmarks, surveys, and the neuroscience of memory. Updated weekly; some carry our notes.

109 papers of 11,817 · showing 51–100Sort Recent · Most cited
  1. 2023
    Orthogonal Subspace Learning for Language Model Continual LearningXiao Wang, Tianze Chen, Qiming Ge … Xuanjing HuangEMNLP
    PDF ↗
  2. 2023PDF ↗
  3. 2023PDF ↗
  4. 2023
    Sub-network Discovery and Soft-masking for Continual Learning of Mixed TasksZixuan Ke, Bing Liu, Wenhan Xiong … Haoran LiEMNLP
    PDF ↗
  5. 2023
    Rationale-Enhanced Language Models are Better Continual Relation LearnersWeimin Xiong, Yifan Song, Peiyi Wang, Sujian LiEMNLP
    PDF ↗
  6. 2023PDF ↗
  7. 2023
    Dynosaur: A Dynamic Growth Paradigm for Instruction-Tuning Data CurationDa Yin, Xiao Liu, Fan Yin … Kai-Wei ChangEMNLP
    PDF ↗
  8. 2023
    Continual Dialogue State Tracking via Example-Guided Question AnsweringHyundong Justin Cho, Andrea Madotto, Zhaojiang Lin … Chinnadhurai SankarEMNLP
    PDF ↗
  9. 2023PDF ↗
  10. 2023
  11. 2023
  12. 2023
    Causal Intervention-based Few-Shot Named Entity RecognitionZhen Yang, Yongbin Liu, Chunping OuyangEMNLP
  13. 2023
  14. 2023
  15. 2022
    DSI++: Updating Transformer Memory with New DocumentsSanket Vaibhav Mehta, Jai Prakash Gupta, Yi Tay … Donald MetzlerEMNLP
    PDF ↗
  16. 2022
    G-MAP: General Memory-Augmented Pre-trained Language Model for Domain TasksZhongwei Wan, Yichun Yin, Wei Zhang … Liu, QunEMNLP
    PDF ↗
  17. 2022
    Semi-Supervised Lifelong Language LearningYingxiu Zhao, Yinhe Zheng, Bowen Yu … Nevin L. ZhangEMNLP
    PDF ↗
  18. 2022PDF ↗
  19. 2022PDF ↗
  20. 2022PDF ↗
  21. 2022PDF ↗
  22. 2022
    Federated Continual Learning for Text Classification via Selective Inter-client TransferYatin Chaudhary, Pranav Rai, Matthias Schubert … Pankaj GuptaEMNLP
    PDF ↗
  23. 2022PDF ↗
  24. 2022PDF ↗
  25. 2022
    Overcoming Catastrophic Forgetting in Zero-Shot Cross-Lingual GenerationTu Vu, Aditya Barua, Brian Lester … Noah ConstantEMNLP
    PDF ↗
  26. 2022PDF ↗
  27. 2022
    Fine-tuned Language Models are Continual LearnersThomas Scialom, Tuhin Chakrabarty, Smaranda MuresanEMNLP · University of Missouri · Columbia University
    PDF ↗
  28. 2022
    What Has Been Enhanced in my Knowledge-Enhanced Language Model?Yifan Hou, Guoji Fu, Mrinmaya SachanEMNLP · ETH Zurich · Southern University of Science and Technology
    PDF ↗
  29. 2022
    LPC: A Logits and Parameter Calibration Framework for Continual LearningXiaodi Li, Zhuoyi Wang, Dingcheng Li … B. ThuraisinghamEMNLP
  30. 2022
  31. 2022
  32. 2022PDF ↗
  33. 2021
    A Unified Speaker Adaptation Approach for ASRYingzhu Zhao, Chongjia Ni, C. Leung … B. MaEMNLP
    PDF ↗
  34. 2021PDF ↗
  35. 2021
    Cross-Attention is All You Need: Adapting Pretrained Transformers for Machine TranslationMozhdeh Gheini, Xiang Ren, Jonathan MayEMNLP · University of Southern California · California Southern University
    PDF ↗
  36. 2020
    Continual Learning in Task-Oriented Dialogue SystemsAndrea Madotto, Zhaojiang Lin, Zhenpeng Zhou … Zhiguang WangEMNLP · Hong Kong University of Science and Technology · Meta (Israel) · +1
    PDF ↗
  37. 2021
    Structural Adapters in Pretrained Language Models for AMR-to-Text GenerationLeonardo F. R. Ribeiro, Yue Zhang, Iryna GurevychEMNLP · Westlake University · Technische Universität Darmstadt · +2
    PDF ↗
  38. 2021
    CLASSIC: Continual and Contrastive Learning of Aspect Sentiment Classification TasksZixuan Ke, Bing Liu, Hu Xu, Lei ShuEMNLP · University of Illinois Chicago · Meta (Israel) · +1
    PDF ↗
  39. 2021
    Learn Continually, Generalize Rapidly: Lifelong Knowledge Accumulation for Few-shot LearningXisen Jin, Bill Yuchen Lin, Mohammad Rostami, Xiang RenEMNLP · University of Southern California · Biomedical Research Institute of Southern California · +1
    PDF ↗
  40. 2021
    Wav-BERT: Cooperative Acoustic and Linguistic Representation Learning for Low-Resource Speech RecognitionGuolin Zheng, Yubei Xiao, Ke Gong … Liang LinEMNLP · Sun Yat-sen University · ARC Centre of Excellence for Dark Matter Particle Physics
    PDF ↗
  41. 2021
    Total Recall: a Customized Continual Learning Method for Neural Semantic ParsersZhuang Li, Lizhen Qu, Gholamreza HaffariEMNLP · Monash University
    PDF ↗
  42. 2021
    Domain-Lifelong Learning for Dialogue State Tracking via Knowledge Preservation NetworksQingbin Liu, Pengfei Cao, Cao Liu … Jun ZhaoEMNLP · Chinese Academy of Sciences · Shandong Institute of Automation · +5
  43. 2021
    Improving the Quality Trade-Off for Neural Machine Translation Multi-Domain AdaptationEva Hasler, Tobias Domhan, Jonay Trénous … Felix HieberEMNLP · Amazon (Germany)
  44. 2021
    Lifelong Explainer for Lifelong LearnersXuelin Situ, Sameen Maruf, Ingrid Zukerman … Gholamreza HaffariEMNLP · Monash University · Commonwealth Scientific and Industrial Research Organisation · +2
  45. 2021
    Lifelong Event Detection with Knowledge TransferPengfei Yu, Heng Ji, P. NatarajanEMNLP
  46. 2020
    Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less ForgettingSanyuan Chen, Yutai Hou, Yiming Cui … Xiangzhan YuEMNLP · Harbin Institute of Technology · Joint Research Centre · +1
    PDF ↗
  47. 2020
    Continual Learning for Natural Language Generation in Task-oriented Dialog SystemsFei Mi, Liangwei Chen, Mengjie Zhao … Boi FaltingsEMNLP · École Polytechnique Fédérale de Lausanne · LMU Klinikum · +2
    PDF ↗
  48. 2020
    Efficient Meta Lifelong-Learning with Limited MemoryZirui Wang, Sanket Vaibhav Mehta, Barnabás Póczos, Jaime CarbonellEMNLP · Carnegie Mellon University
    PDF ↗
  49. 2020
    Disentangle-based Continual Graph Representation LearningXiaoyu Kou, Yankai Lin, Shaobo Liu … Yan ZhangEMNLP · Peking University · Tencent (China)
    PDF ↗
  50. 2020
    Incremental Event Detection via Knowledge Consolidation NetworksPengfei Cao, Yubo Chen, Jun Zhao, Taifeng WangEMNLP
About this index

We keep this list because we read the field and wanted one place to see it. It covers work on continual learning itself, in the core areas of machine learning, and leaves out papers that apply it inside another field, such as medical imaging or fault diagnosis. It is seeded from the community lists kept by ContinualAI and by Xialei Liu, then filled out from OpenAlex, and every week a script looks for new papers on OpenAlex and arXiv. A model reads each candidate and decides whether it belongs; a person reviews the additions before they go live. Authors and affiliations come from OpenAlex, so a recent preprint can lack its institutions for a week or two.

Missing something, or filed under the wrong venue? Write to hello@unify.ai with the arXiv id or DOI.