Related work

The foundational work on continual learning, 1980 to 2026: methods, theory, benchmarks, surveys, and the neuroscience of memory. Updated weekly; some carry our notes.

93 papers of 8,653 · showing 51–93Sort Recent · Most cited
  1. 2023
    Continual Learning for Multilingual Neural Machine Translation via Dual Importance-based Model DivisionJunpeng Liu, Kaiyu Huang, Hao Yu … Degen HuangEMNLP · Dalian University of Technology · Tsinghua University · +1
    PDF ↗
  2. 2023
    Visually Grounded Continual Language Learning with Selective SpecializationKyra Ahrens, Lennart Bengtson, Jae Yeol Lee, Stefan WermterEMNLP · Universität Hamburg
    PDF ↗
  3. 2023
    Incorporating Syntactic Knowledge into Pre-trained Language Model using Optimization for Overcoming Catastrophic ForgettingRan Iwamoto, Issei Yoshida, Hiroshi Kanayama … Masayasu MuraokaEMNLP · Keio University · IBM Research - Tokyo
    PDF ↗
  4. 2023
    DSI++: Updating Transformer Memory with New DocumentsSanket Vaibhav Mehta, Jai Prakash Gupta, Yi Tay … Donald MetzlerEMNLP
    PDF ↗
  5. 2022
    G-MAP: General Memory-Augmented Pre-trained Language Model for Domain TasksZhongwei Wan, Yichun Yin, Wei Zhang … Liu, QunEMNLP
    PDF ↗
  6. 2022
    Semi-Supervised Lifelong Language LearningYingxiu Zhao, Yinhe Zheng, Bowen Yu … Nevin L. ZhangEMNLP
    PDF ↗
  7. 2022PDF ↗
  8. 2022PDF ↗
  9. 2022PDF ↗
  10. 2022PDF ↗
  11. 2022
    Federated Continual Learning for Text Classification via Selective Inter-client TransferYatin Chaudhary, Pranav Rai, Matthias Schubert … Pankaj GuptaEMNLP
    PDF ↗
  12. 2022
    Continual Training of Language Models for Few-Shot LearningZixuan Ke, Haowei Lin, Yijia Shao … Bing LiuEMNLP
    PDF ↗
  13. 2022PDF ↗
  14. 2022PDF ↗
  15. 2022
    Overcoming Catastrophic Forgetting in Zero-Shot Cross-Lingual GenerationTu Vu, Aditya Barua, Brian Lester … Noah ConstantEMNLP
    PDF ↗
  16. 2022PDF ↗
  17. 2022
    Fine-tuned Language Models are Continual LearnersThomas Scialom, Tuhin Chakrabarty, Smaranda MuresanEMNLP · University of Missouri · Columbia University
    PDF ↗
  18. 2022
    LPC: A Logits and Parameter Calibration Framework for Continual LearningXiaodi Li, Zhuoyi Wang, Dingcheng Li … Bhavani ThuraisinghamEMNLP · The University of Texas at Dallas · Baidu (China)
    PDF ↗
  19. 2022
    CLLE: A Benchmark for Continual Language Learning Evaluation in Multilingual Machine TranslationHan Zhang, Sheng Zhang, Yang Xiang … Ruifeng XuEMNLP · Harbin Institute of Technology · Peng Cheng Laboratory · +3
    PDF ↗
  20. 2022
    Entropy-Based Vocabulary Substitution for Incremental Learning in Multilingual Neural Machine TranslationKaiyu Huang, Peng Li, Jin Ma, Yang LiuEMNLP · Tsinghua University · Peng Cheng Laboratory · +5
    PDF ↗
  21. 2022
    Improving Scheduled Sampling with Elastic Weight Consolidation for Neural Machine TranslationMichalis Korakakis, Andreas VlachosEMNLP · University of Cambridge
    PDF ↗
  22. 2021
    Continual Learning in Task-Oriented Dialogue SystemsAndrea Madotto, Zhaojiang Lin, Zhenpeng Zhou … Zhiguang WangEMNLP · Hong Kong University of Science and Technology · Meta (Israel) · +1
    PDF ↗
  23. 2021
    CLASSIC: Continual and Contrastive Learning of Aspect Sentiment Classification TasksZixuan Ke, Bing Liu, Hu Xu, Lei ShuEMNLP · University of Illinois Chicago · Meta (Israel) · +1
    PDF ↗
  24. 2021
    Learn Continually, Generalize Rapidly: Lifelong Knowledge Accumulation for Few-shot LearningXisen Jin, Bill Yuchen Lin, Mohammad Rostami, Xiang RenEMNLP · University of Southern California · Biomedical Research Institute of Southern California · +1
    PDF ↗
  25. 2021
    Lifelong Event Detection with Knowledge TransferPengfei Yu, Heng Ji, Prem NatarajanEMNLP · University of Illinois Urbana-Champaign · Amazon (United States)
    PDF ↗
  26. 2021
    Total Recall: a Customized Continual Learning Method for Neural Semantic ParsersZhuang Li, Lizhen Qu, Gholamreza HaffariEMNLP · Monash University
    PDF ↗
  27. 2021
    Domain-Lifelong Learning for Dialogue State Tracking via Knowledge Preservation NetworksQingbin Liu, Pengfei Cao, Cao Liu … Jun ZhaoEMNLP · Chinese Academy of Sciences · Shandong Institute of Automation · +5
  28. 2021
    Improving the Quality Trade-Off for Neural Machine Translation Multi-Domain AdaptationEva Hasler, Tobias Domhan, Jonay Trénous … Felix HieberEMNLP · Amazon (Germany)
  29. 2021
    Lifelong Explainer for Lifelong LearnersXuelin Situ, Sameen Maruf, Ingrid Zukerman … Gholamreza HaffariEMNLP · Monash University · Commonwealth Scientific and Industrial Research Organisation · +2
  30. 2020
    Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less ForgettingSanyuan Chen, Yutai Hou, Yiming Cui … Xiangzhan YuEMNLP · Harbin Institute of Technology · Joint Research Centre · +1
    PDF ↗
  31. 2020
    Continual Learning for Natural Language Generation in Task-oriented Dialog SystemsFei Mi, Liangwei Chen, Mengjie Zhao … Boi FaltingsEMNLP · École Polytechnique Fédérale de Lausanne · LMU Klinikum · +2
    PDF ↗
  32. 2020
    Efficient Meta Lifelong-Learning with Limited MemoryZirui Wang, Sanket Vaibhav Mehta, Barnabás Póczos, Jaime CarbonellEMNLP · Carnegie Mellon University
    PDF ↗
  33. 2020
    Disentangle-based Continual Graph Representation LearningXiaoyu Kou, Yankai Lin, Shaobo Liu … Yan ZhangEMNLP · Peking University · Tencent (China)
    PDF ↗
  34. 2020
    Incremental Event Detection via Knowledge Consolidation NetworksPengfei Cao, Yubo Chen, Jun Zhao, Taifeng WangEMNLP
  35. 2020PDF ↗
  36. 2020
    Visually Grounded Continual Learning of Compositional PhrasesXisen Jin, Junyi Du, Arka Sadhu … Xiang RenEMNLP · University of Southern California · California Southern University · +1
  37. 2020
    An Empirical Investigation towards Efficient Multi-Domain Language Model Pre-trainingKristjan Arumae, Qing Sun, Parminder BhatiaEMNLP · Amazon (United States) · Seattle University · +1
    PDF ↗
  38. 2020
    Using the Past Knowledge to Improve Sentiment ClassificationQi Qin, Wenpeng Hu, Bing LiuEMNLP · Peking University · King University
  39. 2020
    Continual Learning Long Short Term MemoryXin Guo, Yu Tian, Qinghan Xue … Xiaolong WangEMNLP · University of Delaware
  40. 2020
    Findings of the First Shared Task on Lifelong Learning Machine TranslationLoïc Barrault, Magdalena Biesialska, Marta R. Costa‐jussà … Olivier GalibertEMNLP · University of Sheffield · Universitat Politècnica de Catalunya · +2
  41. 2019
    A Progressive Model to Enable Continual Learning for Semantic Slot FillingYilin Shen, Xiangyu Zeng, Hongxia JinEMNLP · Samsung (South Korea) · Samsung (United States) · +1
  42. 2017
    A Joint Many-Task Model: Growing a Neural Network for Multiple NLP TasksKazuma Hashimoto, Caiming Xiong, Yoshimasa Tsuruoka, Richard SocherEMNLP · Salesforce (United States) · The University of Tokyo
    PDF ↗
  43. 2016PDF ↗
About this index

We keep this list because we read the field and wanted one place to see it. It covers work on continual learning itself, in the core areas of machine learning, and leaves out papers that apply it inside another field, such as medical imaging or fault diagnosis. By default it shows the papers we have a reason to trust: published at a venue like NeurIPS, ICML, ICLR, CVPR or TPAMI, or written by someone who has published there, or cited a few hundred times. The rest are one click away under “All papers”. It is seeded from the community lists kept by ContinualAI and by Xialei Liu, then filled out from OpenAlex, and every week a script looks for new papers on OpenAlex and arXiv. A model reads each candidate and decides whether it belongs; a person reviews the additions before they go live. Authors and affiliations come from OpenAlex, so a recent preprint can lack its institutions for a week or two.

Missing something, or filed under the wrong venue? Write to hello@unify.ai with the arXiv id or DOI.