Preprint · 2022 · Preprint

Elastic Weight Consolidation for Reduction of Catastrophic Forgetting in GPT-2

Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ B. Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, E. Brunskill, Erik Brynjolfsson, S. Buch, Dallas Card, Rodrigo Castellon, Niladri S. Chatterji, Annie Chen, Kathleen A. Creel, J. Davis, Dora Demszky, Chris Donahue, M. Doumbouya, Esin Durmus, S. Ermon, J. Etchemendy, Kawin Ethayarajh, Fei-Fei Li, Chelsea Finn, Trevor Gale, Lauren Gillespie, Karan Goel, Noah D. Goodman, Shelby Grossman, Neel Guha, Tatsunori Hashimoto, Peter Henderson, John Hewitt, Daniel E. Ho, Jenny Hong, Kyle Hsu, Jing Huang, Thomas Icard, Saahil Jain, Dan Jurafsky, P. Kalluri, Siddharth Karamcheti, G. Keeling, Fereshte Khani, Pang Omar Khattab, Wei Koh, M. Krass, Ranjay Kr-303 ishna, Rohith Kuditipudi, Ananya Kumar, Faisal Lad-304 hak, Mina Lee, Tony Lee, J. Leskovec, Isabelle Levent, Xiang Lisa, Xuechen Li, Tengyu Li, Ma Ali, Christopher D Malik, Suvir Mirchan-307 dani Manning, Eric Mitchell, Zanele Munyikwa, Suraj Nair, A. Narayan, Deepak Narayanan, Benjamin Newman, Hongyu Reich, Frieda Ren, Yusuf Rong, Roohani Camilo, J. Ruiz, C. Ryan, Dorsa Ré, S. Sadigh, Keshav Sagawa, Andy Santhanam, Krishnan Shih, A. Srinivasan, Rohan Tamkin, Armin W Taori, Florian Thomas, Rose E Tramèr, William Wang, Bohan Wang, Jiajun Wu, Yuhuai Wu, S. Wu, Michihiro Xie, Jiaxuan Yasunaga, M. You, Michael Zaharia, Tianyi Zhang, Zhang Xikun, Yuhui Zhang, Lucia Zhang, Kaitlyn Zheng, Zhou Percy, Liang, Michael McCloskey, Neal J. Cohen, Catas-351, G. I. Parisi, Ronald Kemker, Jose L. Part, C. Kanan, Stefan Wermter. 2019, Continual, Alec Radford, Jeffrey Wu, R. Child, D. Luan

Source ↗

Citations: 0

Cite

@misc{bommasani2022elastic,
title = {Elastic Weight Consolidation for Reduction of Catastrophic Forgetting in GPT-2},
author = {Rishi Bommasani and Drew A. Hudson and Ehsan Adeli and Russ B. Altman and Simran Arora and Sydney von Arx and Michael S. Bernstein and Jeannette Bohg and Antoine Bosselut and E. Brunskill and Erik Brynjolfsson and S. Buch and Dallas Card and Rodrigo Castellon and Niladri S. Chatterji and Annie Chen and Kathleen A. Creel and J. Davis and Dora Demszky and Chris Donahue and M. Doumbouya and Esin Durmus and S. Ermon and J. Etchemendy and Kawin Ethayarajh and Fei-Fei Li and Chelsea Finn and Trevor Gale and Lauren Gillespie and Karan Goel and Noah D. Goodman and Shelby Grossman and Neel Guha and Tatsunori Hashimoto and Peter Henderson and John Hewitt and Daniel E. Ho and Jenny Hong and Kyle Hsu and Jing Huang and Thomas Icard and Saahil Jain and Dan Jurafsky and P. Kalluri and Siddharth Karamcheti and G. Keeling and Fereshte Khani and Pang Omar Khattab and Wei Koh and M. Krass and Ranjay Kr-303 ishna and Rohith Kuditipudi and Ananya Kumar and Faisal Lad-304 hak and Mina Lee and Tony Lee and J. Leskovec and Isabelle Levent and Xiang Lisa and Xuechen Li and Tengyu Li and Ma Ali and Christopher D Malik and Suvir Mirchan-307 dani Manning and Eric Mitchell and Zanele Munyikwa and Suraj Nair and A. Narayan and Deepak Narayanan and Benjamin Newman and Hongyu Reich and Frieda Ren and Yusuf Rong and Roohani Camilo and J. Ruiz and C. Ryan and Dorsa Ré and S. Sadigh and Keshav Sagawa and Andy Santhanam and Krishnan Shih and A. Srinivasan and Rohan Tamkin and Armin W Taori and Florian Thomas and Rose E Tramèr and William Wang and Bohan Wang and Jiajun Wu and Yuhuai Wu and S. Wu and Michihiro Xie and Jiaxuan Yasunaga and M. You and Michael Zaharia and Tianyi Zhang and Zhang Xikun and Yuhui Zhang and Lucia Zhang and Kaitlyn Zheng and Zhou Percy and Liang and Michael McCloskey and Neal J. Cohen and Catas-351 and G. I. Parisi and Ronald Kemker and Jose L. Part and C. Kanan and Stefan Wermter. 2019 and Continual and Alec Radford and Jeffrey Wu and R. Child and D. Luan},
year = {2022},
url = {https://www.semanticscholar.org/paper/6abe237609c14e34be746b3822d663746b95bb07},
}