Knowledge Management System of Hefei Institute of Physical Science,CAS
Memory Protection Generative Adversarial Network (MPGAN): A Framework to Overcome the Forgetting of GANs Using Parameter Regularization Methods | |
Chang, Yifan1,2; Li, Wenbo2; Peng, Jian3; Li, Haifeng3; Kang, Yu1; Huang, Yingliang4 | |
2020 | |
发表期刊 | IEEE ACCESS |
ISSN | 2169-3536 |
通讯作者 | Kang, Yu(kangduyu@ustc.edu.cn) |
摘要 | Generative adversarial networks (GANs) suffer from catastrophic forgetting when learning multiple consecutive tasks. Parameter regularization methods that constrain the parameters of the new model in order to be close to the previous model through parameter importance are effective in overcoming forgetting. Many parameter regularization methods have been tried, but each of them is only suitable for limited types of neural networks. Aimed at GANs, this paper proposes a unified framework called Memory Protection GAN (MPGAN), in which many parametrization methods can be used to overcome forgetting. The proposed framework includes two modules: Protecting Weights in Generator and Controller. In order to incorporate parameter regularization methods into MPGAN, the Protecting Weights in Generator module encapsulates different parameter regularization methods into a "container", and consolidates the most important parameters in the generator through a parameter regularization method selected from the container. In order to differentiate tasks, the Controller module creates unique tags for the tasks. Another problem with existing parameter regularization methods is their low accuracy in measuring parameter importance. These methods always rely on the first derivative of the output function, and ignore the second derivative. To assess parameter importance more accurately, a new parameter regularization method called Second Derivative Preserver (SDP), which takes advantage of the second derivative of the output function, is designed into MPGAN. Experiments demonstrate that MPGAN is applicable to multiple parameter regularization methods and that SDP achieves high accuracy in parameter importance. |
关键词 | Gallium nitride Task analysis Generators Generative adversarial networks Training Neural networks Knowledge engineering Catastrophic forgetting generative adversarial network parameter regularization methods |
DOI | 10.1109/ACCESS.2020.3028067 |
收录类别 | SCI |
语种 | 英语 |
资助项目 | Development Program of China[2018YFB1004600] ; Development Program of China[2017YFC1503000] ; National Nature Science Foundation of China[41701594] ; National Nature Science Foundation of China[41871302] |
项目资助者 | Development Program of China ; National Nature Science Foundation of China |
WOS研究方向 | Computer Science ; Engineering ; Telecommunications |
WOS类目 | Computer Science, Information Systems ; Engineering, Electrical & Electronic ; Telecommunications |
WOS记录号 | WOS:000576765700001 |
出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.hfcas.ac.cn:8080/handle/334002/104462 |
专题 | 中国科学院合肥物质科学研究院 |
通讯作者 | Kang, Yu |
作者单位 | 1.Univ Sci & Technol China, Dept Automat, Hefei 230026, Peoples R China 2.Chinese Acad Sci, Hefei Inst Phys Sci, Inst Technol Innovat, Hefei 230088, Peoples R China 3.Cent South Univ, Sch Geosci & Infophys, Changsha 410083, Peoples R China 4.Chinese Acad Sci, Inst Intelligent Machines, Hefei 230031, Peoples R China |
推荐引用方式 GB/T 7714 | Chang, Yifan,Li, Wenbo,Peng, Jian,et al. Memory Protection Generative Adversarial Network (MPGAN): A Framework to Overcome the Forgetting of GANs Using Parameter Regularization Methods[J]. IEEE ACCESS,2020,8. |
APA | Chang, Yifan,Li, Wenbo,Peng, Jian,Li, Haifeng,Kang, Yu,&Huang, Yingliang.(2020).Memory Protection Generative Adversarial Network (MPGAN): A Framework to Overcome the Forgetting of GANs Using Parameter Regularization Methods.IEEE ACCESS,8. |
MLA | Chang, Yifan,et al."Memory Protection Generative Adversarial Network (MPGAN): A Framework to Overcome the Forgetting of GANs Using Parameter Regularization Methods".IEEE ACCESS 8(2020). |
条目包含的文件 | 条目无相关文件。 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论