Memory-Augmented Deep Unfolding Network for Guided Image Super-resolution

doi:10.1007/s11263-022-01699-1

HFCAS OpenIR

	Memory-Augmented Deep Unfolding Network for Guided Image Super-resolution
	Zhou, Man 1,2; Yan, Keyu 1,2; Pan, Jinshan 3; Ren, Wenqi 4; Xie, Qi 5; Cao, Xiangyong 6,7
	2022-10-15
发表期刊	INTERNATIONAL JOURNAL OF COMPUTER VISION
ISSN	0920-5691
通讯作者	Cao, Xiangyong(caoxiangyong@mail.xjtu.edu.cn)
摘要	Guided image super-resolution (GISR) aims to obtain a high-resolution (HR) target image by enhancing the spatial resolution of a low-resolution (LR) target image under the guidance of a HR image. However, previous model-based methods mainly take the entire image as a whole, and assume the prior distribution between the HR target image and the HR guidance image, simply ignoring many non-local common characteristics between them. To alleviate this issue, we firstly propose a maximum a posteriori (MAP) estimation model for GISR with two types of priors on the HR target image, i.e., local implicit prior and global implicit prior. The local implicit prior aims to model the complex relationship between the HR target image and the HR guidance image from a local perspective, and the global implicit prior considers the non-local auto-regression property between the two images from a global perspective. Secondly, we design a novel alternating optimization algorithm to solve this model for GISR. The algorithm is in a concise framework that facilitates to be replicated into commonly used deep network structures. Thirdly, to reduce the information loss across iterative stages, the persistent memory mechanism is introduced to augment the information representation by exploiting the Long short-term memory unit (LSTM) in the image and feature spaces. In this way, a deep network with certain interpretation and high representation ability is built. Extensive experimental results validate the superiority of our method on a variety of GISR tasks, including Pan-sharpening, depth image super-resolution, and MR image super-resolution. Code will be released at https://github.com/manman1995/pansharpening.
关键词	Guided image super-resolution Deep unfolding network Persistent memory mechanism Pan-sharpening Depth image super-resolution MR image super-resolution
DOI	10.1007/s11263-022-01699-1
关键词[WOS]	MULTI-CONTRAST SUPERRESOLUTION ; SPARSE REPRESENTATION ; FUSION ; INTERPOLATION ; ENHANCEMENT ; RECOVERY ; MRI
收录类别	SCI
语种	英语
资助项目	National Key Research and Development Project of China[2021ZD0110700] ; National Natural Science Foundation of China[62272375] ; National Natural Science Foundation of China[61906151] ; National Natural Science Foundation of China[62050194] ; National Natural Science Foundation of China[62037001] ; Innovative Research Group of the National Natural Science Foundation of China[61721002] ; Innovation Research Team of Ministry of Education[IRT_17R86] ; Project of China Knowledge Centre for Engineering Science and Technology ; Project of XJTU Undergraduate Teaching Reform[20JX04Y]
项目资助者	National Key Research and Development Project of China ; National Natural Science Foundation of China ; Innovative Research Group of the National Natural Science Foundation of China ; Innovation Research Team of Ministry of Education ; Project of China Knowledge Centre for Engineering Science and Technology ; Project of XJTU Undergraduate Teaching Reform
WOS研究方向	Computer Science
WOS类目	Computer Science, Artificial Intelligence
WOS记录号	WOS:000869199700001
出版者	SPRINGER
引用统计	被引频次：18[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://ir.hfcas.ac.cn:8080/handle/334002/129318
专题	中国科学院合肥物质科学研究院
通讯作者	Cao, Xiangyong
作者单位	1.Chinese Acad Sci, Hefei Inst Phys Sci, Hefei, Peoples R China 2.Univ Sci & Technol China, Hefei, Peoples R China 3.Nanjing Univ Sci & Technol, Nanjing, Peoples R China 4.Chinese Acad Sci, Inst Informat Engn, Beijing, Peoples R China 5.Xi An Jiao Tong Univ, Sch Math & Stat, Xian, Peoples R China 6.Xi An Jiao Tong Univ, Sch Comp Sci & Technol, Xian, Peoples R China 7.Xi An Jiao Tong Univ, Minist Educ, Key Lab Intelligent Networks & Network Secur, Xian, Peoples R China
推荐引用方式 GB/T 7714	Zhou, Man,Yan, Keyu,Pan, Jinshan,et al. Memory-Augmented Deep Unfolding Network for Guided Image Super-resolution[J]. INTERNATIONAL JOURNAL OF COMPUTER VISION,2022.
APA	Zhou, Man,Yan, Keyu,Pan, Jinshan,Ren, Wenqi,Xie, Qi,&Cao, Xiangyong.(2022).Memory-Augmented Deep Unfolding Network for Guided Image Super-resolution.INTERNATIONAL JOURNAL OF COMPUTER VISION.
MLA	Zhou, Man,et al."Memory-Augmented Deep Unfolding Network for Guided Image Super-resolution".INTERNATIONAL JOURNAL OF COMPUTER VISION (2022).