A convolution-transformer dual branch network for head-pose and occlusion facial expression recognition

doi:10.1007/s00371-022-02413-5

HFCAS OpenIR

	A convolution-transformer dual branch network for head-pose and occlusion facial expression recognition
	Liang, Xingcan 1,2; Xu, Linsen5 ; Zhang, Wenxiang 3; Zhang, Yan 4; Liu, Jinfu 1,2; Liu, Zhipeng 1,2
	2022-02-13
发表期刊	VISUAL COMPUTER
ISSN	0178-2789
通讯作者	Xu, Linsen(lsxu@hhu.edu.cn)
摘要	Facial expression recognition (FER) has attracted much more attention due to its broad range of applications. Occlusions and head-pose variations are two major obstacles for automatic FER. In this paper, we propose a convolution-transformer dual branch network (CT-DBN) that takes advantage of local and global facial information to tackle the real-word occlusions and head-pose variant robust FER. The CT-DBN contains two branches. Taking into account local modeling ability of CNN, the first branch utilizes CNN to capture local edge information. Inspired by transformers' successful application in natural language processing, we employ transformer to the second branch to be responsible for obtaining better global representation. Then, a local-global feature fusion module is proposed to adaptively integrate both features to hybrid features and model the relationship between them. With the help of feature fusion module, our network not only integrates local and global features in an adaptive weighting manner but can also learn the corresponding distinguishable features autonomously. Experimental results under inner-database and cross-database evaluation on four leading facial expression databases illustrate that our proposed CT-DBN outperforms other state-of-the-art methods and achieves robust performance under in-the-wild condition.
关键词	Facial expression recognition CNNs Transformers Feature fusion Robust on occlusions and head-pose variations
DOI	10.1007/s00371-022-02413-5
收录类别	SCI
语种	英语
资助项目	National Key R&D Program of China[2017YFB1303200] ; Jiangsu Special Project for Frontier Leading Base Technology[BK20192004] ; Key Support Project of Dean Fund of Hefei Institutes of Physical Science, CAS[YZJJZX202017] ; Strategic High-tech Innovation Fund of Chinese Academy of Sciences[GQRC-19-15]
项目资助者	National Key R&D Program of China ; Jiangsu Special Project for Frontier Leading Base Technology ; Key Support Project of Dean Fund of Hefei Institutes of Physical Science, CAS ; Strategic High-tech Innovation Fund of Chinese Academy of Sciences
WOS研究方向	Computer Science
WOS类目	Computer Science, Software Engineering
WOS记录号	WOS:000754455000001
出版者	SPRINGER
引用统计	被引频次：17[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://ir.hfcas.ac.cn:8080/handle/334002/127809
专题	中国科学院合肥物质科学研究院
通讯作者	Xu, Linsen
作者单位	1.Chinese Acad Sci, Inst Intelligent Machines, Hefei Inst Phys Sci, Hefei 230031, Peoples R China 2.Univ Sci & Technol China, Hefei 230026, Peoples R China 3.Changzhou Univ, Sch Microelect & Control Engn, Changzhou 213164, Peoples R China 4.Anhui Jianzhu Univ, Sch Elect & Informat Engn, Hefei 230009, Peoples R China 5.Hohai Univ, Coll Mech & Elect Engn, Changzhou 213022, Peoples R China
推荐引用方式 GB/T 7714	Liang, Xingcan,Xu, Linsen,Zhang, Wenxiang,et al. A convolution-transformer dual branch network for head-pose and occlusion facial expression recognition[J]. VISUAL COMPUTER,2022.
APA	Liang, Xingcan,Xu, Linsen,Zhang, Wenxiang,Zhang, Yan,Liu, Jinfu,&Liu, Zhipeng.(2022).A convolution-transformer dual branch network for head-pose and occlusion facial expression recognition.VISUAL COMPUTER.
MLA	Liang, Xingcan,et al."A convolution-transformer dual branch network for head-pose and occlusion facial expression recognition".VISUAL COMPUTER (2022).