HFCAS OpenIR  > 中科院合肥智能机械研究所
A realistic 3D articulatory animation system for emotional visual pronunciation
Yu, Lingyun1; Yu, Jun1; Wang, Zengfu1,2
2017-09-01
Source PublicationMULTIMEDIA TOOLS AND APPLICATIONS
Volume76Issue:18Pages:19241-19262
AbstractThis paper proposes a realistic 3D articulatory animation system for emotional visual pronunciation driven by the emotional articulatory movement trajectory. Firstly, the articulatory movements, recorded by Electro-Magnetic Articulatory (EMA), are trained by Hidden Markov Model (HMM) while the fully context-dependent model is taken into account by making full use of the rich linguistic features. Secondly, owing to the independency in the manipulation of articulators, the articulatory movements are more remarkably adjusted to express different emotions. Thirdly, the emotional speech is generated by adjusting the neutral speech parameters, such as fundamental frequency (F0), duration and intensity, based on Praat. Then the corresponding articulatory movements are synthesized by the HMM prediction rules which are used to drive the head mesh model along with the emotional speech simultaneously. The experiments intend to synthesize accurate emotional speech synchronized animation of articulators based on the system at phoneme level.
SubtypeArticle
KeywordArticulatory Movement Hidden Markov Model Fully Context-dependent Modeling Emotional Speech
WOS HeadingsScience & Technology ; Technology
Funding OrganizationNational Natural Science Foundation of China(61572450 ; National Natural Science Foundation of China(61572450 ; Anhui Provincial Natural Science Foundation(1708085QF138) ; Anhui Provincial Natural Science Foundation(1708085QF138) ; Fundamental Research Funds for the Central Universities(WK2350000002) ; Fundamental Research Funds for the Central Universities(WK2350000002) ; Open Funding Project of State Key Lab of Virtual Reality Technology and Systems, Beihang University(BUAA-VR-16KF-12) ; Open Funding Project of State Key Lab of Virtual Reality Technology and Systems, Beihang University(BUAA-VR-16KF-12) ; Open Funding Project of State Key Lab of Novel Software Technology, Nanjing University(KFKT2016B08) ; Open Funding Project of State Key Lab of Novel Software Technology, Nanjing University(KFKT2016B08) ; 61303150) ; 61303150) ; National Natural Science Foundation of China(61572450 ; National Natural Science Foundation of China(61572450 ; Anhui Provincial Natural Science Foundation(1708085QF138) ; Anhui Provincial Natural Science Foundation(1708085QF138) ; Fundamental Research Funds for the Central Universities(WK2350000002) ; Fundamental Research Funds for the Central Universities(WK2350000002) ; Open Funding Project of State Key Lab of Virtual Reality Technology and Systems, Beihang University(BUAA-VR-16KF-12) ; Open Funding Project of State Key Lab of Virtual Reality Technology and Systems, Beihang University(BUAA-VR-16KF-12) ; Open Funding Project of State Key Lab of Novel Software Technology, Nanjing University(KFKT2016B08) ; Open Funding Project of State Key Lab of Novel Software Technology, Nanjing University(KFKT2016B08) ; 61303150) ; 61303150)
DOI10.1007/s11042-017-4578-0
WOS KeywordSPEECH SYNTHESIS ; MOVEMENTS
Indexed BySCI
Language英语
Funding OrganizationNational Natural Science Foundation of China(61572450 ; National Natural Science Foundation of China(61572450 ; Anhui Provincial Natural Science Foundation(1708085QF138) ; Anhui Provincial Natural Science Foundation(1708085QF138) ; Fundamental Research Funds for the Central Universities(WK2350000002) ; Fundamental Research Funds for the Central Universities(WK2350000002) ; Open Funding Project of State Key Lab of Virtual Reality Technology and Systems, Beihang University(BUAA-VR-16KF-12) ; Open Funding Project of State Key Lab of Virtual Reality Technology and Systems, Beihang University(BUAA-VR-16KF-12) ; Open Funding Project of State Key Lab of Novel Software Technology, Nanjing University(KFKT2016B08) ; Open Funding Project of State Key Lab of Novel Software Technology, Nanjing University(KFKT2016B08) ; 61303150) ; 61303150) ; National Natural Science Foundation of China(61572450 ; National Natural Science Foundation of China(61572450 ; Anhui Provincial Natural Science Foundation(1708085QF138) ; Anhui Provincial Natural Science Foundation(1708085QF138) ; Fundamental Research Funds for the Central Universities(WK2350000002) ; Fundamental Research Funds for the Central Universities(WK2350000002) ; Open Funding Project of State Key Lab of Virtual Reality Technology and Systems, Beihang University(BUAA-VR-16KF-12) ; Open Funding Project of State Key Lab of Virtual Reality Technology and Systems, Beihang University(BUAA-VR-16KF-12) ; Open Funding Project of State Key Lab of Novel Software Technology, Nanjing University(KFKT2016B08) ; Open Funding Project of State Key Lab of Novel Software Technology, Nanjing University(KFKT2016B08) ; 61303150) ; 61303150)
WOS Research AreaComputer Science ; Engineering
WOS SubjectComputer Science, Information Systems ; Computer Science, Software Engineering ; Computer Science, Theory & Methods ; Engineering, Electrical & Electronic
WOS IDWOS:000407861800050
Citation statistics
Document Type期刊论文
Identifierhttp://ir.hfcas.ac.cn:8080/handle/334002/33566
Collection中科院合肥智能机械研究所
Affiliation1.Univ Sci & Technol China, Dept Automat, Hefei, Anhui, Peoples R China
2.Chinese Acad Sci, Inst Intelligent Machines, Hefei, Anhui, Peoples R China
Recommended Citation
GB/T 7714
Yu, Lingyun,Yu, Jun,Wang, Zengfu. A realistic 3D articulatory animation system for emotional visual pronunciation[J]. MULTIMEDIA TOOLS AND APPLICATIONS,2017,76(18):19241-19262.
APA Yu, Lingyun,Yu, Jun,&Wang, Zengfu.(2017).A realistic 3D articulatory animation system for emotional visual pronunciation.MULTIMEDIA TOOLS AND APPLICATIONS,76(18),19241-19262.
MLA Yu, Lingyun,et al."A realistic 3D articulatory animation system for emotional visual pronunciation".MULTIMEDIA TOOLS AND APPLICATIONS 76.18(2017):19241-19262.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Yu, Lingyun]'s Articles
[Yu, Jun]'s Articles
[Wang, Zengfu]'s Articles
Baidu academic
Similar articles in Baidu academic
[Yu, Lingyun]'s Articles
[Yu, Jun]'s Articles
[Wang, Zengfu]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Yu, Lingyun]'s Articles
[Yu, Jun]'s Articles
[Wang, Zengfu]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.