Affiliations: [a] The College of IoT Engineering, Hohai University, Jiangsu, China | [b] Department of Stomatology, Affiliated Third Hospital of Soochow University, Suzhou, Jiangsu, China | [c] The First People’s Hospital of Changzhou, Changzhou, Jiangsu, China
Correspondence:
[*]
Corresponding author: Zhengyan Sheng, The College of IoT Engineering, Hohai University, Jiangsu, China. E-mail: [email protected].
Abstract: In order to improve the robustness of speech recognition systems, this study attempts to classify stressed speech caused by the psychological stress under multitasking workloads. Due to the transient nature and ambiguity of stressed speech, the stress characteristics is not represented in all the segments in stressed speech as labeled. In this paper, we propose a multi-feature fusion model based on the attention mechanism to measure the importance of segments for stress classification. Through the attention mechanism, each speech frame is weighted to reflect the different correlations to the actual stressed state, and the multi-channel fusion of features characterizing the stressed speech to classify the speech under stress. The proposed model further adopts SpecAugment in view of the feature spectrum for data augment to resolve small sample sizes problem among stressed speech. During the experiment, we compared the proposed model with traditional methods on CASIA Chinese emotion corpus and Fujitsu stressed speech corpus, and results show that the proposed model has better performance in speaker-independent stress classification. Transfer learning is also performed for speaker-dependent classification for stressed speech, and the performance is improved. The attention mechanism shows the advantage for continuous speech under stress in authentic context comparing with traditional methods.
Keywords: Attention mechanism, speech under stress, multi-feature fusion, SpecAugment, transfer learning
