Affiliations: [a] School of Electrical Engineering, Vellore Institute Technology Vellore, India
| [b]
Saint-Petersburg State University, Russia | [c] School of Information Technology and Engineering, Vellore Institute Technology Vellore, India
| [d] Key Laboratory of Digital Signal and Image Processing of Guangdong Province, Department of Electronics Engineering, College of Engineering, Shantou University, China
Correspondence:
[*]
Corresponding author. Nersisson Ruban, Associate Professor, School of Electrical Engineering, Vellore Institute of Technology, Vellore, 632014, India. E-mail: [email protected].
Abstract: Over the years the need for differentiating various emotions from oral communication plays an important role in emotion based studies. There have been different algorithms to classify the kinds of emotion. Although there is no measure of fidelity of the emotion under consideration, which is primarily due to the reason that most of the readily available datasets that are annotated are produced by actors and not generated in real-world scenarios. Therefore, the predicted emotion lacks an important aspect called authenticity, which is whether an emotion is actual or stimulated. In this research work, we have developed a transfer learning and style transfer based hybrid convolutional neural network algorithm to classify the emotion as well as the fidelity of the emotion. The model is trained on features extracted from a dataset that contains stimulated as well as actual utterances. We have compared the developed algorithm with conventional machine learning and deep learning techniques by few metrics like accuracy, Precision, Recall and F1 score. The developed model performs much better than the conventional machine learning and deep learning models. The research aims to dive deeper into human emotion and make a model that understands it like humans do with precision, recall, F1 score values of 0.994, 0.996, 0.995 for speech authenticity and 0.992, 0.989, 0.99 for speech emotion classification respectively.
Keywords: Deep learning, speech fidelity classification, linear prediction cepstral coefficients (LPCC), mel frequency cepstral coefficients (MFCC), speech emotion recognition
