Continuous Speech Emotion Recognition with Convolutional Neural Networks

Vryzas N, Vrysis L, Matsiola M, Kotsakis R, Dimoulas C, Kalliris G. Continuous Speech Emotion Recognition with Convolutional Neural Networks. J Audio Eng Soc. 2020;68(1/2):14-24. doi:10.17743/jaes.2019.0043

@article{Vryzas2020_20714,
  author = {Vryzas, Nikolaos and Vrysis, Lazaros and Matsiola, Maria and Kotsakis, Rigas and Dimoulas, Charalampos and Kalliris, George},
  title = {{Continuous Speech Emotion Recognition with Convolutional Neural Networks}},
  journal = {Journal of the Audio Engineering Society},
  volume = {68},
  number = {1/2},
  pages = {14--24},
  year = {2020},
  month = jan,
  publisher = {Audio Engineering Society},
  doi = {10.17743/jaes.2019.0043},
  url = {https://doi.org/10.17743/jaes.2019.0043}
}

TY  - JOUR
TI  - Continuous Speech Emotion Recognition with Convolutional Neural Networks
AU  - Vryzas, Nikolaos
AU  - Vrysis, Lazaros
AU  - Matsiola, Maria
AU  - Kotsakis, Rigas
AU  - Dimoulas, Charalampos
AU  - Kalliris, George
T2  - Journal of the Audio Engineering Society
J2  - J. Audio Eng. Soc.
VL  - 68
IS  - 1/2
SP  - 14
EP  - 24
PY  - 2020
DA  - 2020/01/06
DO  - 10.17743/jaes.2019.0043
UR  - https://doi.org/10.17743/jaes.2019.0043
PB  - Audio Engineering Society
LA  - en
AB  - Emotional speech is a separate channel of communication that carries the paralinguistic aspects of spoken language. Affective information knowledge can be crucial for contextual speech recognition, which can also provide elements from the personality and psychological state of the speaker enriching the communication. That kind of data may play an important role as semantic analysis features of web content and would also apply in intelligent affective new media and social interaction domains. A model for Speech Emotion Recognition (SER), based on Convolutional Neural Networks (CNN) architecture is proposed and evaluated. Recognition is performed on successive time frames of continuous speech. The dataset used for training and testing the model is the Acted Emotional Speech Dynamic Database (AESDD), a publicly available corpus in the Greek language. Experiments involving the subjective evaluation of the AESDD are presented to serve as a reference for human-level recognition accuracy. The proposed CNN architecture outperforms previous baseline machine learning models (Support Vector Machines) by 8.4% in terms of accuracy and it is also more efficient because it bypasses the stage of handcrafted feature extraction. Data augmentation of the database did not affect classification accuracy in the validation tests but is expected to improve robustness and generalization. Besides performance improvements, the unsupervised feature-extraction stage of the proposed topology also makes it feasible to create real-time systems.
ER  -