Real-time Speech Emotion Recognition for Human-robot Interaction

Jun J, Kim HK. Real-time Speech Emotion Recognition for Human-robot Interaction. In: Express Paper. Audio Engineering Society; 2024. Paper 288. Available from: https://aes.org/publications/elibrary-page/?id=22745

@inproceedings{Jun2024_22745,
  author = {Jun, Jimin and Kim, Hong Kook},
  title = {{Real-time Speech Emotion Recognition for Human-robot Interaction}},
  note = {Paper 288},
  year = {2024},
  month = sep,
  publisher = {Audio Engineering Society},
  url = {https://aes.org/publications/elibrary-page/?id=22745}
}

TY  - CPAPER
TI  - Real-time Speech Emotion Recognition for Human-robot Interaction
AU  - Jun, Jimin
AU  - Kim, Hong Kook
M1  - Paper 288
PY  - 2024
DA  - 2024/09/27
UR  - https://aes.org/publications/elibrary-page/?id=22745
PB  - Audio Engineering Society
LA  - en
AB  - In this paper, we propose a novel method for real-time speech emotion recognition (SER) tailored for human-robot interaction. Traditional SER techniques, which analyze entire utterances, often struggle in real-time scenarios due to their high latency. To overcome this challenge, the proposed method breaks down speech into short, overlapping segments and uses a soft voting mechanism to aggregate emotion probabilities in real time. The proposed real-time method is applied to an SER model comprising the pre-trained wav2vec 2.0 and a convolutional network for feature extraction and emotion classification, respectively. The performance of the proposed method was evaluated on the KEMDy19 dataset, a Korean emotion dataset focusing on four key emotions: anger, happiness, neutrality, and sadness. Consequently, applying the real-time method, which processed each segment with a duration of 0.5 or 3.0 seconds, resulted in relative reduction of unweighted accuracy by 10.61% or 5.08%, respectively, compared to the method that processed entire utterances. However, the real-time factor (RTF) was significantly improved.
ER  -