Multichannel speaker diarization with arbitrary microphone arrays

Xylogiannis P, Vryzas N, Bountourakis V, Dimoulas C. Multichannel speaker diarization with arbitrary microphone arrays. In: AES Convention 154. Audio Engineering Society; 2023. Paper 89. Available from: https://aes.org/publications/elibrary-page/?id=22114

@inproceedings{Xylogiannis2023_22114,
  author = {Xylogiannis, Paris and Vryzas, Nikolaos and Bountourakis, Vasileios and Dimoulas, Charalampos},
  title = {{Multichannel speaker diarization with arbitrary microphone arrays}},
  booktitle = {AES Convention 154},
  note = {Paper 89},
  year = {2023},
  month = may,
  publisher = {Audio Engineering Society},
  url = {https://aes.org/publications/elibrary-page/?id=22114}
}

TY  - CPAPER
TI  - Multichannel speaker diarization with arbitrary microphone arrays
AU  - Xylogiannis, Paris
AU  - Vryzas, Nikolaos
AU  - Bountourakis, Vasileios
AU  - Dimoulas, Charalampos
T2  - AES Convention 154
M1  - Paper 89
PY  - 2023
DA  - 2023/05/06
UR  - https://aes.org/publications/elibrary-page/?id=22114
PB  - Audio Engineering Society
LA  - en
AB  - Speaker diarization remains a field with potential for improvement. In meeting scenarios, the task of labeling audio with the corresponding speaker identities, can be further assisted with the exploitation of spatial features. In the present work, a framework is designed, to evaluate the combination of speaker embeddings with Time Difference Of Arrival (TDOA) values. Speaker embeddings are extracted using two popular pre-trained models, ECAPA – TDNN and Xvectors. TDOA values for every speech segment are calculated using the Generalized Cross Correlation (GCC) method with phase transform (PHAT) weights (GCC – PHAT). The outputs of GCC – PHAT and deep neural network (DNN) systems are fused by concatenation and used as the input to spectral clustering. The objective of the proposed framework is to evaluate the potential of exploiting available microphone arrays in meetings and the investigation of complementary information between TDOA and speaker embeddings. The system is evaluated on two different datasets, the AVLab Speaker Localization and a multichannel dataset created in the context of the present work. Furthermore, an additional dataset using mobile phones embedded microphones is created and openly distributed to assist research groups to find solutions to complex problems such as speaker localization and diarization with arbitrary arrays comprising microphones of different characteristics and quality.
ER  -