A Scalable AI Architecture for Audio and Multimodal Analysis on Mobile Devices: A Case of Environmental Monitoring

Stamatiadou ME, Mpesmerti A, Vryzas N, Vrysis L, Dimoulas C. A Scalable AI Architecture for Audio and Multimodal Analysis on Mobile Devices: A Case of Environmental Monitoring. In: 2025 AES International Conference on Artificial Intelligence and Machine Learning for Audio. Audio Engineering Society; 2025. Paper 13. Available from: https://aes.org/publications/elibrary-page/?id=23002

@inproceedings{Stamatiadou2025_23002,
  author = {Stamatiadou, Marina Eirini and Mpesmerti, Athanasia and Vryzas, Nikolaos and Vrysis, Lazaros and Dimoulas, Charalampos},
  title = {{A Scalable AI Architecture for Audio and Multimodal Analysis on Mobile Devices: A Case of Environmental Monitoring}},
  booktitle = {2025 AES International Conference on Artificial Intelligence and Machine Learning for Audio},
  note = {Paper 13},
  year = {2025},
  month = sep,
  publisher = {Audio Engineering Society},
  url = {https://aes.org/publications/elibrary-page/?id=23002}
}

TY  - CPAPER
TI  - A Scalable AI Architecture for Audio and Multimodal Analysis on Mobile Devices: A Case of Environmental Monitoring
AU  - Stamatiadou, Marina Eirini
AU  - Mpesmerti, Athanasia
AU  - Vryzas, Nikolaos
AU  - Vrysis, Lazaros
AU  - Dimoulas, Charalampos
T2  - 2025 AES International Conference on Artificial Intelligence and Machine Learning for Audio
M1  - Paper 13
PY  - 2025
DA  - 2025/09/02
UR  - https://aes.org/publications/elibrary-page/?id=23002
PB  - Audio Engineering Society
LA  - en
AB  - The increasing need for real-time environmental monitoring has made Artificial Intelligence (AI) and Machine Learning (ML) models for audio classification and multimodal sensing essential for detecting and analyzing pollution-related sounds. In cases where mobile devices are used for capturing and processing audiovisual content, such models offer significant potential for research, storytelling, and public awareness. This study proposes a scalable and modular architecture that enables direct and flexible access to AI models for multimodal and audio processing. A CNN-LSTM hybrid model is trained for real-time environmental sound classification and deployed as a service. Building on prior 1D CNN-based approaches and incorporating temporal dependencies, the new model achieves an AUC of 0.91, demonstrating improved accuracy and generalization. The system leverages a REST API and Docker-based containerization, allowing deployment of independent AI services and supporting mobile and IoT use cases. The architecture accommodates both pre-trained and custom models, accessible from any device via a unified interface, confirming that a hybrid CNN-LSTM topology can support effective real-time sound classification, within a modular, containerized framework.
ER  -