Real-Time Audio Pattern Detection for Smart Musical Instruments

Silva N, Turchet L. Real-Time Audio Pattern Detection for Smart Musical Instruments. J Audio Eng Soc. 2026;74(3):130-140. doi:10.17743/jaes.2022.0250

@article{Silva2026_23129,
  author = {Silva, Nishal and Turchet, Luca},
  title = {{Real-Time Audio Pattern Detection for Smart Musical Instruments}},
  journal = {Journal of the Audio Engineering Society},
  volume = {74},
  number = {3},
  pages = {130--140},
  year = {2026},
  month = mar,
  publisher = {Audio Engineering Society},
  doi = {10.17743/jaes.2022.0250},
  url = {https://doi.org/10.17743/jaes.2022.0250}
}

TY  - JOUR
TI  - Real-Time Audio Pattern Detection for Smart Musical Instruments
AU  - Silva, Nishal
AU  - Turchet, Luca
T2  - Journal of the Audio Engineering Society
J2  - J. Audio Eng. Soc.
VL  - 74
IS  - 3
SP  - 130
EP  - 140
PY  - 2026
DA  - 2026/03/06
DO  - 10.17743/jaes.2022.0250
UR  - https://doi.org/10.17743/jaes.2022.0250
PB  - Audio Engineering Society
LA  - en
AB  - This paper presents an algorithm for real-time polyphonic audio pattern detection designed to be integrated into smart musical instruments. Such instruments will enable musicians to use predefined musical patterns as triggers for external peripherals, such as stage lights, haptic wearables, and mixed reality headsets, during live performances. The paper first introduces a data set of polyphonic patterns containing musical patterns with expressive variations recorded by 20 musicians. Secondly, it presents an approach to classify audio segments in real time, consisting of a recurrent neural network architecture. Thirdly, it compares the proposed method against a traditional dynamic time warping approach as a benchmark. Results show that, although the benchmark achieves higher precision (0.8 versus 0.74), the proposed method yields a better overall performance, with an F1 score of 0.76 compared with 0.65. Most importantly, the proposed method exhibits a lower computational inference time (14 ms versus 1038 ms) and reduced CPU usage (31.36% versus 48.9%) when deployed on a Raspberry Pi 4. The low inference time meets the latency requirements necessary for real-time musical applications, and the system demonstrates the feasibility of embedded intelligence in musical instruments that can recognize polyphonic patterns with expressive variations during live performances.
ER  -