On the Lack of a Perceptually Motivated Evaluation Metric for Packet Loss Concealment in Networked Music Performances

Vignati L, Turchet L. On the Lack of a Perceptually Motivated Evaluation Metric for Packet Loss Concealment in Networked Music Performances. J Audio Eng Soc. 2025;73(10):660-670. doi:10.17743/jaes.2022.0227

@article{Vignati2025_23028,
  author = {Vignati, Luca and Turchet, Luca},
  title = {{On the Lack of a Perceptually Motivated Evaluation Metric for Packet Loss Concealment in Networked Music Performances}},
  journal = {Journal of the Audio Engineering Society},
  volume = {73},
  number = {10},
  pages = {660--670},
  year = {2025},
  month = oct,
  publisher = {Audio Engineering Society},
  doi = {10.17743/jaes.2022.0227},
  url = {https://doi.org/10.17743/jaes.2022.0227}
}

TY  - JOUR
TI  - On the Lack of a Perceptually Motivated Evaluation Metric for Packet Loss Concealment in Networked Music Performances
AU  - Vignati, Luca
AU  - Turchet, Luca
T2  - Journal of the Audio Engineering Society
J2  - J. Audio Eng. Soc.
VL  - 73
IS  - 10
SP  - 660
EP  - 670
PY  - 2025
DA  - 2025/10/01
DO  - 10.17743/jaes.2022.0227
UR  - https://doi.org/10.17743/jaes.2022.0227
PB  - Audio Engineering Society
LA  - en
AB  - Packet loss concealment (PLC) is vital in preserving audio quality for networked music performances. Although existing PLC techniques primarily target speech transmission, the unique challenges in music signals, such as complex harmonic structures and diverse timbral ranges, have yet to be adequately addressed. This is in part a result of the fact that a satisfactory objective evaluation metric for music PLC methods is missing. As a first foundational step toward this direction, this paper proposes a novel evaluation metric that leverages insights from music psychoacoustics and uses the constant-Q transform to better quantify glitch audibility induced by unconcealed packet loss (i.e., replaced with zeros) compared with existing metrics. The authors conducted extensive subjective listening tests leading to the creation of a publicly available ground truth data set, mapping objective audio features to human assessments of glitch audibility. Results show that the developed metric outperforms other measures (such as mean squared error and mean absolute error) in predicting perceptual impacts, taking a step toward addressing the need for a specialized metric for PLC in the domain of networked music performances. However, further improvements are needed to match human perceptual accuracy, which calls for further research on the development of a reliable perceptually motivated evaluation metric.
ER  -