Objective Quality Models for Decision-Making in Speech Coding

Lamballe C, Gournay P, Lefebvre R. Objective Quality Models for Decision-Making in Speech Coding. In: Express Paper. Audio Engineering Society; 2026. Paper 430. Available from: https://aes.org/publications/elibrary-page/?id=23156

@inproceedings{Lamballe2026_23156,
  author = {Lamballe, Clémence and Gournay, Philippe and Lefebvre, Roch},
  title = {{Objective Quality Models for Decision-Making in Speech Coding}},
  note = {Paper 430},
  year = {2026},
  month = may,
  publisher = {Audio Engineering Society},
  url = {https://aes.org/publications/elibrary-page/?id=23156}
}

TY  - CPAPER
TI  - Objective Quality Models for Decision-Making in Speech Coding
AU  - Lamballe, Clémence
AU  - Gournay, Philippe
AU  - Lefebvre, Roch
M1  - Paper 430
PY  - 2026
DA  - 2026/05/28
UR  - https://aes.org/publications/elibrary-page/?id=23156
PB  - Audio Engineering Society
LA  - en
AB  - Objective quality evaluation is widely used in speech coding, yet objective estimates often exhibit limited agreement with subjective listening-test results. Rather than focusing on absolute score accuracy, this paper evaluates objective speech quality models from a decision-making perspective, defined as their ability to support comparative judgments between speech codecs or codec configurations. A formal ITU-R P.800 Absolute Category Rating (ACR) listening test was conducted with 30 listeners across 24 conditions, covering conventional and neural monophonic speech codecs operating under clear-channel conditions at sampling frequencies from 16 to 48 kHz and bitrates ranging from below 1 kbps to above 16 kbps. The speech material consisted of internally recorded, clean French-language speech that was not used in the development or training of any of the evaluated codecs or objective quality models. Seven objective quality models, namely PESQ, VISQOL Speech, VISQOL Audio, WARP-Q, NISQA, UTMOS, and DistillMOS, were evaluated on the same material. Decision-making performance was assessed by comparing subjective and objective rankings using Kendalls rank correlation coefficient and by assessing the consistency of statistical discrimination across pairwise codec comparisons using two-sided t-tests at a 95% confidence level. The results indicate that some objective quality models are effective for comparing variations within a given speech coding technology under consistent operating conditions, such as fixed sampling frequency. However, all models exhibit limitations, including systematic over- or underestimation for certain coding technologies and reduced reliability when applied across different sampling frequencies. Despite its conventional origins, PESQ remains capable of supporting decision-making when applied to neural speech codecs.
ER  -