Methods of Assessing the Rank Order of Prediction Models with Respect to Variance of Listening Test Ratings

Pearce A, Isabelle S, Francois H, Oh E. Methods of Assessing the Rank Order of Prediction Models with Respect to Variance of Listening Test Ratings. J Audio Eng Soc. 2019;67(1/2):60-68. doi:10.17743/jaes.2018.0070

@article{Pearce2019_19887,
  author = {Pearce, Andy and Isabelle, Scott and Francois, Holly and Oh, Eunmi},
  title = {{Methods of Assessing the Rank Order of Prediction Models with Respect to Variance of Listening Test Ratings}},
  journal = {Journal of the Audio Engineering Society},
  volume = {67},
  number = {1/2},
  pages = {60--68},
  year = {2019},
  month = jan,
  publisher = {Audio Engineering Society},
  doi = {10.17743/jaes.2018.0070},
  url = {https://doi.org/10.17743/jaes.2018.0070}
}

TY  - JOUR
TI  - Methods of Assessing the Rank Order of Prediction Models with Respect to Variance of Listening Test Ratings
AU  - Pearce, Andy
AU  - Isabelle, Scott
AU  - Francois, Holly
AU  - Oh, Eunmi
T2  - Journal of the Audio Engineering Society
J2  - J. Audio Eng. Soc.
VL  - 67
IS  - 1/2
SP  - 60
EP  - 68
PY  - 2019
DA  - 2019/01/06
DO  - 10.17743/jaes.2018.0070
UR  - https://doi.org/10.17743/jaes.2018.0070
PB  - Audio Engineering Society
LA  - en
AB  - Although predictive models are widely used to predict the results of listening tests, there are currently no standardized statistical metrics for assessing the rank order. Commonly used rank-order metrics do not consider the variance of the listening test data. This paper proposes two novel metrics for assessing rank order with respect to variance by adapting Spearman’s Rho and Kendall’s Tau and assesses the performance of these metrics against actual listening test data with standardized prediction models.
ER  -