AES E-Library

← Back to search

Conference Paper

Are Traditional Evaluation Methods Still Reliable for Neural Audio Codecs?

Authors: Heinmüller, Alexander; Brendel, Andreas; Delgado, Pablo; Herre, Juergen

AES Show 2026 Nashville · Paper 37 · October 2026

Abstract

Neural audio codecs (NACs) represent a fundamentally different approach to audio compression compared to established signal processing-based codecs. Due to their relative novelty, there is currently no common consensus
on best practices in their evaluation. For application in commercial services, it is of special interest to guarantee that audio codec quality does not drop below a certain guaranteed level independently of the input signal type. Therefore, audio codecs are traditionally evaluated on sets of test items that contain both generic and critical items to achieve a balanced and comprehensive system evaluation. This raises the question of whether the same approach and test items remain sufficient for NACs. This paper investigates this question by conducting a side-by-side performance evaluation of two classic audio codecs (CACs) and three well-known NACs with subjective and objective methods. We conduct a MUSHRA listening test with both generic and critical items and a P.808 DCR test with a larger, less critical test set. We state that both test methods lead to similar quality rankings. Experiments further show interesting results in that strong transients, a typical CAC worst-case signal, are usually not challenging for several of the evaluated NACs while highly tonal signals remain critical items. NACs can exhibit types of artifacts that have not been observed before with CACs, such as hallucination effects. When re-evaluating the reliability of
various state-of-the-art perceptual quality measures, this behavior was found to make quality prediction difficult for most investigated measurement schemes.

Details

Published in
AES Show 2026 Nashville
Paper number
37
Publication date
October 30, 2026
Session subject
Artificial Intelligence & Machine LearningAudio Coding & CompressionAudio Quality
Affiliation
International Audio Laboratories Erlangen; Fraunhofer IIS; Fraunhofer IIS; Fraunhofer Institute for Integrated Circuits IIS; International Audio Laboratories Erlangen (See document for exact affiliation information.)
Type
Conference Paper

Save to list