The Great Debate: Subjective Evaluation

Lipshitz SP, Vanderkooy J. The Great Debate: Subjective Evaluation. J Audio Eng Soc. 1981;29(7/8):482-491. Available from: https://aes.org/publications/elibrary-page/?id=3899

@article{Lipshitz1981_3899,
  author = {Lipshitz, Stanley P. and Vanderkooy, John},
  title = {{The Great Debate: Subjective Evaluation}},
  journal = {Journal of the Audio Engineering Society},
  volume = {29},
  number = {7/8},
  pages = {482--491},
  year = {1981},
  month = aug,
  publisher = {Audio Engineering Society},
  url = {https://aes.org/publications/elibrary-page/?id=3899}
}

TY  - JOUR
TI  - The Great Debate: Subjective Evaluation
AU  - Lipshitz, Stanley P.
AU  - Vanderkooy, John
T2  - Journal of the Audio Engineering Society
J2  - J. Audio Eng. Soc.
VL  - 29
IS  - 7/8
SP  - 482
EP  - 491
PY  - 1981
DA  - 1981/08/06
UR  - https://aes.org/publications/elibrary-page/?id=3899
PB  - Audio Engineering Society
LA  - en
AB  - A polarization of people has occurred regarding subjective evaluation, separating those who believe that audible differences are related to measurable differences in controlled tests, from those who believe that such differences have no direct relationship to measurements. Tests are necessary to resolve such differences of opinion, and to further the state of audio and open new areas of understanding. We argue that highly controlled tests are necessary to transform subjective evaluation to an objective plane so that preferences and bias can be eliminated, in the quest for determining the accuracy of an audio component. In order for subjective tests to be meaningful to others, the following should be observed. (1) There must be technical competence to prevent obvious and/or subtle effects from affecting the test. (2) Linear differences must be thoroughly excised before conclusions about nonlinear errors can be reached. (3) The subjective judgment required in the test must be simple, such as the ability to discriminate between two components, using an absolute reference wherever possible. (4) The test must be blind or preferably double-blind. To implement such tests we advocate the use of A/B switchboxes. The box itself can be tested for audibly intrusive effects, and several embellishments are described which allow double-blind procedures to be used in listening tests. We believe that the burden of proof must lie with those who make new hypotheses regarding subjective tests. This alone would wipe out most criticisms of the controlled tests reported in the literature. Speculation is changed to fact only by careful experimentation. Recent references are given which support out point of view. The significance of differences in audio components is discussed, and in conclusion we detail some of our tests, hypotheses and speculations.
ER  -