Reverse Engineering a Nonlinear Mix of a Multitrack Recording

Colonel JT, Reiss J. Reverse Engineering a Nonlinear Mix of a Multitrack Recording. J Audio Eng Soc. 2023;71(9):586-595. doi:10.17743/jaes.2022.0105

@article{Colonel2023_22234,
  author = {Colonel, Joseph T and Reiss, Joshua},
  title = {{Reverse Engineering a Nonlinear Mix of a Multitrack Recording}},
  journal = {Journal of the Audio Engineering Society},
  volume = {71},
  number = {9},
  pages = {586--595},
  year = {2023},
  month = sep,
  publisher = {Audio Engineering Society},
  doi = {10.17743/jaes.2022.0105},
  url = {https://doi.org/10.17743/jaes.2022.0105}
}

TY  - JOUR
TI  - Reverse Engineering a Nonlinear Mix of a Multitrack Recording
AU  - Colonel, Joseph T
AU  - Reiss, Joshua
T2  - Journal of the Audio Engineering Society
J2  - J. Audio Eng. Soc.
VL  - 71
IS  - 9
SP  - 586
EP  - 595
PY  - 2023
DA  - 2023/09/06
DO  - 10.17743/jaes.2022.0105
UR  - https://doi.org/10.17743/jaes.2022.0105
PB  - Audio Engineering Society
LA  - en
AB  - In the field of intelligent audio production, neural networks have been trained to automatically mix a multitrack to a stereo mixdown. Although these algorithms contain latent models of mix engineering, there is still a lack of approaches that explicitly model the decisions a mix engineer makes while mixing. In this work, a method to retrieve the parameters used to create a multitrack mix using only raw tracks and the stereo mixdown is presented. This method is able to model a multitrack mix using gain, panning, equalization, dynamic range compression, distortion, delay, and reverb with the aid of greybox differentiable digital signal processing modules. This method allows for a fully interpretable representation of the mixing signal chain by explicitly modeling the audio effects one may expect in a typical engineer's mixing chain. The modeling capacities of several different mixing chains are measured using both objective and subjective measures on a dataset of student mixes. Results show that the full signal chain performs best on objective measures and that there is no statistically significant difference between the participants' perception of the full mixing chain and reference mixes.
ER  -