A Scalable Two-Stage Automatic Mixing System Integrating Machine Learning and Domain Knowledge

Shi J, Xie K, Ma Y, Reiss J. A Scalable Two-Stage Automatic Mixing System Integrating Machine Learning and Domain Knowledge. In: Convention Paper. Audio Engineering Society; 2025. Paper 10232. Available from: https://aes.org/publications/elibrary-page/?id=23076

@inproceedings{Shi2025_23076,
  author = {Shi, Jinjie and Xie, Kunzhu and Ma, Yinghao and Reiss, Joshua},
  title = {{A Scalable Two-Stage Automatic Mixing System Integrating Machine Learning and Domain Knowledge}},
  note = {Paper 10232},
  year = {2025},
  month = oct,
  publisher = {Audio Engineering Society},
  url = {https://aes.org/publications/elibrary-page/?id=23076}
}

TY  - CPAPER
TI  - A Scalable Two-Stage Automatic Mixing System Integrating Machine Learning and Domain Knowledge
AU  - Shi, Jinjie
AU  - Xie, Kunzhu
AU  - Ma, Yinghao
AU  - Reiss, Joshua
M1  - Paper 10232
PY  - 2025
DA  - 2025/10/14
UR  - https://aes.org/publications/elibrary-page/?id=23076
PB  - Audio Engineering Society
LA  - en
AB  - Music mixing involves transforming clean, individual tracks into a cohesive final mix using audio effects and expert knowledge. While rule-based and machine learning methods have shown promise, scaling them to real-world situations remains challenging. We propose a two-stage mixing architecture that combines domain knowledge with deep learning, enabling the system to handle over 100 input tracks with high perceptual quality. The first stage uses a rule-based level balancing system to mix grouped tracks into stems. The second stage employs a differentiable mixing style transfer model guided by a reference mix. To enhance intra-group (within subgroup) robustness, we refine loudness estimation by incorporating spectral centroid and fundamental frequency features, addressing limitations of Loudness Units relative to Full Scale (LUFS) on narrowband signals. Subjective listening tests demonstrate that our enhanced intra-group mixing approach consistently outperforms LUFS-based baselines across multiple musical genres. Furthermore, our proposed two-step system enables deep learning to successfully handle projects with over 100 tracks for the first time, achieving mixing results that significantly surpass those of traditional rule-based systems. Code and audio examples are available at https://doi.org/10.5281/zenodo.17171082.
ER  -