Singing Voice Synthesis via Guided Diffusion

Li S, Liu S, Lu W. Singing Voice Synthesis via Guided Diffusion. J Audio Eng Soc. 2026;74(11):811-824. doi:10.17743/jaes.2026.0303

@article{Li2026_23501,
  author = {Li, Sijia and Liu, Shiguang and Lu, Wenhuan},
  title = {{Singing Voice Synthesis via Guided Diffusion}},
  journal = {Journal of the Audio Engineering Society},
  volume = {74},
  number = {11},
  pages = {811--824},
  url = {https://doi.org/10.17743/jaes.2026.0303},
  doi = {10.17743/jaes.2026.0303},
  year = {2026},
  month = nov,
  publisher = {Audio Engineering Society}
}

TY  - JOUR
TI  - Singing Voice Synthesis via Guided Diffusion
AU  - Li, Sijia
AU  - Liu, Shiguang
AU  - Lu, Wenhuan
T2  - Journal of the Audio Engineering Society
J2  - J. Audio Eng. Soc.
VL  - 74
IS  - 11
SP  - 811
EP  - 824
PY  - 2026
DA  - 2026/11/10
DO  - 10.17743/jaes.2026.0303
UR  - https://doi.org/10.17743/jaes.2026.0303
PB  - Audio Engineering Society
LA  - en
AB  - Singing voice synthesis has emerged as a popular research area recently. Different from standard text-to-speech, it demands simultaneous adherence to both lyrical and musical constraints. This multiconditional nature makes singing voice synthesis systems highly dependent on accurately annotated datasets, making data preparation laborious and expensive. Therefore, the authors propose a novel dual-classifier guided diffusion framework that employs guidance functions for a pretrained, unconditional diffusion model, reducing the dependence on precise labels during diffusion training. This approach consists of an unconditional diffusion model and two classifiers as external guidance: a phoneme classifier to ensure accurate pronunciation and a pitch classifier to adjust pitch alignment. Techniques like gradient conflict projection are also explored and validated to harmonize the potentially conflicting gradients from the two classifiers. The authors investigate different guidance strategies, including the noisy-space and the more general clean-space guidance, which estimate gradients in noisy and clean feature spaces, respectively. Experimental results demonstrate that this method is capable of generating singing voices without annotations during diffusion training, providing a more convenient solution for the development of virtual singer applications.
ER  -