Do We Need a Source Separation for Automated Subtitling on Media Contents?

Lee H, Lee GW, Kim HK. Do We Need a Source Separation for Automated Subtitling on Media Contents? In: Express Paper. Audio Engineering Society; 2024. Paper 289. Available from: https://aes.org/publications/elibrary-page/?id=22746

@inproceedings{Lee2024_22746,
  author = {Lee, Hyun and Lee, Geon Woo and Kim, Hong Kook},
  title = {{Do We Need a Source Separation for Automated Subtitling on Media Contents?}},
  note = {Paper 289},
  year = {2024},
  month = sep,
  publisher = {Audio Engineering Society},
  url = {https://aes.org/publications/elibrary-page/?id=22746}
}

TY  - CPAPER
TI  - Do We Need a Source Separation for Automated Subtitling on Media Contents?
AU  - Lee, Hyun
AU  - Lee, Geon Woo
AU  - Kim, Hong Kook
M1  - Paper 289
PY  - 2024
DA  - 2024/09/27
UR  - https://aes.org/publications/elibrary-page/?id=22746
PB  - Audio Engineering Society
LA  - en
AB  - In this paper, we investigate the efficacy of a speech and music source separation technique on an automated subtitling system under different signal-to-noise ratios (SNRs). To this end, we compare the generated subtitle errors by measuring the word error rate (WER) with and without source separation applied to speech in music. Experiments are first performed on a dataset by mixing speech from the LibriSpeech dataset with music from the MUSDB18-HQ dataset. Accordingly, it is revealed that when the SNR is below 5 dB, using separated speech yields the lowest subtitle error. By contrast, when the SNR exceeds 5 dB, using the mixture audio shows the lowest subtitle error. On the basis of these findings, we propose an automated subtitling system that dynamically chooses between using mixture audio or separated speech to generate subtitles. The system utilizes the estimated SNR as a threshold to decide whether to apply source separation, achieving the lowest average WER under various SNR conditions.
ER  -