Automatic Surtitle Generation Scheme for Live Cantonese Opera

He W, Chau K. Automatic Surtitle Generation Scheme for Live Cantonese Opera. In: Express Paper. Audio Engineering Society; 2024. Paper 212. Available from: https://aes.org/publications/elibrary-page/?id=22558

@inproceedings{He2024_22558,
  author = {He, Weike and Chau, Kevin},
  title = {{Automatic Surtitle Generation Scheme for Live Cantonese Opera}},
  note = {Paper 212},
  year = {2024},
  month = jun,
  publisher = {Audio Engineering Society},
  url = {https://aes.org/publications/elibrary-page/?id=22558}
}

TY  - CPAPER
TI  - Automatic Surtitle Generation Scheme for Live Cantonese Opera
AU  - He, Weike
AU  - Chau, Kevin
M1  - Paper 212
PY  - 2024
DA  - 2024/06/06
UR  - https://aes.org/publications/elibrary-page/?id=22558
PB  - Audio Engineering Society
LA  - en
AB  - Presently reported is an automatic surtitle generation scheme for live Cantonese opera based on beat tracking. The individual Chinese characters in the surtitles are animated in a karaoke style in sync with the performance. The scheme requires a music-vocal score of the opera but no human intervention during the live performance. A major advantage of the method is that the next Chinese character can be predictively highlighted at the onset of its vocalization with almost zero latency after compensating for the digitizing and processing time of the streaming audio. Specifically, a regularly spaced pulse train registering the onset position of every monosyllabic Chinese character to be sung or spoken is initially prepared from the music-vocal score. On-going synchronization of the pulse train to the live performance is obtained by monitoring the beats in the audio. Assuming beat tracking is accurate, the alignment of the surtitle characters should be 100% correct. An exception is speech that does not have a beat structure for which only an estimated speech rate can be established. The objective of this research is to promote Cantonese opera to those who do not speak Cantonese with the hope that the present surtitle generation
ER  -