Deep learning sonification synthesizer for the CaT stellar spectra library

Garcia Riber A, Serradilla F. Deep learning sonification synthesizer for the CaT stellar spectra library. In: Express Paper. Audio Engineering Society; 2024. Paper 191. Available from: https://aes.org/publications/elibrary-page/?id=22537

@inproceedings{GarciaRiber2024_22537,
  author = {Garcia Riber, Adrian and Serradilla, Francisco},
  title = {{Deep learning sonification synthesizer for the CaT stellar spectra library}},
  note = {Paper 191},
  year = {2024},
  month = jun,
  publisher = {Audio Engineering Society},
  url = {https://aes.org/publications/elibrary-page/?id=22537}
}

TY  - CPAPER
TI  - Deep learning sonification synthesizer for the CaT stellar spectra library
AU  - Garcia Riber, Adrian
AU  - Serradilla, Francisco
M1  - Paper 191
PY  - 2024
DA  - 2024/06/06
UR  - https://aes.org/publications/elibrary-page/?id=22537
PB  - Audio Engineering Society
LA  - en
AB  - This paper describes the design and implementation strategies proposed in the development of a synthesizer that allows the multimodal exploration of the Ca II triplet (CaT) stellar spectra library from the Spanish Virtual Observatory (SVO). The prototype reacts "on the fly" to provide harmonic accompaniment for each note of the piano keyboard, generated from real sky objects data using deep learning. A two-layer autoencoder architecture reduces each stellar spectrum to a six-note MIDI chord. The algorithm looks for the root note of the resulting "stellar chords" matching the user pressed piano key note, and generates the sonification as function of its encoded latent vector and the number of absorption lines detected in the spectrum. The UI provides the name of the object, the spectrum, and the accuracy of the auditory representation via R2 calculated between the original and the decoded spectrum. The proposal introduces a musical extension of the sonification of astronomical data to be used in sound design and musical composition for the creation of complex timbres and textures, and in educational and outreach activities to bring astronomy concepts closer to all audiences using the engagement potential of sound and music.
ER  -