Sonifying time series via music generated with machine learning

Diaz K, Villegas J. Sonifying time series via music generated with machine learning. In: AES Convention 155. Audio Engineering Society; 2023. Paper 10675. Available from: https://aes.org/publications/elibrary-page/?id=22256

@inproceedings{Diaz2023_22256,
  author = {Diaz, Kevin and Villegas, Julián},
  title = {{Sonifying time series via music generated with machine learning}},
  booktitle = {AES Convention 155},
  note = {Paper 10675},
  year = {2023},
  month = oct,
  publisher = {Audio Engineering Society},
  url = {https://aes.org/publications/elibrary-page/?id=22256}
}

TY  - CPAPER
TI  - Sonifying time series via music generated with machine learning
AU  - Diaz, Kevin
AU  - Villegas, Julián
T2  - AES Convention 155
M1  - Paper 10675
PY  - 2023
DA  - 2023/10/06
UR  - https://aes.org/publications/elibrary-page/?id=22256
PB  - Audio Engineering Society
LA  - en
AB  - Conventional sonifications directly assign different aspects of data to auditory features and the results are not always “musical” as they do not adhere to a recognizable structure, genre, style, etc. Our system tackles this problem by learning orthogonal features in the latent space of a given musical corpus and using those features to create derivative compositions. We propose using a Singular Autoencoder (SAE) algorithm that identifies the most important Principal Components (PCs) in the latent space. As a proof-of-concept, we created sonifications of ionizing radiation measurements obtained from the Safecast project. Although the system successfully generates new compositions by manipulating the latent space, with each principal component changing different musical aspects, these changes may not be readily noticeable by listeners, despite the PCs being mathematically decorrelated. This finding suggests that higher-level features (such as associated emotion, etc.) may be needed for better results.
ER  -