Using Neural Networks to Compute Time Offsets from Musical Instruments

Pereira I, Distante C, Silveira LF, Gonçalves L. Using Neural Networks to Compute Time Offsets from Musical Instruments. J Audio Eng Soc. 2020;68(3):157-167. doi:10.17743/jaes.2019.0057

@article{Pereira2020_20726,
  author = {Pereira, Igor and Distante, Cosimo and Silveira, Luiz F. and Gonçalves, Luiz},
  title = {{Using Neural Networks to Compute Time Offsets from Musical Instruments}},
  journal = {Journal of the Audio Engineering Society},
  volume = {68},
  number = {3},
  pages = {157--167},
  url = {https://doi.org/10.17743/jaes.2019.0057},
  doi = {10.17743/jaes.2019.0057},
  year = {2020},
  month = mar,
  publisher = {Audio Engineering Society}
}

TY  - JOUR
TI  - Using Neural Networks to Compute Time Offsets from Musical Instruments
AU  - Pereira, Igor
AU  - Distante, Cosimo
AU  - Silveira, Luiz F.
AU  - Gonçalves, Luiz
T2  - Journal of the Audio Engineering Society
J2  - J. Audio Eng. Soc.
VL  - 68
IS  - 3
SP  - 157
EP  - 167
PY  - 2020
DA  - 2020/03/06
DO  - 10.17743/jaes.2019.0057
UR  - https://doi.org/10.17743/jaes.2019.0057
PB  - Audio Engineering Society
LA  - en
AB  - This research proposes an approach for computing the time offsets between audio sequences that contain musical sounds from different instruments produced in a distributed way and which have a set of weak features that are not useful as alignment points. It is therefore necessary to apply transformations in order to find a set of distinctive features to compute the offset values in a suitable way. The main issue that occurs with such a system is nonlinearity that does not allow the delay to be predicted by using a linear function. To solve this problem, the authors propose a set of long short-term memory (LSTM) layers to create a neural network model capable of learning such features transformations in a supervised approach, using a gradient-descent optimizer. This demonstrates the use of a recurrence matrix to extract timing information from a set of transformed features given by the neural network output. With this approach, the algorithm can classify up to 60% of a specific combination from the MedleyDB data set, and reduce the search space to five possibilities with accuracy up to 90% while keeping the precision of 10 ms. This performance is equal or better than state-of-the-art methods.
ER  -