Toward a data driven approach to uncover the acoustical correlates of musical instruments timbre perception

Although extensively studied since years through dissimilarity ratings experiments, the acoustical correlates of musical instruments timbre perception remain not fully understood and somewhat controversial. Whereas it’s now commonly accepted that spectral and temporal envelops correlate well with the two first dimensions of so-called perceptual timbre spaces, critical questions remain: is it relevant to look for global audio descriptors correlating with the higher order dimensions of timbre spaces? can human dissimilarity ratings be computed directly from acoustic representations? and more globally, to which extent the knowledge on timbre is consistent between the plenty of studies? These questions were here addressed by re-analysing 13 dissimilarity ratings experiments from 7 different studies spanning 40 years of research. For each experiment, (1) a traditional dimensional analysis (MDS) has been run to assess their reproducibility (2) a data-driven perspective has been adopted to computationally learn a distance metrics between acoustic representations of sounds with the objective of maximally correlating the learned distances to human dissimilarity ratings. We observed that (1) the dimensional approach is not robust between studies (2) the metrics learned with cortical representations of sounds (STRFs) allow to nearly perfectly simulate human ratings. More strikingly, for each experiment, a significantly high proportion of the learned metrics variance is explained by the dataset’s STRFs variability revealing the context dependence of timbre perception. These works point out the limitations of the dimensional approach as well as the malleable nature of musical instruments timbre perception.