Measuring embodied conceptualizations of pitch insinging performances: insights from an OpenPose study

<p>People conceptualize auditory pitch as vertical space: low and high pitch correspond to low and high space respectively. The strength of this cross-modal correspondence, however, seems to vary across different cultural contexts and a debate on the different factors underlying this variation is currently taking place. According to one hypothesis, pitch mappings are semantically mediated. For instance, the use of conventional metaphors such as &lsquo;falling&rsquo; or &lsquo;rising&rsquo; melodies strengthens a pitch-height mapping to the detriment of other possible mappings (e.g. pitch as bright/dark color or small/big size). Hence, entrenched pitch terms shape specific conceptualizations. The deterministic role of language is called into question by the hypothesis that different pitch mappings share a less constraining conceptual basis. As such, conceptual primitives may be concretized <em>ad hoc</em> into specific domains so that more local variation is possible<em>.</em> This claim is supported, for instance, by the finding that musicians use language-congruent (conventional) and language-incongruent (<em>ad hoc</em>) mappings interchangeably. The present paper substantiates this observation by investigating the head movements of musically trained and untrained speakers of Dutch in a melody reproduction task, as embodied instantiations of a vertical conceptualization of pitch. The OpenPose algorithm was used to track the movement trajectories in detail. The results show that untrained participants systematically made language-congruent movements, while trained participants showed more diverse behaviors, including language-incongruent movements. The difference between the two groups could not be attributed to the level of accuracy in the singing performances. In sum, this study argues for a joint consideration of more entrenched (e.g. linguistic metaphors) and more context-dependent (e.g. musical training and task) factors in accounting for variability in pitch representations.</p>