Towards Music Structural Segmentation across Genres: Features, Structural Hypotheses, and Annotation Principles

This work is supported by China Scholarship Council (CSC) and EPSRC project (EP/L019981/1) Fusing Semantic and Audio Technologies for Intelligent Music Production and Consumption (FAST-IMPACt). Sandler acknowledges the support of the Royal Society as a recipient of a Wolfson Research Merit Award

What makes music memorable? Relationships between acoustic musical features and music-evoked emotions and memories in older adults

Background and objectives Music has a unique capacity to evoke both strong emotions and vivid autobiographical memories. Previous music information retrieval (MIR) studies have shown that the emotional experience of music is influenced by a combination of musical features, including tonal, rhythmic, and loudness features. Here, our aim was to explore the relationship between music-evoked emotions and music-evoked memories and how musical features (derived with MIR) can predict them both. Methods Healthy older adults (N = 113, age ≥ 60 years) participated in a listening task in which they rated a total of 140 song excerpts comprising folk songs and popular songs from 1950s to 1980s on five domains measuring the emotional (valence, arousal, emotional intensity) and memory (familiarity, autobiographical salience) experience of the songs. A set of 24 musical features were extracted from the songs using computational MIR methods. Principal component analyses were applied to reduce multicollinearity, resulting in six core musical components, which were then used to predict the behavioural ratings in multiple regression analyses. Results All correlations between behavioural ratings were positive and ranged from moderate to very high (r = 0.46–0.92). Emotional intensity showed the highest correlation to both autobiographical salience and familiarity. In the MIR data, three musical components measuring salience of the musical pulse (Pulse strength), relative strength of high harmonics (Brightness), and fluctuation in the frequencies between 200–800 Hz (Low-mid) predicted both music-evoked emotions and memories. Emotional intensity (and valence to a lesser extent) mediated the predictive effect of the musical components on music-evoked memories. Conclusions The results suggest that music-evoked emotions are strongly related to music-evoked memories in healthy older adults and that both music-evoked emotions and memories are predicted by the same core musical features.Peer reviewe

Glimpse: A gaze-based measure of temporal salience

Temporal salience considers how visual attention varies over time. Although visual salience has been widely studied from a spatial perspective, its temporal dimension has been mostly ignored, despite arguably being of utmost importance to understand the temporal evolution of attention on dynamic contents. To address this gap, we proposed GLIMPSE, a novel measure to compute temporal salience based on the observer-spatio-temporal consistency of raw gaze data. The measure is conceptually simple, training free, and provides a semantically meaningful quantification of visual attention over time. As an extension, we explored scoring algorithms to estimate temporal salience from spatial salience maps predicted with existing computational models. However, these approaches generally fall short when compared with our proposed gaze-based measure. GLIMPSE could serve as the basis for several downstream tasks such as segmentation or summarization of videos. GLIMPSE’s software and data are publicly available

Tracking linguistic and attentional influences on preferential looking in infancy

textOne unresolved issue in early word learning research is the relationship between word learning, categorization, and attention. Two distinct cognitive processes, attentional preferences related to categorical processing and inter-modal matching are involved in this relationship. Keeping the effects of these processes separate and controlled can be a difficult task. Not doing so can potentially confound the interpretation of research in this area. In a series of four preferential looking studies, the effects of referential assignment and novelty seeking in infancy were teased apart. In Study 1, 13-month olds preferred to look toward a monitor on which the stimuli changed category on every trial, and away from a monitor on which the stimuli were drawn from a single category. This preference developed in conditions in which infants listened to labels, non-language sound, or participated in silence. In Study 2, 18-month-olds developed the same preference when listening to non-language sounds or when participating in silence, but developed no preference when listening to labels. Results of studies 3 and 4 suggest that the lack of preference by 18-month-olds in the label condition result from competing behaviors of novelty seeking and referential assignment.Psycholog

An fMRI investigation of the relationship between future imagination and cognitive flexibility

While future imagination is largely considered to be a cognitive process grounded in default mode network activity, studies have shown that future imagination recruits regions in both default mode and frontoparietal control networks. In addition, it has recently been shown that the ability to imagine the future is associated with cognitive flexibility, and that tasks requiring cognitive flexibility result in increased coupling of the default mode network with frontoparietal control and salience networks. In the current study, we investigated the neural correlates underlying the association between cognitive flexibility and future imagination in two ways. First, we experimentally varied the degree of cognitive flexibility required during future imagination by manipulating the disparateness of episodic details contributing to imagined events. To this end, participants generated episodic details (persons, locations, objects) within three social spheres; during fMRI scanning they were presented with sets of three episodic details all taken from the same social sphere (Congruent condition) or different social spheres (Incongruent condition) and required to imagine a future event involving the three details. We predicted that, relative to the Congruent condition, future simulation in the Incongruent condition would be associated with increased activity in regions of the default mode, frontoparietal and salience networks. Second, we hypothesized that individual differences in cognitive flexibility, as measured by performance on the Alternate Uses Task, would correspond to individual differences in the brain regions recruited during future imagination. A task partial least squares (PLS) analysis showed that the Incongruent condition resulted in an increase in activity in regions in salience networks (e.g. the insula) but, contrary to our prediction, reduced activity in many regions of the default mode network (including the hippocampus). A subsequent functional connectivity (within-subject seed PLS) analysis showed that the insula exhibited increased coupling with default mode regions during the Incongruent condition. Finally, a behavioral PLS analysis showed that individual differences in cognitive flexibility were associated with differences in activity in a number of regions from frontoparietal, salience and default-mode networks during both future imagination conditions, further highlighting that the cognitive flexibility underlying future imagination is grounded in the complex interaction of regions in these networks

An investigation of memory specificity and generalization in young children and adults

Optimal behavior in familiar and novel contexts depends on retrieval and consideration of past experiences. In adults, hippocampus supports retrieval of prior memories based on partially overlapping cues (Mack & Preston, 2016). Given that the hippocampus develops through childhood and adolescence (Keresztes et al., 2017), in the present research we investigated developmental differences in flexible memory retrieval during new experiences. Four-year-olds (N=15) and adults (N=20) learned a series of common object-novel shape associations. Following learning, participants were cued with a shape and tasked with retrieving the target object associate. On half of the trials, participants were cued with an identical shape from learning. On the remaining trials, participants were cued with a similar but non-identical shape morph, enabling examination of whether participants can flexibly generalize across similar but non-identical experiences to retrieve related memories. Accuracy and response times were measured for adults, and accuracy was measured for children. Both adults and children demonstrated reliable retrieval when cued with similar yet non-identical shapes. Whereas adults showed slower and less accurate retrieval for the non-identical versus identical cues, children showed no differences in retrieval as a function of cue similarity. These findings have important implications for our understanding of how mnemonic specificity and generalization interact across development. In particular, our findings suggest that mnemonic generalization in early childhood is a consequence of less detailed memory representation. Conversely, the more mature form of generalization evidenced in adulthood is accomplished through dual processing of the commonalities and specific differences between similar yet non-identical experiences.Neuroscienc

Explaining Listener Differences in the Perception of Musical Structure

PhDState-of-the-art models for the perception of grouping structure in music do not attempt to account for disagreements among listeners. But understanding these disagreements, sometimes regarded as noise in psychological studies, may be essential to fully understanding how listeners perceive grouping structure. Over the course of four studies in different disciplines, this thesis develops and presents evidence to support the hypothesis that attention is a key factor in accounting for listeners' perceptions of boundaries and groupings, and hence a key to explaining their disagreements. First, we conduct a case study of the disagreements between two listeners. By studying the justi cations each listener gave for their analyses, we argue that the disagreements arose directly from differences in attention, and indirectly from differences in information, expectation, and ontological commitments made in the opening moments. Second, in a large-scale corpus study, we study the extent to which acoustic novelty can account for the boundary perceptions of listeners. The results indicate that novelty is correlated with boundary salience, but that novelty is a necessary but not su cient condition for being perceived as a boundary. Third, we develop an algorithm that optimally reconstructs a listener's analysis in terms of the patterns of similarity within a piece of music. We demonstrate how the output can identify good justifications for an analysis and account for disagreements between two analyses. Finally, having introduced and developed the hypothesis that disagreements between listeners may be attributable to differences in attention, we test the hypothesis in a sequence of experiments. We find that by manipulating the attention of participants, we are able to influence the groupings and boundaries they find most salient. From the sum of this research, we conclude that a listener's attention is a crucial factor affecting how listeners perceive the grouping structure of music.Social Sciences and Humanities Research Council; a PhD studentship from Queen Mary University of London; a Provost's Ph.D. Fellowship from the University of Southern California. This material is also based in part on work supported by the National Science Foundation under Grant No. 0347988