Time Series Analysis as a Method to Examine Acoustical Influences on Real-time Perception of Music

Multivariate analyses of dynamic correlations between continuous acoustic properties (intensity and spectral flatness) and real-time listener perceptions of change and expressed affect (arousal and valence) in music are developed, by an extensive application of autoregressive Time Series Analysis (TSA). TSA offers a large suite of techniques for modeling autocorrelated time series, such as constitute both music’s acoustic properties and its perceptual impacts. A logical analysis sequence from autoregressive integrated moving average regression with exogenous variables (ARIMAX), to vector autoregression (VAR) is established. Information criteria discriminate amongst models, and Granger Causality indicates whether a correlation might be a causal one. A 3 min electroacoustic extract from Wishart’s Red Bird is studied. It contains digitally generated and transformed sounds, and animate sounds, and our approach also permits an analysis of their impulse action on the temporal evolution and the variance in the perceptual time series. Intensity influences perceptions of change and expressed arousal substantially. Spectral flatness influences valence, while animate sounds influence the valence response and its variance. This TSA approach is applicable to a wide range of questions concerning acoustic- perceptual relationships in music

Modelling Perception of Structure and Affect in Music: Spectral Centroid and Wishart's Red Bird

Pearce (2011) provides a positive and interesting response to our article on time series analysis of the influences of acoustic properties on real-time perception of structure and affect in a section of Trevor Wishart’s Red Bird (Dean & Bailes, 2010). We address the following topics raised in the response and our paper. First, we analyse in depth the possible influence of spectral centroid, a timbral feature of the acoustic stream distinct from the high level general parameter we used initially, spectral flatness. We find that spectral centroid, like spectral flatness, is not a powerful predictor of real-time responses, though it does show some features that encourage its continued consideration. Second, we discuss further the issue of studying both individual responses, and as in our paper, group averaged responses. We show that a multivariate Vector Autoregression model handles the grand average series quite similarly to those of individual members of our participant groups, and we analyse this in greater detail with a wide range of approaches in work which is in press and continuing. Lastly, we discuss the nature and intent of computational modelling of cognition using acoustic and music- or information theoretic data streams as predictors, and how the music- or information theoretic approaches may be applied to electroacoustic music, which is ‘sound-based’ rather than note-centred like Western classical music

Comparative time series analysis of perceptual responses to electroacoustic music

This study investigates the relationship between acoustic patterns in contemporary electroacoustic compositions, and listeners’ real-time perceptions of their structure and affective content. Thirty-two participants varying in musical expertise (nonmusicians, classical musicians, expert computer musicians) continuously rated the affect (arousal and valence) and structure (change in sound) they perceived in four compositions of approximately three minutes duration. Time series analyses tested the hypotheses that sound intensity influences listener perceptions of structure and arousal, and spectral flatness influences perceptions of structure and valence. Results suggest that intensity strongly influences perceived change in sound, and to a lesser extent listener perceptions of arousal. Spectral flatness measures were only weakly related to listener perceptions, and valence was not strongly shaped by either acoustic measure. Differences in response by composition and musical expertise suggest that, particularly with respect to the perception of valence, individual experience (familiarity and liking), and meaningful sound associations mediate perception

Facilitation and Coherence Between the Dynamic and Retrospective Perception of Segmentation in Computer-Generated Music

We examined the impact of listening context (sound duration and prior presentation) on the human perception of segmentation in sequences of computer music. This research extends previous work by the authors (Bailes & Dean, 2005), which concluded that context-dependent effects such as the asymmetrical detection of an increase in timbre compared to a decrease of the same magnitude have a significant bearing on the cognition of sound structure. The current study replicated this effect, and demonstrated that listeners (N = 14) are coherent in their detection of segmentation between real-time and retrospective tasks. In addition, response lag was reduced from a first hearing to a second hearing, and following long (7 s) rather than short (1 or 3 s) segments. These findings point to the role of short-term memory in dynamic structural perception of computer music

Relationships Between Acoustic Features and Perceptual Segmentation of Music Audio

Presented at the 20th International Conference on Auditory Display (ICAD2014), June 22-25, 2014, New York, NY.For a successful practical sonification, boundaries where the information state changes need to be made readily apparent perceptually. I describe time series analysis techniques for the detection of segments perceived implicitly during continuous music audio. The detected segments are compared with those measured computationally, or determined musicologically, on the input audio. The degree to which perceptual segmentation can be predicted is discussed, together with some of the factors apparently responsible. This may give useful cues as to how best to structure sonifications for informational purposes

Encouraging attention and exploration in a hybrid recommender system for libraries of unfamiliar music

There are few studies of user interaction with music libraries comprising solely of unfamiliar music, despite such music being represented in national music information centre collections. We aim to develop a system that encourages exploration of such a library. This study investigates the influence of 69 users’ pre-existing musical genre and feature preferences on their ongoing continuous real-time psychological affect responses during listening and the acoustic features of the music on their liking and familiarity ratings for unfamiliar art music (the collection of the Australian Music Centre) during a sequential hybrid recommender-guided interaction. We successfully mitigated the unfavorable starting conditions (no prior item ratings or participants’ item choices) by using each participant’s pre-listening music preferences, translated into acoustic features and linked to item view count from the Australian Music Centre database, to choose their seed item. We found that first item liking/familiarity ratings were on average higher than the subsequent 15 items and comparable with the maximal values at the end of listeners’ sequential responses, showing acoustic features to be useful predictors of responses. We required users to give a continuous response indication of their perception of the affect expressed as they listened to 30-second excerpts of music, with our system successfully providing either a “similar” or “dissimilar” next item, according to—and confirming—the utility of the items’ acoustic features, but chosen from the affective responses of the preceding item. We also developed predictive statistical time series analysis models of liking and familiarity, using music preferences and preceding ratings. Our analyses suggest our users were at the starting low end of the commonly observed inverted-U relationship between exposure and both liking and perceived familiarity, which were closely related. Overall, our hybrid recommender worked well under extreme conditions, with 53 unique items from 100 chosen as “seed” items, suggesting future enhancement of our approach can productively encourage exploration of libraries of unfamiliar music

A Systematic Review of Interpersonal Violence Perpetration and Victimization Risk Examined Within Single Study Cohorts, Including in Relation to Mental Illness

Rates of both violent victimization and violence perpetration are known to be elevated among individuals with mental illness compared with those in the general population, though the relative risk of each outcome is less well established. In this systematic review, PubMed, Embase, Web of Science, PsycINFO, and Criminal Justice Abstracts were searched for articles published any time before October 2021 that reported the prevalence or incidence of both violent victimization and perpetration. We performed two searches to identify studies using samples or cohorts of (1) persons with mental illnesses and (2) persons in the general population. A total of 25 studies (9 examining persons with mental illnesses, 13 examining persons in the general population, and 3 examining both sample/cohort types) were identified and data was extracted to describe the type and size of cohort or sample, definitions and terminology (i.e., mental illness, violence victimization, violence perpetration), data source(s), observation period, prevalence/incidence of victimization, and prevalence/incidence of perpetration. The Joanna Briggs Institute Critical Appraisal Checklist for Studies Reporting Prevalence Data was used to conduct a quality assessment of all included studies. Due to marked study heterogeneity, results were presented using a narrative synthesis approach. Across studies, findings were mixed, and the methodological approaches varied greatly. Broadly, the review provides evidence for (1) higher rates of victimization than perpetration for both individuals with mental illness and those in the general population and (2) higher rates of both victimization and perpetration for those with mental illness compared to those in the general population