Machine listening, musicological analysis and the creative process : the production of song-based music influenced by augmented listening techniques

The creative process of making recordings in the popular music sphere is impossible to disconnect from the concept of influence. Whether practitioners are influenced consciously or subconsciously, and push toward, or away from their influences, they are shaped by the music they hear. This research-led practice project augments the influencing factors in the creation of an album of song-based music by foregrounding the listening process. This is approached by conducting an in-depth analysis of a set of tracks using a combined methodology integrating traditional popular music analysis techniques, with music information retrieval (MIR) tools. My methodology explores the novel applicability of these computational tools in a musicological context, with one goal being to show the value of machine listening in popular musicological research and the processes of composition and production. The emerging field of Digital Musicology takes advantage of big data and statistical analysis to allow for large scale observation and comparison of datasets in a way that would be unrealistic for one person to attempt without the aid of machine listening. Setting aside the intention and reception components of musicology, the goal of musical output suggests a feature-based approach is well suited to the task of investigating these methods. Utilising the musical ideas generated through the combined analysis of tracks compiled from the Billboard Alternative, year-end charts of 2011-2015, the songs written and recordings produced for the album Stay Still | Please Hear are a result of allowing a conscious subversion of my usual creative process through the expansion of my field of musical influence. This discursive component shows the development of my combined analysis methodology, highlights the points where creative influence occurred in the arrangement and production of Stay Still | Please Hear, emphasises the value of MIR tools in expanding the scope of musicological analysis, and demonstrates a unique approach to the development of artistic practice from the perspective of a creative practitioner

Influence of Music on Perceived Emotions in Film

Film music plays a core role in film production and reception as it not only contributes to the film aesthetics and creativity, but it also affects viewers' experience and enjoyment. Film music composers often aim to serve the film narrative, immerse viewers into the setting and story, convey clues, and importantly, act on their emotions. Yet, how film music influences viewers is still misunderstood. We conducted a perceptual study to analyse the impact of music on the perception of emotions in film. We developed an online interface for time-based emotion annotation of audio/video media clips based on the Valence/Arousal (VA) two-dimensional model. Participants reported their perceived emotions over time in the VA space for three media conditions: film scene presented without sound (video only), film music presented without video (audio only), and film scene with accompanying music and sound effects (both video and audio modalities). 16 film clips were selected covering four clips for each of four genres (action & drama, romance, comedy, and horror). 38 participants completed the study (12 females and 26 males from many countries, average age: 28.9). Density scatter plots are used to visualise the spread of emotion ratings in the VA space and differences across media conditions and film clips. Results from linear mixed effect models show significant effects of the audiovisual media condition and film genre on VA ratings, in line with previous results by Parke et al. [1]. Perceived VA ratings across media conditions follow an almost linear relationship with an increase in strength in the following order: film alone, film with music/sound, music alone. We illustrate this effect by plotting the VA rating centre of mass across conditions. VA ratings for the film-alone condition are closer to the origin of the space compared to the two other media conditions, indicating that the inclusion of music yields stronger emotions (higher VA ratings). Certain individual factors (musical ability, familiarity, preference) also seem to impact the perception of arousal and valence while viewing films. Our online emotion annotation interface was on overall well received and suggestions to improve the display of reference emotion tags are discussed

Silver Linings Playlist: Exploring the Effectiveness of Music as an Emotion Regulation Tool

Music not only entertains listeners, but it also evokes emotions and facilitates emotion regulation (Gabrielsson, 2001; Krahe & Bienick, 2012). Specifically, music helps listeners to express their emotions and alter their mood through cognitive reappraisal (Chin & Rickard, 2014; Witvliet & Vrana, 2007). Listening to music also enhances relaxation and reduces physiological arousal after experiencing a stressful event (Yehuda, 2011). Stress often involves an influx of negative emotions, which when left unmitigated, may result in fewer positive emotions, increased depression, and maladaptive coping (Flynn & Rudolph, 2010; Lazenby et al., 2019). While music appears to be an effective stress management tool, less is known about how music impacts positive and negative emotions in the context of stressors. Thus, the current study examined the ability of different kinds of music interventions to reduce negative emotions and increase positive emotions after a stressful event. The study was conducted in two phases using undergraduate samples. Seven hundred and ninety-five students completed a music preferences questionnaire during Phase 1. During Phase 2, 63 students who participated in Phase 1 were randomly assigned to one of three conditions (Empowering Music, Calming Music, Control Group) following a stress induction task. Measures of positive and negative emotions and stress levels were administered three times during the experiment. A series of 3 (Time [Time 1, Time 2, Time 3]) x 3 (Condition [Control Task, Calming Music, Empowering Music]) factorial ANOVAs were used to determine the impact of the music interventions on positive and negative emotion scores. Regarding positive emotions, results revealed a significant Time x Condition interaction effect, where individuals in the Empowering Music group reported greater positive emotions compared to those in the control group. Results revealed a significant Time x Condition interaction effect for negative emotions as well. However, there were some methodological concerns which prevented me from clearly interpreting these findings. Results also revealed a significant main effect for condition at Time 3 stress scores, where individuals in both music intervention conditions reported significantly lower stress scores compared to those in the control group. These findings highlight the benefits of using empowering music to build positive emotions, which may help individuals find well-being, even in the face of moderately distressing events

Evaluation of Poetic Creativity: Predictors and the Role of Expertise - A Multilevel Approach

Poetry is one of the most creative expressions of language, but how we evaluate the creativity of a poem is not properly characterized. The present study investigated the role of various subjective qualities – clarity, aesthetic appeal, felt valence, arousal, and surprise – in predicting the creativity judgment of English poems. Participants (N=129) were presented with a broad range of English poems; they rated each poem on six characteristics: clarity, aesthetic appeal, felt valence, felt arousal, surprise and overall creativity. Linear multilevel analysis showed that aesthetic appeal was the strongest predictor of poetic creativity, followed by surprise and felt valence. Multilevel mediation analysis indicated significant mediation by surprise and felt valence on the relationship between aesthetic appeal and creativity at both within and between-participant levels. Further, expertise in English literature was found to significantly moderate the effects of all three predictors on the evaluation of creativity. The study simultaneously captured the surprise-evoking line(s). Using the semantic distance computing approach, we have shown the objective validation of the subjectively chosen line(s) of surprise. Altogether, our findings suggest a parsimonious model of evaluation of creativity of poems and its interaction with expertise

Modelling affect for horror soundscapes

The feeling of horror within movies or games relies on the audience’s perception of a tense atmosphere — often achieved through sound accompanied by the on-screen drama — guiding its emotional experience throughout the scene or game-play sequence. These progressions are often crafted through an a priori knowledge of how a scene or game-play sequence will playout, and the intended emotional patterns a game director wants to transmit. The appropriate design of sound becomes even more challenging once the scenery and the general context is autonomously generated by an algorithm. Towards realizing sound-based affective interaction in games this paper explores the creation of computational models capable of ranking short audio pieces based on crowdsourced annotations of tension, arousal and valence. Affect models are trained via preference learning on over a thousand annotations with the use of support vector machines, whose inputs are low-level features extracted from the audio assets of a comprehensive sound library. The models constructed in this work are able to predict the tension, arousal and valence elicited by sound, respectively, with an accuracy of approximately 65%, 66% and 72%.peer-reviewe

Personal relevance in story reading: a research review

Although personal relevance is key to sustaining an audience’s interest in any given narrative, it has received little systematic attention in scholarship to date. Across centuries and media, adaptations have been used extensively to bring temporally or geographically distant narratives “closer” to the recipient under the assumption that their impact will increase. In this review article, we review experimental and other empirical evidence on narrative processing in order to unravel which types of personal relevance are more likely to be impactful than others, which types of impact (e.g. aesthetic, therapeutic, persuasive) they have been found to generate, and where their power may become excessive or outright detrimental to reader experience

Evaluating Musical Foreshadowing of Videogame Narrative Experiences

We experiment with mood-expressing, procedurally gener-ated music for narrative foreshadowing in videogames, in-vestigating the relationship between music and the player’s experience of narrative events in a game. We designed and conducted a user study in which the game’s music expresses true foreshadowing in some trials (e.g. foreboding music before a negative event) and false foreshadowing in others (e.g. happy music that does not lead to a positive event). We observed players playing the game, recorded analytics data, and had them complete a survey upon completion of the gameplay. Thirty undergraduate and graduate students participated in the study. Statistical analyses suggest that the use of musical cues for narrative foreshadowing induces a better perceived consistency between music and game narra-tive. Surprisingly, false foreshadowing was found to enhance the player’s enjoyment