Is Vivaldi smooth and takete? Non-verbal sensory scales for describing music qualities

Studies on the perception of music qualities (such as induced or perceived emotions, performance styles, or timbre nuances) make a large use of verbal descriptors. Although many authors noted that particular music qualities can hardly be described by means of verbal labels, few studies have tried alternatives. This paper aims at exploring the use of non-verbal sensory scales, in order to represent different perceived qualities in Western classical music. Musically trained and untrained listeners were required to listen to six musical excerpts in major key and to evaluate them from a sensorial and semantic point of view (Experiment 1). The same design (Experiment 2) was conducted using musically trained and untrained listeners who were required to listen to six musical excerpts in minor key. The overall findings indicate that subjects\u2019 ratings on non-verbal sensory scales are consistent throughout and the results support the hypothesis that sensory scales can convey some specific sensations that cannot be described verbally, offering interesting insights to deepen our knowledge on the relationship between music and other sensorial experiences. Such research can foster interesting applications in the field of music information retrieval and timbre spaces explorations together with experiments applied to different musical cultures and contexts

Aesthetic Sensitivity

[eng] Aesthetic sensitivity is a central idea in the field of empirical aesthetics. The present research contributes a historical-critical review of its origin and development through the history of the discipline, a new theoretical approach aligned with current knowledge, novel methodological tools to investigate this and other relevant psychological constructs, and empirical evidence based on this conception that advances scientific understanding of sensory valuation.[spa] La sensibilidad estética es una idea central en el campo de la estética empírica. La presente investigación aporta una revisión histórico-crítica de su origen y desarrollo a través de la historia de la disciplina, un nuevo enfoque teórico de acuerdo con los conocimientos actuales, novedosas herramientas metodológicas para investigar éste y otros constructos psicológicos relevantes, y evidencia empírica basada en esta concepción que avanza la comprensión científica de la valoración sensorial.[cat] La sensibilitat estètica és una idea central en el camp de l'estètica empírica. La present investigació aporta una revisió històric-crítica del seu origen i desenvolupament a través de la història de la disciplina, un nou enfocament teòric alineat amb els coneixements actuals, noves eines metodològiques per investigar aquest i altres constructes psicològics rellevants, i evidència empírica basada en aquesta concepció que avança la comprensió científica de la valoració sensorial

La Petite Mort: Techniques of Orgasm in Electronic Dance Music

Portrayals of sexualized narratives are rife in ethnographic accounts of Electronic Dance Music (EDM), where the visceral musical and haptic experiences of dancing to club music are intertwined in a tantalizing symbiosis. This is music that is habitually understood through erotic semiosis: protracted build ups of stimulation and intensity, layer upon layer of musical repetition being added in service of arousal, and a teasing denial of harmonic and sonic resolution are expanded and amplified, until there is a final fulfilment granted by the DJ at the moment of the “drop”. Bringing together musicological discussions of climax with socio-psychological and neurological research on the orgasm, this article explores the manifold ways in which technologies of arousal in dance music mediate the sexual space of the dancefloor and consider how metaphors of pleasure and fulfilment are actualised in EDM in ways which destabilise the prevalent body of heteronormative, male-focussed narratives around stimulation and arousal.Les récits sexualisés sont fréquents dans les approches ethnographiques de l’Electronic Dance Music (EDM), où en dansant au son de la musique les expériences musicales et haptiques ressenties de manière viscérale se mélangent dans une symbiose fascinante. Cette musique est souvent comprise en termes d’une sémiosis érotique, à savoir comme de longues progressions de stimulation et d’intensité, où des couches musicales répétitives se superposent au service de l’excitation, et où la résolution harmonique et sonore est refusée de manière aiguichante jusqu’à ce que la satisfaction finale soit déclenchée par le DJ au moment du “drop”. En croisant des contributions musicologiques sur le climax et des recherches socio-psychologiques et neurologiques sur l’orgasme, cet article explore comment les technologies de l’excitation de la dance music médiatisent l’espace sexuel du dancefloor. Il se penche sur comment les métaphores du plaisir et la satisfaction s’actualisent dans l’EDM de manière à déstabiliser le corpus dominant, focalisé sur l’expérience masculine, de récits hétéronormatifs sur la stimulation et l’excitation

The perception of timescales in electroacoustic music

The purpose of this doctoral research is to explore the nature and perception of timescales in electroacoustic music, to examine modes of experiencing time, and to discover a method that uses this knowledge to the advantage of the composer. Although the main focus is on acousmatic works, much of the research presented here has a broader scope and is relevant to music and sound art in general. This thesis is initially inspired by Deleuze’s philosophical views on time to discover relationships between the flow of time and music, and continues to investigate time perception by exploring prevalent theories in the fields of psychology and psychoacoustics. In parallel, it identifies and systematically analyses a set of factors that influence time perception and the formation and segregation of timescales. Theoretical analysis, hypotheses and reasoning were practically tested in the five electroacoustic pieces composed for this particular research. The study revealed and reinforced the importance of psychological time in perception and interpretation of structures in music, developed the idea of using parallel temporal forms in composition, and through an exploration of timescales, it necessitated a redefinition of microsound. Moreover, an analysis of extrinsic and intrinsic factors that affect our perception of time and thus our interpretation of a musical work reinforced the notion of acousmatic music as a holistic experience that comprises all its surrounding elements at the time of listening. This research is useful for both the composer and the analyst because it offers insights into time structures, and a better understanding of the listener’s response to temporal constructs

Multisensory Integration Design in Music for Cochlear Implant Users

Cochlear implant (CI) users experience several challenges when listening to music. However, their hearing abilities are greatly diverse and their musical experiences may significantly vary from each other. In this research, we investigate this diversity in CI users' musical experience, preferences, and practices. We integrate multisensory feedback into their listening experiences to support the perception of specific musical features and elements. Four installations are implemented, each exploring a different sensory modality assisting or supporting CI users' listening experience. We study these installations throughout semi-structured and exploratory workshops with participants. We report the results of our process-oriented assessment of CI users' experience with music. Because the CI community is a minority participant group in music, musical instrument design frameworks and practices vary from those of hearing cultures. We share guidelines for designing multisensory integration that derived from our studies with individual CI users and specifically aimed to enrich their experiences

Designing Haptic Clues for Touchscreen Kiosks

Most interactive touchscreen kiosks are a challenge to accessibility: if graphics and sound fail in communication, the interaction process halts. In such a case, turning to the only remaining environmentally suited sense - the touch - is an intuitive option. To reinforce the interaction with interactive touchscreen kiosks it is possible to add haptic (touchable) feedback into the features of the device. The range of touchscreen-suited haptic technologies already enables some touch feedback from touchscreen surfaces and significant leaps still forward are being made at a constant rate. Due to this development it is relevant to review the human-centred factors affecting the design of haptic touchscreen in public kiosks. This thesis offers an overview for designing haptic clues for touchscreen kiosks. It emphasizes context sensitivity and the meaningfulness and communicability of different haptic design variants. As the main contribution, this thesis collects together the important considerations for the conscious design of haptic features in interactive kiosks and offers points of multimodal design considerations for designers intending to enrich their touchscreen interaction with haptic features

Neural processing of poems and songs is based on melodic properties

The neural processing of speech and music is still a matter of debate. A long tradition that assumes shared processing capacities for the two domains contrasts with views that assume domain-specific processing. We here contribute to this topic by investigating, in a functional magnetic imaging (fMRI) study, ecologically valid stimuli that are identical in wording and differ only in that one group is typically spoken (or silently read), whereas the other is sung: poems and their respective musical settings. We focus on the melodic properties of spoken poems and their sung musical counterparts by looking at proportions of significant autocorrelations (PSA) based on pitch values extracted from their recordings. Following earlier studies, we assumed a bias of poem-processing towards the left and a bias for song-processing on the right hemisphere. Furthermore, PSA values of poems and songs were expected to explain variance in left- vs. right-temporal brain areas, while continuous liking ratings obtained in the scanner should modulate activity in the reward network. Overall, poem processing compared to song processing relied on left temporal regions, including the superior temporal gyrus, whereas song processing compared to poem processing recruited more right temporal areas, including Heschl's gyrus and the superior temporal gyrus. PSA values co-varied with activation in bilateral temporal regions for poems, and in right-dominant fronto-temporal regions for songs. Continuous liking ratings were correlated with activity in the default mode network for both poems and songs. The pattern of results suggests that the neural processing of poems and their musical settings is based on their melodic properties, supported by bilateral temporal auditory areas and an additional right fronto-temporal network known to be implicated in the processing of melodies in songs. These findings take a middle ground in providing evidence for specific processing circuits for speech and music in the left and right hemisphere, but simultaneously for shared processing of melodic aspects of both poems and their musical settings in the right temporal cortex. Thus, we demonstrate the neurobiological plausibility of assuming the importance of melodic properties in spoken and sung aesthetic language alike, along with the involvement of the default mode network in the aesthetic appreciation of these properties

Musical expectancy within movement sonification to overcome low self-efficacy

While engaging in physical activity is important for a healthy lifestyle, low self-efficacy, i.e. one's belief in one's own ability, can prevent engagement. Sound has been used in a variety of ways for physical activity: movement sonification to inform about movement, music to encourage and direct movement, and auditory illusions to adapt people's bodily representation and movement behaviour. However, no approach provides the whole picture when considering low self-efficacy. For example, sonification does not encourage movement past a person's expectation of their ability, music gives no information of one's capabilities, and auditory illusions do not direct changes in movement behaviour in a directed way. This thesis proposes a combined method that leverages the agency felt over sonification, our embodiment of music and movement altering feedback to design \textit{``musical expectancy sonifications''} which incorporate musical expectancy within sonification to alter movement perception and behaviour. This thesis proposes a Movement Sonification Expectation Model (MoSEM), which explores expectation within a movement sonification impact on people's perception of their abilities and the way they move. This MoSEM is then interrogated and developed in four initial control studies that investigate these sonifications for different types of movement as well as how they interact with one's expectation of a given movement. These findings led to an exploration of how the MoSEM can be applied to design sonification to support low-self efficacy in two case study populations: chronic pain rehabilitation, including one control study and one mixed methods study, and general well-being, including one interview study and two control studies. These studies show the impact of musical expectation on people's movement perception and behaviour. The findings from this thesis demonstrate not only how sonifications can be designed to use musical expectancy, but also shows a number of considerations that are needed when designing movement sonifications

Multisensory learning in adaptive interactive systems

The main purpose of my work is to investigate multisensory perceptual learning and sensory integration in the design and development of adaptive user interfaces for educational purposes. To this aim, starting from renewed understanding from neuroscience and cognitive science on multisensory perceptual learning and sensory integration, I developed a theoretical computational model for designing multimodal learning technologies that take into account these results. Main theoretical foundations of my research are multisensory perceptual learning theories and the research on sensory processing and integration, embodied cognition theories, computational models of non-verbal and emotion communication in full-body movement, and human-computer interaction models. Finally, a computational model was applied in two case studies, based on two EU ICT-H2020 Projects, "weDRAW" and "TELMI", on which I worked during the PhD