Line Harp: Importance-Driven Sonification for Dense Line Charts

Masteroppgave i informatikkINF399MAMN-PROGMAMN-IN

Bodies That Scatter: Sound, Cinema, Figure, Form

This dissertation argues for an understanding of “form” in two senses: as both the plasticity of the cinematic medium and the contour of the human body. Form is most often understood in relation to visible traits such as line, shape, or figure; whereas sound is described as being without form, as any number of sonorous metaphors attests (sound as aqueous, as “disembodied,” as invisible or indeterminate). Cinema, therefore, constituted as it is by sound and image, exhibits a tension between form and formlessness that manifests itself in contemporary films such as Her, Ratcatcher, Morvern Callar, We Need to Talk About Kevin, The Tree of Life, and Upstream Color at the level of the human body: corporeal boundaries blur as characters incorporate into new configurations. I conclude that we must reckon with the image in terms of sound, for thinking sonically allows us to see form—bodily and cinematic—as inherently protean. Unlike most scholarship on film sound that is predominantly formalist or historical in nature, my study takes an interdisciplinary approach that draws on theories of sound and image hailing from outside film studies proper, such as Nietzsche’s theorization of Greek tragedy, Jean-Luc Nancy’s ethics of listening, Deleuze’s exegesis on painter Francis Bacon, Nicole Brenez’s “figural” method of film criticism, and Karen Barad’s “agential realism.” I unite these ideas and others and bring them to bear on cinema, showing in the process what an attentiveness to sound has to offer not only film studies but also the humanities writ large

Silent Light, Luminous Noise: Photophonics, Machines and the Senses

This research takes the basic physical premise that sound can be synthesized using light, explores how this has historically been, and still is achieved, and how it can still be a fertile area for creative, theoretical and critical exploration in sound and the arts. Through the author's own artistic practice, different techniques of generating sound using the sonification of light are explored, and these techniques are then contextualised by their historical and theoretical setting in the time-based arts. Specifically, this text draws together diverse strands of scholarship on experimental sound and film practices, cultural histories, the senses, media theory and engineering to address effects and outcomes specific to photophonic sound and its relation to the moving image, and the sculptural and media works devised to produce it. The sonifier, or device engendering the transformations discussed is specifically addressed in its many forms, and a model proposed, whereby these devices and systems are an integral, readably inscribed component - both materially and culturally - in both the works they produce, and via our reflexive understanding of the processes involved, of the images or light signals used to produce them. Other practitioners' works are critically engaged to demonstrate how a sense of touch, or the haptic, can be thought of as an emergent property of moving image works which readably and structurally make use of photophonic sound (including the author's), and sound's essential role in this is examined. In developing, through an integration of theory and practice, a new approach in this under-researched field of sound studies, the author hopes to show how photophonic sound can act as both a metaphorical and material interface between experimental sound and image, and hopefully point the way towards a more comprehensive study of both

Portfolio of original compositions

In the last few decades the area of touch research has become vibrant, exciting and urgent, yet research into touch in music – in the way outlined here as cutaneous contact – is close to non-existent. Sounds emanate from touch but still, touch has not been included in the aesthetics of music. Examining touch in musical literature I argue that touch is fundamental to music while music’s philosophical and aesthetic definition has excluded it. My research reveals that what were previously argued to be the main reasons to marginalize touch in music, are in fact facets of a style of music which has, so far, been marginalized because touch’s inclusion problematizes the tangibility of the musical experience. By contrasting my portfolio of original works with my literary review I demonstrate my interrogation of these marginalized facets and show how music changes by moving away from sounding description to tactile demonstration and exploration of sounding shapes, textures and resonating volumes. By including touch in music instruments become tailor-made for individual bodies engaging with and disrupting the technical skill of the musician, the score stops being a singularly imaginable picture and becomes a fragmented assemblage of tangible forms as a musical instance, which I have called the composed musical instrument. Thus, changing musical organization from the written to the assembled composition changes from writing in time to feeling the mapping of topologies and the manipulation of materials through cutaneous contact. Performance of a composition stops being stage oriented and begins at the first sound of making the composed musical instrument. Most strangely, silence, a key component of music so far, stops being necessary and listening can be done with the skin. Retaining its communicative value as a sound the inclusion of touch creates a semantic sign system that can only be experienced through touch

Integrating passive ubiquitous surfaces into human-computer interaction

Mobile technologies enable people to interact with computers ubiquitously. This dissertation investigates how ordinary, ubiquitous surfaces can be integrated into human-computer interaction to extend the interaction space beyond the edge of the display. It turns out that acoustic and tactile features generated during an interaction can be combined to identify input events, the user, and the surface. In addition, it is shown that a heterogeneous distribution of different surfaces is particularly suitable for realizing versatile interaction modalities. However, privacy concerns must be considered when selecting sensors, and context can be crucial in determining whether and what interaction to perform.Mobile Technologien ermöglichen den Menschen eine allgegenwärtige Interaktion mit Computern. Diese Dissertation untersucht, wie gewöhnliche, allgegenwärtige Oberflächen in die Mensch-Computer-Interaktion integriert werden können, um den Interaktionsraum über den Rand des Displays hinaus zu erweitern. Es stellt sich heraus, dass akustische und taktile Merkmale, die während einer Interaktion erzeugt werden, kombiniert werden können, um Eingabeereignisse, den Benutzer und die Oberfläche zu identifizieren. Darüber hinaus wird gezeigt, dass eine heterogene Verteilung verschiedener Oberflächen besonders geeignet ist, um vielfältige Interaktionsmodalitäten zu realisieren. Bei der Auswahl der Sensoren müssen jedoch Datenschutzaspekte berücksichtigt werden, und der Kontext kann entscheidend dafür sein, ob und welche Interaktion durchgeführt werden soll

Haptics: Science, Technology, Applications

This open access book constitutes the proceedings of the 12th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, EuroHaptics 2020, held in Leiden, The Netherlands, in September 2020. The 60 papers presented in this volume were carefully reviewed and selected from 111 submissions. The were organized in topical sections on haptic science, haptic technology, and haptic applications. This year's focus is on accessibility

Emotions in everyday sounds. An experimental study on the influence of colour in emotional knocking sounds.

This thesis is an experimental project on sound design, aimed at exploring the world of sound and its artistic and communicative applications concerned with emotion elicitation. When designing any type of communication, whether it is an informative content, a marketing campaign or a multimedia product, it is important to know how to express specific concepts, what their effect on the audience might be and how they can be implemented and reinforced by means of communication tools. Everyday sounds, for instance walking and knocking on a door, are more powerful means of information that one may think. It is this last type of sounds, more precisely knocking sounds, the object of the present study. In particular, it will be discussed the potential sound has, especially when correlated with the elicitation of emotions. The perception of emotions will be studied by means of an empirical experiment aiming at investigating the influence of certain colours, proved to be associated with specific emotions, on emotionally performed knocking sounds. The study hypothesis is that the association between knocking sounds and emotions changes if the knocking sound is performed on a door of a different colour. Our initial hypotheses of colour-emotion associations were: yellow-happiness, grey-sadness, red-anger, and purple-fear. After the analysis of the data gathered through an online questionnaire, only two of our hypotheses were confirmed: happiness and sadness. Anger and fear, on the contrary, did not meet expectations. In the first case, the colour that received a higher score was white (neutral), followed by red and purple. Whereas in the second case, the audio stimuli were averagely confused with anger, independently of the colour. In general, the aural modality prevailed over the visual one. However, colours still influenced the perception of the emotion, even if weakly, especially in the cases of happiness and sadness.This thesis is an experimental project on sound design, aimed at exploring the world of sound and its artistic and communicative applications concerned with emotion elicitation. When designing any type of communication, whether it is an informative content, a marketing campaign or a multimedia product, it is important to know how to express specific concepts, what their effect on the audience might be and how they can be implemented and reinforced by means of communication tools. Everyday sounds, for instance walking and knocking on a door, are more powerful means of information that one may think. It is this last type of sounds, more precisely knocking sounds, the object of the present study. In particular, it will be discussed the potential sound has, especially when correlated with the elicitation of emotions. The perception of emotions will be studied by means of an empirical experiment aiming at investigating the influence of certain colours, proved to be associated with specific emotions, on emotionally performed knocking sounds. The study hypothesis is that the association between knocking sounds and emotions changes if the knocking sound is performed on a door of a different colour. Our initial hypotheses of colour-emotion associations were: yellow-happiness, grey-sadness, red-anger, and purple-fear. After the analysis of the data gathered through an online questionnaire, only two of our hypotheses were confirmed: happiness and sadness. Anger and fear, on the contrary, did not meet expectations. In the first case, the colour that received a higher score was white (neutral), followed by red and purple. Whereas in the second case, the audio stimuli were averagely confused with anger, independently of the colour. In general, the aural modality prevailed over the visual one. However, colours still influenced the perception of the emotion, even if weakly, especially in the cases of happiness and sadness

Perceptual Organization

Perceiving the world of real objects seems so easy that it is difficult to grasp just how complicated it is. Not only do we need to construct the objects quickly, the objects keep changing even though we think of them as having a consistent, independent existence (Feldman, 2003). Yet, we usually get it right, there are few failures. We can perceive a tree in a blinding snowstorm, a deer bounding across a tree line, dodge a snowball, catch a baseball, detect the crack of a branch breaking in a strong windstorm amidst the rustling of trees, predict the sounds of a dripping faucet, or track a street musician strolling down the road

Haptics: Science, Technology, Applications

This open access book constitutes the proceedings of the 12th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, EuroHaptics 2020, held in Leiden, The Netherlands, in September 2020. The 60 papers presented in this volume were carefully reviewed and selected from 111 submissions. The were organized in topical sections on haptic science, haptic technology, and haptic applications. This year's focus is on accessibility

Embodied geosensification-models, taxonomies and applications for engaging the body in immersive analytics of geospatial data

This thesis examines how we can use immersive multisensory displays and body-focused interaction technologies to analyze geospatial data. It merges relevant aspects from an array of interdisciplinary research areas, from cartography to the cognitive sciences, to form three taxonomies that describe the senses, data representations, and interactions made possible by these technologies. These taxonomies are then integrated into an overarching design model for such "Embodied Geosensifications". This model provides guidance for system specification and is validated with practical examples