Interdisciplinary Film & Digital Media 2015 APR Self-Study & Documents

UNM Interdisciplinary Film & Digital Media APR self-study report, review team report, response to review report, and initial action plan for Spring 2015, fulfilling requirements of the Higher Learning Commission. IFDM was absorbed by the Cinematic Arts Department following this review

Sonophenology: A Tangible Interface for Sonification of Geo-Spatial Phenological Data at Multiple Time-scales

Presented at the 16th International Conference on Auditory Display (ICAD2010) on June 9-15, 2010 in Washington, DC.Phenology is the study of periodic biological processes, such as when plants flower and birds arrive in the spring. In this paper we sonify phenology data and control the sonification process through a tangible interface consisting of a physical paper map and tracking of fiducial markers. The designed interface enables one or more users to concurrently specify point and range queries in both time and space and receive immediate sonic feedback. This system can be used to study and explore the effects of climate change, both as tool to be used by scientists, and as a way to educate members of the general public

Communicating Air: Alternative Pathways to Environmental Knowing through Computational Ecomedia

This dissertation, Communicating Air: Alternative Pathways to Environmental Knowing through Computational Ecomedia, is the culmination of an art practice-led investigation into ways in which the production of ecomedia may open alternative pathways to environmental knowing in a time of urgent climate crisis. This thesis traces the author’s artistic, personal and political development across the period of study and presents an extended argument for greater public engagement with weather and climate science, greater public and private support for long-term collaborations between media art and climate science, and increased public open access to global weather and climate monitoring and computationally modelled data

A novel sonification approach to support the diagnosis of Alzheimer's dementia

Alzheimer’s disease is the most common neurodegenerative form of dementia that steadily worsens and eventually leads to death. Its set of symptoms include loss of cognitive function and memory decline. Structural and functional imaging methods such as CT, MRI, and PET scans play an essential role in the diagnosis process, being able to identify specific areas of cerebral damages. While the accuracy of these imaging techniques increases over time, the severity assessment of dementia remains challenging and susceptible to cognitive and perceptual errors due to intra-reader variability among physicians. Doctors have not agreed upon standardized measurement of cell loss used to specifically diagnose dementia among individuals. These limitations have led researchers to look for supportive diagnosis tools to enhance the spectrum of diseases characteristics and peculiarities. Here is presented a supportive auditory tool to aid in diagnosing patients with different levels of Alzheimer’s. This tool introduces an audible parameter mapped upon three different brain’s lobes. The motivating force behind this supportive auditory technique arise from the fact that AD is distinguished by a decrease of the metabolic activity (hypometabolism) in the parietal and temporal lobes of the brain. The diagnosis is then performed by comparing metabolic activity of the affected lobes to the metabolic activity of other lobes that are not generally affected by AD (i.e., sensorimotor cortex). Results from the diagnosis process compared with the ground truth show that physicians were able to categorize different levels of AD using the sonification generated in this study with higher accuracy than using a standard diagnosis procedure, based on the visualization alone

Responsive Sensate Environments: Past and Future Directions Designing Space as an Interface with Socio-Spatial Information

Abstract: This paper looks at ways in which recent developments in sensing technologies and gestural control of data in 3D space provide opportunities to interact with information. Social and spatial data, the utilisation of space, flows of people and dense abstract data lend themselves to visual and auditory representation to enhance our understanding of socio-spatial patterns. Mapping information to visualisation and sonification leads to gestural interaction with information representation, dissolving the visibility and tangibility of traditional computational interfaces and hardware. The purpose of this integration of new technologies is to blur boundaries between computational and spatial interaction and to transform building spaces into responsive, intelligent interfaces for display and information access. INTRODUCTION Rather than the traditional computer aided architectural design and information communication technology (ICT) integration into architecture, this paper looks designing computer-aided architecture, i.e. spaces and structures enhanced by embedded sensor technologies and responsive (computational) building intelligence. Architecture's responsibility to society could be viewed as designing a sympathetic environment for human experience and interaction. Emerging sensing technologies and intelligence research illuminate interesting opportunities for designing this experience. RESPONSIVE ENVIRONMENTS Responsive environments include sensate spaces, enabled by spatially-and sociallytriggered devices, intelligent and smart houses (utilising video tracking and data capture), networked sensor environments, pervasive mobile computing solutions and ambient visual and auditory displays. This paper briefly reviews the benefits of extant responsive technologies that have developed since last century until th

Building Materials: an installed composition

This research project extends my creative work and unpacks my interest in the use of sonification and mapping as compositional strategies, both in my own practice and more broadly. The thesis reflects on the installed composition, 'Building Materials', synthesising a methodology for the creation of similar works by exploring research problems arising from its creation. The thesis considers the tension between the apparently objective process of mapping and the personal, intuitive, nature of creative practice. This tension establishes a space of uncertainty into which viewers can respond imaginatively to a work built on unseen mappings, granting an audience a sense of the sonified phenomenon. These themes are discussed, and two discrete terms are arrive at: 'installed composition' and 'reverse mapping'. The first contextualises my practice with a descriptor that can help an audience usefullly situate the work and by extension others similar, while the second proposes a model for reading work made using these processes that centres on the relationship between the actual mapped phenomenon and a speculative version in an audience's mind

Spatialized data sonification in a 3D virtual environment

Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2013.Cataloged from PDF version of thesis.Includes bibliographical references (pages 67-69).This thesis explores new ways to communicate sensor data by combining spatialized sonification with data visualiation in a 3D virtual environment. A system for sonifying a space using spatialized recorded audio streams is designed, implemented, and integrated into an existing 3D graphical interface. Exploration of both real-time and archived data is enabled. In particular, algorithms for obfuscating audio to protect privacy, and for time-compressing audio to allow for exploration on diverse time scales are implemented. Synthesized data sonification in this context is also explored.by Nicholas D. Joliat.M. Eng

Data Sonification in Creative Practice

Sonification is the process of data transmission with non-speech audio. While finding increasing acceptance as a scientific method, particularly where a visual representation of data is inadequate, it is still often derided as a ‘gimmick’. Composers have also shown growing interest in sonification as a compositional method. Both in science and in music, the criticism towards this method relates to poor aesthetics and gratuitous applications. This thesis aims to address these issues through an accompanying portfolio of pieces which use sonification as a compositional tool. It establishes the principles of ‘musification’, which can be defined as a sonification which uses musical structures; a sonification organised by musical principles. The practice-as-research portfolio explores a number of data sources, musical genres and science-music collaborations. The main contributions to knowledge derived from the project are a portfolio of compositions, a compositional framework for sonification and an evaluation framework for musification. This thesis demonstrates the validity of practice-as-research as a methodology in sonification research

Using Real-Time Data Flux In Art – The Mediation Of A Situation As It Unfolds: RoadMusic – An Experimental Case Study.

The practice driving this research is called RoadMusic. The project uses a small computer based system installed in a car that composes music from the flux of information it captures about the journey as it unfolds. It uses a technique known as sonification that consists of mapping data to sound. In the case of RoadMusic, this data capture is realtime, external to the computer and mobilised with the user. This dissertation investigates ways in which such a sonification can become an artistic form. To interrogate the specificity of an art of real-time it considers philosophical theories of the fundamental nature of time and immediacy and the ways in which the human mind ‘makes sense’ of this flux. After extending this scrutiny via theories of system and environment, it proceeds to extract concepts and principles leading to a possible art of real-time flux. Time, immediacy and the everyday are recurring questions in art and music, this study reviews practices that address these questions, essentially through three landmark composers of the twentieth century: Iannis Xenakis, John Cage and Murray Schafer. To gain precision in regards to the nature of musical listening it then probes theories of audio cognition and reflects on ways in which these can apply to real-time composing. The art of sonifying data extracted from the environment is arguably only as recent as the computer programs it depends on. This study reviews different practices that contribute towards a corpus of sonification-art, paying special attention to those practices where this process takes place in real-time. This is extended by an interrogation of the effect that mobility has on our listening experience. RoadMusic is now a fully functional device generating multi-timbral music from immediate data about its surroundings. This dissertation argues that this process can be an alternative to mainstream media systems; it describes how RoadMusic’s programs function and the ways in which they have evolved to incorporate the ideas developed in this thesis. It shows how RoadMusic is now developing beyond my own personal practice and how it intends to reach a wider audience

Expanding Eco-Visualization: Sculpting Corn Production

This dissertation expands upon the definition of eco-visualization artwork. EV was originally defined in 2006 by Tiffany Holmes as a way to display the real time consumption statistics of key environmental resources for the goal of promoting ecological literacy. I assert that the final forms of EV artworks are not necessarily dependent on technology, and can differ in terms of media used, in that they can be sculptural, video-based, or static two-dimensional forms that communicate interpreted environmental information. There are two main categories of EV: one that is predominantly screen-based and another that employs a variety of modes of representation to visualize environmental information. EVs are political acts, situated in a charged climate of rising awareness, operating within the context of environmentalism and sustainability. I discuss a variety of EV works within the frame of ecopsychology, including EcoArtTech’s Eclipse and Keith Deverell’s Building Run; Andrea Polli’s Cloud Car and Particle Falls; Nathalie Miebach’s series, The Sandy Rides; and Natalie Jeremijenko’s Mussel Choir. The range of EV works provided models for my creative project, Sculpting Corn Production, and a foundation from which I developed a creative methodology. Working to defeat my experience of solastalgia, Sculpting Corn Production is a series of discrete paper sculptures focusing on American industrial corn farming. This EV also functions as a way for me to understand our devastated monoculture landscapes and the politics, economics, and related areas of ecology of our food production