Tabletop Tangible Interfaces for Music Performance: Design and Evaluation

This thesis investigates a new generation of collaborative systems: tabletop tangible interfaces (TTIs) for music performance or musical tabletops. Musical tabletops are designed for professional musical performance, as well as for casual interaction in public settings. These systems support co-located collaboration, offered by a shared interface. However, we still know little about their challenges and opportunities for collaborative musical practice: in particular, how to best support beginners or experts or both. This thesis explores the nature of collaboration on TTIs for music performance between beginners, experts, or both. Empirical work was done in two stages: 1) an exploratory stage; and 2) an experimental stage. In the exploratory stage we studied the Reactable, a commercial musical tabletop designed for beginners and experts. In particular, we explored its use in two environments: a multi-session study with expert musicians in a casual lab setting; and a field study with casual visitors in a science centre. In the experimental stage we conducted a controlled experiment for mixed groups using a bespoke musical tabletop interface, SoundXY4. The design of this study was informed by the previous stage about a need to support better real-time awareness of the group activity (workspace awareness) in early interactions. For the three studies, groups musical improvisation was video-captured unobtrusively with the aim of understanding natural uses during group musical practice. Rich video data was carefully analysed focusing on the nature of social interaction and how workspace awareness was manifested. The findings suggest that musical tabletops can support peer learning during multiple sessions; fluid between-group social interaction in public settings; and a democratic and ecological approach to music performance. The findings also point to how workspace awareness can be enhanced in early interactions with TTIs using auditory feedback with ambisonics spatialisation. The thesis concludes with theoretical, methodological, and practical implications for future research in New Interfaces for Musical Expression (NIME), tabletop studies, and Human-Computer Interaction (HCI)

Ashitaka: an audiovisual instrument

This thesis looks at how sound and visuals may be linked in a musical instrument, with a view to creating such an instrument. Though it appears to be an area of significant interest, at the time of writing there is very little existing - written, or theoretical - research available in this domain. Therefore, based on Michel Chion’s notion of synchresis in film, the concept of a fused, inseparable audiovisual material is presented. The thesis then looks at how such a material may be created and manipulated in a performance situation. A software environment named Heilan was developed in order to provide a base for experimenting with different approaches to the creation of audiovisual instruments. The software and a number of experimental instruments are discussed prior to a discussion and evaluation of the final ‘Ashitaka’ instrument. This instrument represents the culmination of the work carried out for this thesis, and is intended as a first step in identifying the issues and complications involved in the creation of such an instrument

The generative, analytic and instructional capacities of sound in architecture : fundamentals, tools and evaluation of a design methodology

The disciplines of space and time form two domains to which it is daring to compare, since it is obvious that they are of a different nature. Music happens in time, while architecture happens in space. However, from the first treatises on both architecture and music, repeated calls for comparison, complementarity and influence of both disciplines can be read, at least to the observation of certain common orders between the two domains. In this doctoral thesis we do not question this whole theoretical corpus that has been enriching the relationship between both disciplines. We received it and joined that stream of knowledge. What we do notice, however, is the almost impertinent question that follows: can sound help the architect in his daily tasks? And, therefore, what are the contributions of sound to the architect? To do this we must seek the connection in the principles of both arts, where we can detach ourselves from time and space, and approach the most universal of art forms. The architect, in his daily work, is faced with three particular tasks: the architectural project, the architectural analysis and the teaching of architecture. Each of the three tasks is connected with the other two tasks: the project is carried out again with the analysis and transmitted to the new architect; the analysis supports the project decisions and gives tools to the disciple; and the teaching has the project as its purpose and the analysis as its method. The thesis presented here shows what sound offers to the task of the project, to that of analysis and to that of teaching. These three tasks are approached from three premises: theoretical foundations, tools and evaluation. The interaction of the three tasks with the three premises gives rise to nine lines of work that articulate the chapters of the thesis. The first, fourth and seventh chapters approach the three tasks from the premise of theoretical foundations, foundations that perhaps because they are obvious, have been ignored or overlooked but which constitute the nature of both disciplines. The first shows, by the hand of two 20th century authors - the architect Dom Hans van der Laan and the composer Olivier Messiaen - that creation in both disciplines is of a systematic nature. The fourth one revaluates the analytical systems of representation of form both in architecture and in music which, starting with the basic characteristics of its elements, lead to a symbolic notation and a tool for the analysis of the work: the plan and the score. The seventh introduces the student of architecture to the growing separation between music and architecture that has been accentuated to this day. The second, fifth and eighth chapters approach the three particular tasks from the premise of tools, working tools that help to understand more directly the influence of architecture on sound. The second places virtual reality and auralization techniques at the service of the architectural and urban planning project, enhancing the sound experience in these projects. The fifth deals with the acoustic analysis of exterior spaces and their relationship with the urban configuration of these spaces. The eighth section presents the study of acoustic heritage as an educational tool. The third, sixth and ninth chapters deal with the three tasks from the premise of evaluation, a check that ensures the influence of sound on them through teaching experiments. The third argues and exemplifies that a sound landscape can be the engine and generator of an architectural design. The sixth one reviews the methods for evaluating the subjective and objective parameters of architectural acoustics. The ninth shows that in teaching sound to architects, "learning by listening" should be given priority over "passive learning".Las disciplinas del espacio y del tiempo forman dos dominios a los que resulta atrevido comparar, pues es obvio que son de naturaleza distinta. La música ocurre en el tiempo, mientras que la arquitectura en el espacio. No obstante, desde los primeros tratados tanto de arquitectura como de música, se pueden leer repetidas llamadas a la comparación, al complemento y a la influencia de ambas disciplinas, cuanto menos a la constatación de ciertos órdenes comunes entre ambos dominios. En esta tesis doctoral no ponemos en cuestión todo este corpus teórico que ha venido enriqueciendo la relación entre ambas disciplinas. La recibimos y nos unimos a esa corriente de conocimiento. En lo que sí reparamos, en cambio, es en la pregunta casi impertinente que surge seguidamente: ¿puede el sonido ayudar al arquitecto en sus tareas diarias? Y, por tanto, ¿cuáles son las contribuciones del sonido para el arquitecto? Para ello debemos buscar la conexión en los principios de ambas artes, allí donde podemos despegarnos del tiempo y del espacio, y acercarnos a la más universal de las formas de arte. El arquitecto, en su tarea diaria, se enfrenta a tres tareas particulares: el proyecto arquitectónico, el análisis arquitectónico y la enseñanza de la arquitectura. Cada una de las tres tareas está conectada con las otras dos: el proyecto se reconduce con el análisis y se transmite al nuevo arquitecto; el análisis soporta las decisiones de proyecto y da herramientas al discípulo; y la enseñanza tiene como fin el proyecto y como método el análisis. La tesis aquí presentada pone de manifiesto lo que el sonido ofrece a la tarea del proyecto, a la del análisis y a la de la enseñanza. Estas tres tareas son abordadas desde tres premisas: los fundamentos teóricos, las herramientas y la evaluación. La interacción de las tres tareas con las tres premisas da lugar a nueve líneas de trabajo que articulan los capítulos de la tesis. Los capítulos primero, cuarto y séptimo abordan las tres tareas desde la premisa de los fundamentos teóricos, fundamentos que quizá por ser obvios, se han obviado o pasado por alto pero que constituyen la naturaleza de ambas disciplinas. El primero muestra, de la mano de dos autores del siglo XX ?el arquitecto Dom Hans van der Laan y el compositor Olivier Messiaen- que la creación en ambas disciplinas es de naturaleza sistemática. El cuarto revaloriza los sistemas analíticos de representación de la forma tanto en arquitectura como en música que, empezando por las características básicas de sus elementos, conducen a una notación simbólica y una herramienta de análisis de la obra: el plano y la partitura. El séptimo presenta al estudiante de arquitectura la creciente separación entre la música y la arquitectura que se ha venido acentuando hasta nuestros días. Los capítulos segundo, quinto y octavo abordan las tres tareas particulares desde la premisa de las herramientas, útiles de trabajo que ayudan a comprender de modo más directo la influencia de la arquitectura en el sonido. El segundo sitúa la realidad virtual y las técnicas de auralización al servicio del proyecto de arquitectura y urbanismo, potenciando la experiencia sonora en estos proyectos. El quinto aborda el análisis acústico de espacios exteriores y su relación con la configuración urbana de estos espacios. El octavo presenta el estudio del patrimonio acústico como herramienta pedagógica. Los capítulos tercero, sexto y noveno abordan las tres tareas desde la premisa de la evaluación, comprobación que asegura mediante experimentos docentes la influencia del sonido en ellas. El tercero argumenta y ejemplifica que un paisaje sonoro puede ser el motor y generador de un diseño arquitectónico. El sexto realiza una revisión de los métodos de evaluación de los parámetros subjetivos y objetivos de la acústica arquitectónica. El noveno muestra que en la enseñanza del sonido para los arquitectos debe priorizarse "aprender escuchando" antes que el "aprendizaje pasivo"

The generative, analytic and instructional capacities of sound in architecture : fundamentals, tools and evaluation of a design methodology

Premi extraordinari doctorat UPC curs 2017-2018, Àmbit d’Arquitectura, Urbanisme i EdificacióThe disciplines of space and time form two domains to which it is daring to compare, since it is obvious that they are of a different nature. Music happens in time, while architecture happens in space. However, from the first treatises on both architecture and music, repeated calls for comparison, complementarity and influence of both disciplines can be read, at least to the observation of certain common orders between the two domains. In this doctoral thesis we do not question this whole theoretical corpus that has been enriching the relationship between both disciplines. We received it and joined that stream of knowledge. What we do notice, however, is the almost impertinent question that follows: can sound help the architect in his daily tasks? And, therefore, what are the contributions of sound to the architect? To do this we must seek the connection in the principles of both arts, where we can detach ourselves from time and space, and approach the most universal of art forms. The architect, in his daily work, is faced with three particular tasks: the architectural project, the architectural analysis and the teaching of architecture. Each of the three tasks is connected with the other two tasks: the project is carried out again with the analysis and transmitted to the new architect; the analysis supports the project decisions and gives tools to the disciple; and the teaching has the project as its purpose and the analysis as its method. The thesis presented here shows what sound offers to the task of the project, to that of analysis and to that of teaching. These three tasks are approached from three premises: theoretical foundations, tools and evaluation. The interaction of the three tasks with the three premises gives rise to nine lines of work that articulate the chapters of the thesis. The first, fourth and seventh chapters approach the three tasks from the premise of theoretical foundations, foundations that perhaps because they are obvious, have been ignored or overlooked but which constitute the nature of both disciplines. The first shows, by the hand of two 20th century authors - the architect Dom Hans van der Laan and the composer Olivier Messiaen - that creation in both disciplines is of a systematic nature. The fourth one revaluates the analytical systems of representation of form both in architecture and in music which, starting with the basic characteristics of its elements, lead to a symbolic notation and a tool for the analysis of the work: the plan and the score. The seventh introduces the student of architecture to the growing separation between music and architecture that has been accentuated to this day. The second, fifth and eighth chapters approach the three particular tasks from the premise of tools, working tools that help to understand more directly the influence of architecture on sound. The second places virtual reality and auralization techniques at the service of the architectural and urban planning project, enhancing the sound experience in these projects. The fifth deals with the acoustic analysis of exterior spaces and their relationship with the urban configuration of these spaces. The eighth section presents the study of acoustic heritage as an educational tool. The third, sixth and ninth chapters deal with the three tasks from the premise of evaluation, a check that ensures the influence of sound on them through teaching experiments. The third argues and exemplifies that a sound landscape can be the engine and generator of an architectural design. The sixth one reviews the methods for evaluating the subjective and objective parameters of architectural acoustics. The ninth shows that in teaching sound to architects, "learning by listening" should be given priority over "passive learning".Las disciplinas del espacio y del tiempo forman dos dominios a los que resulta atrevido comparar, pues es obvio que son de naturaleza distinta. La música ocurre en el tiempo, mientras que la arquitectura en el espacio. No obstante, desde los primeros tratados tanto de arquitectura como de música, se pueden leer repetidas llamadas a la comparación, al complemento y a la influencia de ambas disciplinas, cuanto menos a la constatación de ciertos órdenes comunes entre ambos dominios. En esta tesis doctoral no ponemos en cuestión todo este corpus teórico que ha venido enriqueciendo la relación entre ambas disciplinas. La recibimos y nos unimos a esa corriente de conocimiento. En lo que sí reparamos, en cambio, es en la pregunta casi impertinente que surge seguidamente: ¿puede el sonido ayudar al arquitecto en sus tareas diarias? Y, por tanto, ¿cuáles son las contribuciones del sonido para el arquitecto? Para ello debemos buscar la conexión en los principios de ambas artes, allí donde podemos despegarnos del tiempo y del espacio, y acercarnos a la más universal de las formas de arte. El arquitecto, en su tarea diaria, se enfrenta a tres tareas particulares: el proyecto arquitectónico, el análisis arquitectónico y la enseñanza de la arquitectura. Cada una de las tres tareas está conectada con las otras dos: el proyecto se reconduce con el análisis y se transmite al nuevo arquitecto; el análisis soporta las decisiones de proyecto y da herramientas al discípulo; y la enseñanza tiene como fin el proyecto y como método el análisis. La tesis aquí presentada pone de manifiesto lo que el sonido ofrece a la tarea del proyecto, a la del análisis y a la de la enseñanza. Estas tres tareas son abordadas desde tres premisas: los fundamentos teóricos, las herramientas y la evaluación. La interacción de las tres tareas con las tres premisas da lugar a nueve líneas de trabajo que articulan los capítulos de la tesis. Los capítulos primero, cuarto y séptimo abordan las tres tareas desde la premisa de los fundamentos teóricos, fundamentos que quizá por ser obvios, se han obviado o pasado por alto pero que constituyen la naturaleza de ambas disciplinas. El primero muestra, de la mano de dos autores del siglo XX -el arquitecto Dom Hans van der Laan y el compositor Olivier Messiaen- que la creación en ambas disciplinas es de naturaleza sistemática. El cuarto revaloriza los sistemas analíticos de representación de la forma tanto en arquitectura como en música que, empezando por las características básicas de sus elementos, conducen a una notación simbólica y una herramienta de análisis de la obra: el plano y la partitura. El séptimo presenta al estudiante de arquitectura la creciente separación entre la música y la arquitectura que se ha venido acentuando hasta nuestros días. Los capítulos segundo, quinto y octavo abordan las tres tareas particulares desde la premisa de las herramientas, útiles de trabajo que ayudan a comprender de modo más directo la influencia de la arquitectura en el sonido. El segundo sitúa la realidad virtual y las técnicas de auralización al servicio del proyecto de arquitectura y urbanismo, potenciando la experiencia sonora en estos proyectos. El quinto aborda el análisis acústico de espacios exteriores y su relación con la configuración urbana de estos espacios. El octavo presenta el estudio del patrimonio acústico como herramienta pedagógica. Los capítulos tercero, sexto y noveno abordan las tres tareas desde la premisa de la evaluación, comprobación que asegura mediante experimentos docentes la influencia del sonido en ellas. El tercero argumenta y ejemplifica que un paisaje sonoro puede ser el motor y generador de un diseño arquitectónico. El sexto realiza una revisión de los métodos de evaluación de los parámetros subjetivos y objetivos de la acústica arquitectónica. El noveno muestra que en la enseñanza del sonido para los arquitectos debe priorizarse "aprender escuchando" antes que el "aprendizaje pasivo".Award-winningPostprint (published version

Between Air and Electricity

After the sound reproduction industry had claimed “perfect high fidelity” for sound recordings already at the beginning of the twentieth century, composers and sound artists challenged this perfection by tweaking microphones and loudspeakers to make them act as a musical instrument instead of a mere sound reproduction device. This book explores the instrumental use of microphones and loudspeakers in music beginning in the 1950s. The popular noise musician Merzbow, over-minimalist classic Alvin Lucier, cult instrument inventor Hugh Davies, and contemporary visual artist Lynn Pook made audible what was supposed to remain silent

Between Air and Electricity

After the sound reproduction industry had claimed “perfect high fidelity” for sound recordings already at the beginning of the twentieth century, composers and sound artists challenged this perfection by tweaking microphones and loudspeakers to make them act as a musical instrument instead of a mere sound reproduction device. This book explores the instrumental use of microphones and loudspeakers in music beginning in the 1950s. The popular noise musician Merzbow, over-minimalist classic Alvin Lucier, cult instrument inventor Hugh Davies, and contemporary visual artist Lynn Pook made audible what was supposed to remain silent

Exploring Animal Behavior Through Sound: Volume 1

This open-access book empowers its readers to explore the acoustic world of animals. By listening to the sounds of nature, we can study animal behavior, distribution, and demographics; their habitat characteristics and needs; and the effects of noise. Sound recording is an efficient and affordable tool, independent of daylight and weather; and recorders may be left in place for many months at a time, continuously collecting data on animals and their environment. This book builds the skills and knowledge necessary to collect and interpret acoustic data from terrestrial and marine environments. Beginning with a history of sound recording, the chapters provide an overview of off-the-shelf recording equipment and analysis tools (including automated signal detectors and statistical methods); audiometric methods; acoustic terminology, quantities, and units; sound propagation in air and under water; soundscapes of terrestrial and marine habitats; animal acoustic and vibrational communication; echolocation; and the effects of noise. This book will be useful to students and researchers of animal ecology who wish to add acoustics to their toolbox, as well as to environmental managers in industry and government