126 research outputs found
The Seaboard: discreteness and continuity in musical interface design
The production of acoustic music bridges two senses—touch and hearing—by connecting
physical movements, gestures, and tactile interactions with the creation of sound.
Mastery of acoustic music depends on the development and refinement of muscle
memory and ear training in concert. This process leads to a capacity for great depth of
expression even though the actual timbral palette of each given acoustic instrument is
relatively limited. By contrast, modern modes of music creation involving recorded music
and digital sound manipulation sacrifice this immediate bridge and substitute more
abstract processes that enable sonic possibilities extending far beyond the acoustic
palette. Mastery in abstract approaches to music making doesn’t necessarily rely on
muscle memory or ear training, as many key processes do not need to happen in realtime.
This freedom from the limits of time and practiced physical manipulation radically
increases the range of achievable sounds, rhythms and effects, but sometimes results in a
loss of subtlety of expressiveness.
This practice-based PhD asks whether it is possible, and if so how, to achieve an
integration of relevant sensor technologies, design concepts, and formation techniques
to create a new kind of musical instrument and sound creation tool that bridges this gap
with a satisfying result for musicians and composers. In other words, can one create new,
multi-dimensional interfaces which provide more effective ways to control the expressive
capabilities of digital music creation in real-time? In particular, can one build on the
intuitive, logical, and well-known layout of the piano keyboard to create a new instrument
that more fully enables both continuous and discrete approaches to music making?
My research practice proposes a new musical instrument called the Seaboard, documents
its invention, development, design, and refinement, and evaluates the extent to which it
positively answers the above question. The Seaboard is a reinterpretation of the piano
keyboard as a soft, continuous wavelike surface that places polyphonic pitch bend,
vibrato and continuous touch right at the musician’s fingertips. The addition of new realtime
parameters to a familiar layout means it combines the intuitiveness of the traditional
instrument with some of the versatility of digital technology.
Designing and prototyping the Seaboard to the point of successfully proving that a new
synthesis between acoustic techniques and digital technologies is possible is shown to
require significant coordination and integration of a range of technical disciplines. The
research approach has been to build and refine a series of prototypes that successively
grapple with the integration of these elements, whilst rigorously documenting the design
issues, engineering challenges, and ultimate decisions that determine whether an
intervention in the field of musical instrumentation is fruitful
Multisensory learning in adaptive interactive systems
The main purpose of my work is to investigate multisensory perceptual learning and sensory integration in the design and development of adaptive user interfaces for educational purposes. To this aim, starting from renewed understanding from neuroscience and cognitive science on multisensory perceptual learning and sensory integration, I developed a theoretical computational model for designing multimodal learning technologies that take into account these results. Main theoretical foundations of my research are multisensory perceptual learning theories and the research on sensory processing and integration, embodied cognition theories, computational models of non-verbal and emotion communication in full-body movement, and human-computer interaction models. Finally, a computational model was applied in two case studies, based on two EU ICT-H2020 Projects, "weDRAW" and "TELMI", on which I worked during the PhD
Safe and Sound: Proceedings of the 27th Annual International Conference on Auditory Display
Complete proceedings of the 27th International Conference on Auditory Display (ICAD2022), June 24-27. Online virtual conference
Interactive Sound in Performance Ecologies: Studying Connections among Actors and Artifacts
This thesis’s primary goal is to investigate performance ecologies, that is the compound
of humans, artifacts and environmental elements that contribute to the result of a per-
formance. In particular, this thesis focuses on designing new interactive technologies for
sound and music. The goal of this thesis leads to the following Research Questions (RQs):
• RQ1 How can the design of interactive sonic artifacts support a joint expression
across different actors (composers, choreographers, and performers, musicians, and
dancers) in a given performance ecology?
• RQ2 How does each different actor influence the design of different artifacts, and
what impact does this have on the overall artwork?
• RQ3 How do the different actors in the same ecology interact, and appropriate an
interactive artifact?
To reply to these questions, a new framework named ARCAA has been created. In this
framework, all the Actors of a given ecology are connected to all the Artifacts throughout
three layers: Role, Context and Activity. This framework is then applied to one systematic
literature review, two case studies on music performance and one case study in dance
performance. The studies help to better understand the shaded roles of composers, per-
formers, instrumentalists, dancers, and choreographers, which is relevant to better design
interactive technologies for performances. Finally, this thesis proposes a new reflection on
the blurred distinction between composing and designing a new instrument in a context
that involves a multitude of actors.
Overall, this work introduces the following contributions to the field of interaction
design applied to music technology: 1) ARCAA, a framework to analyse the set of inter-
connected relationship in interactive (music) performances, validated through 2 music
studies, 1 dance study and 1 systematic literature analysis; 2) Recommendations for de-
signing music interactive system for performance (music or dance), accounting for the
needs of the various actors and for the overlapping on music composition and design of in-
teractive technology; 3) A taxonomy of how scores have shaped performance ecologies in NIME, based on a systematic analysis of the literature on score in the NIME proceedings;
4) Proposal of a methodological approach combining autobiographical and idiographical
design approaches in interactive performances.O objetivo principal desta tese Ă© investigar as ecologias performativas, conjunto formado
pelos participantes humanos, artefatos e elementos ambientais que contribuem para o
resultado de uma performance. Em particular, esta tese foca-se na conceção de novas
tecnologias interativas para som e mĂşsica. O objetivo desta tese originou as seguintes
questões de investigação (Research Questions RQs):
• RQ1 Como o design de artefatos sonoros interativos pode apoiar a expressão con-
junta entre diferentes atores (compositores, coreĂłgrafos e performers, mĂşsicos e
dançarinos) numa determinada ecologia performativa?
• RQ2 Como cada ator influencia o design de diferentes artefatos e que impacto isso
tem no trabalho artĂstico global?
• RQ3 Como os diferentes atores de uma mesma ecologia interagem e se apropriam
de um artefato interativo?
Para responder a essas perguntas, foi criado uma nova framework chamada ARCAA.
Nesta framework, todos os atores (Actores) de uma dada ecologia estĂŁo conectados a todos
os artefatos (Artefacts) através de três camadas: Role, Context e Activity. Esta framework
foi então aplicada a uma revisão sistemática da literatura, a dois estudos de caso sobre
performance musical e a um estudo de caso em performance de dança. Estes estudos aju-
daram a comprender melhor os papéis desempenhados pelos compositores, intérpretes,
instrumentistas, dançarinos e coreógrafos, o que é relevante para melhor projetar as tec-
nologias interativas para performances. Por fim, esta tese propõe uma nova reflexão sobre
a distinção entre compor e projetar um novo instrumento num contexto que envolve uma
multiplicidade de atores.
Este trabalho apresenta as seguintes contribuições principais para o campo do design
de interação aplicado à tecnologia musical: 1) ARCAA, uma framework para analisar o
conjunto de relações interconectadas em performances interativas, validado através de
dois estudos de caso relacionados com a mĂşsica, um estudo de caso relacionado com
a dança e uma análise sistemática da literatura; 2) Recomendações para o design de sistemas interativos musicais para performance (música ou dança), tendo em conta as
necessidades dos vários atores e a sobreposição entre a composição musical e o design de
tecnologia interactiva; 3) Uma taxonomia sobre como as partituras musicais moldaram
as ecologias performativas no NIME, com base numa análise sistemática da literatura
dos artigos apresentados e publicados nestas conferĂŞncia; 4) Proposta de uma aborda-
gem metodológica combinando abordagens de design autobiográfico e idiográfico em
performances interativas
Recommended from our members
The Birmingham screwdriver is a golden hammer: developing strategies for performing scientific objects
The demonstration of science is a complex activity that can offer more than a passive one way transmission of specialist knowledge from experts to a non specialist audience. Artists can play an active role in creating 'matters of fact' and at the same time inform the cultural conditions which put these matters of fact into context. In examining some current art/science collaborations between performance art and physics the project identifies some problems relating to a pre-occupation with veracity and authenticity, technological fetishism and issues to do with the representation of data.
An in depth study of prominent art duo Semiconductor offers insights into the difficulties and opportunities encountered in this work, particularly in the context of the artist in residence in the science institution. My own three month residency at the British Geological Society's Space Geodesy Facility in Herstmonceux provided an opportunity to engage with these issues first hand.
Through the creation of a body of practical work, the project explores strategies for performing scientific objects. Through the application of a heuristic here called 'the Birmingham Screwdriver' several forms of creative resistance (Norman 2013) are identified and put to use. I examine the hypothesis that the 'wrong tool' is an essential and inevitable characteristic of knowledge exchange, whether between an expert and a lay audience, experts from different fields or between research institutions and the wider cultural context in which they take place. Using the artist in science as an analogy for art practice being involved in other research institutions and as a model of conducting research applicable to more general circumstances I aim to contribute to understandings of an 'artistic epistemology' (Schwab 2015)
UTB/TSC Legacy Degree Programs and Courses 2010 – 2011
https://scholarworks.utrgv.edu/brownsvillelegacycatalogs/1026/thumbnail.jp
UTB/TSC Legacy Degree Programs and Courses 2011 – 2012
https://scholarworks.utrgv.edu/brownsvillelegacycatalogs/1027/thumbnail.jp
- …