What does " Evaluation " mean for the NIME community?

International audienceEvaluation has been suggested to be one of the main trends in current NIME research. However, the meaning of the term for the community may not be as clear as it seems. In order to explore this issue, we have analyzed all papers and posters published in the proceedings of the NIME conference from 2012 to 2014. For each publication that explicitly mentioned the term "evaluation", we looked for: a) What targets and stakeholders were considered? b) What goals were set? c) What criteria were used? d) What methods were used? e) How long did the evaluation last? Results show different understandings of evaluation, with little consistency regarding the usage of the word. Surprisingly in some cases, not even basic information such as goal, criteria and methods were provided. In this paper, we attempt to provide an idea of what "evaluation" means for the NIME community, pushing the discussion towards how could we make a better use of evaluation on NIME design and what criteria should be used regarding each goal

Interactive Sound in Performance Ecologies: Studying Connections among Actors and Artifacts

This thesis’s primary goal is to investigate performance ecologies, that is the compound of humans, artifacts and environmental elements that contribute to the result of a per- formance. In particular, this thesis focuses on designing new interactive technologies for sound and music. The goal of this thesis leads to the following Research Questions (RQs): • RQ1 How can the design of interactive sonic artifacts support a joint expression across different actors (composers, choreographers, and performers, musicians, and dancers) in a given performance ecology? • RQ2 How does each different actor influence the design of different artifacts, and what impact does this have on the overall artwork? • RQ3 How do the different actors in the same ecology interact, and appropriate an interactive artifact? To reply to these questions, a new framework named ARCAA has been created. In this framework, all the Actors of a given ecology are connected to all the Artifacts throughout three layers: Role, Context and Activity. This framework is then applied to one systematic literature review, two case studies on music performance and one case study in dance performance. The studies help to better understand the shaded roles of composers, per- formers, instrumentalists, dancers, and choreographers, which is relevant to better design interactive technologies for performances. Finally, this thesis proposes a new reflection on the blurred distinction between composing and designing a new instrument in a context that involves a multitude of actors. Overall, this work introduces the following contributions to the field of interaction design applied to music technology: 1) ARCAA, a framework to analyse the set of inter- connected relationship in interactive (music) performances, validated through 2 music studies, 1 dance study and 1 systematic literature analysis; 2) Recommendations for de- signing music interactive system for performance (music or dance), accounting for the needs of the various actors and for the overlapping on music composition and design of in- teractive technology; 3) A taxonomy of how scores have shaped performance ecologies in NIME, based on a systematic analysis of the literature on score in the NIME proceedings; 4) Proposal of a methodological approach combining autobiographical and idiographical design approaches in interactive performances.O objetivo principal desta tese é investigar as ecologias performativas, conjunto formado pelos participantes humanos, artefatos e elementos ambientais que contribuem para o resultado de uma performance. Em particular, esta tese foca-se na conceção de novas tecnologias interativas para som e música. O objetivo desta tese originou as seguintes questões de investigação (Research Questions RQs): • RQ1 Como o design de artefatos sonoros interativos pode apoiar a expressão con- junta entre diferentes atores (compositores, coreógrafos e performers, músicos e dançarinos) numa determinada ecologia performativa? • RQ2 Como cada ator influencia o design de diferentes artefatos e que impacto isso tem no trabalho artístico global? • RQ3 Como os diferentes atores de uma mesma ecologia interagem e se apropriam de um artefato interativo? Para responder a essas perguntas, foi criado uma nova framework chamada ARCAA. Nesta framework, todos os atores (Actores) de uma dada ecologia estão conectados a todos os artefatos (Artefacts) através de três camadas: Role, Context e Activity. Esta framework foi então aplicada a uma revisão sistemática da literatura, a dois estudos de caso sobre performance musical e a um estudo de caso em performance de dança. Estes estudos aju- daram a comprender melhor os papéis desempenhados pelos compositores, intérpretes, instrumentistas, dançarinos e coreógrafos, o que é relevante para melhor projetar as tec- nologias interativas para performances. Por fim, esta tese propõe uma nova reflexão sobre a distinção entre compor e projetar um novo instrumento num contexto que envolve uma multiplicidade de atores. Este trabalho apresenta as seguintes contribuições principais para o campo do design de interação aplicado à tecnologia musical: 1) ARCAA, uma framework para analisar o conjunto de relações interconectadas em performances interativas, validado através de dois estudos de caso relacionados com a música, um estudo de caso relacionado com a dança e uma análise sistemática da literatura; 2) Recomendações para o design de sistemas interativos musicais para performance (música ou dança), tendo em conta as necessidades dos vários atores e a sobreposição entre a composição musical e o design de tecnologia interactiva; 3) Uma taxonomia sobre como as partituras musicais moldaram as ecologias performativas no NIME, com base numa análise sistemática da literatura dos artigos apresentados e publicados nestas conferência; 4) Proposta de uma aborda- gem metodológica combinando abordagens de design autobiográfico e idiográfico em performances interativas

Designing speculative artifacts

In recent decades, Mark Weiser's vision of ubiquitous computing has become today's reality through embedded electronics, the rise of machine learning, and the proliferation of wireless Internet access. This development brings not only opportunities but also new challenges for the Human-Computer Interaction (HCI) community. Technology, for example, is (1) entering novel application scenarios and pristine interaction environments, (2) leaving the screen and conquering the physical world, and (3) changing the previous tool-like nature of computers to social, self-learning, and pro-active entities. As a result, there are no already established interaction paradigms, metaphors, and design strategies for designing such systems. Thus, HCI faces questions about the interaction design in the context of embedded systems and novel materials, the conceptualization of intelligent systems in everyday environments, and, subsequently, the consequences on human-technology relations. To approach such questions, a standard tool in HCI research is the human-centered design process, which creates knowledge about user needs and considers user perspectives to inform design decisions. Observations and interviews are used to understand the context, workflow, or tools, before developing ideas and concepts for technological improvements or solutions. This process has proven to be effective when dealing with matters familiar to users, such as their workplaces, leading to improved workflows and experiences. However, it remains open how design can be grounded if future technologies result in unfamiliar situations. When people can no longer contribute with their domain knowledge, what are novel interaction concepts, paradigms, and designs based on? To tackle these problems, I present and discuss a programmatic design approach to generate original design ideas and concepts. This approach builds on Speculative and Critical Design practices within the HCI context. The main idea central to this work is to create designs using real-world patterns to inform HCI since these patterns still comply with users' prior knowledge, experiences, and perception of fundamental social or natural principles. Such Real-World Patterns (RWP) can be familiar metaphors, morphologies, or mental models - e.g., understanding causality in the physical world or knowing the basic working principles of musical instruments. These patterns are chosen and then transferred into designs to meet or contradict users' expectations of the technology in order to create confrontational situations in which new perspectives are opened up. Based on the confrontation with the speculative artifacts, implications and requirements are deduced, which in reverse can be applied to future technologies. The projects presented apply this approach in various HCI research domains, including human-robot interaction, new interfaces for musical expression, and deformable and flexible interfaces. In this dissertation, I reflect on the approach using three questions: (Q1) How does the use of RWPs complement the Research through Design practice?; (Q2) How can RWPs be instrumentalized in the design of HCI systems?; and (Q3) How does the use of RWPs in design affect the relation between humans and technology? This thesis contributes (1) an overview of the approach as well as three perspectives which are instrumental in understanding and applying RWPs in the design of HCI, (2) eight speculative artifacts, which exemplify the approach, (3) theories and concepts inspired by the used RWPs, and (4) empirical knowledge deduced from the associated studies and surveys.In den letzten Jahrzehnten ist Mark Weisers Vision des Ubiquitous Computing durch eingebettete Systeme, den Aufstieg künstlicher Intelligenz und die Verbreitung des drahtlosen Internetzugangs zur heutigen Lebensrealität geworden. Diese Entwicklung bringt nicht nur Chancen, sondern auch neue Herausforderungen für die Human Computer Interaction (HCI) Community mit sich. Technologien dringen bspw. in neuartige Anwendungsszenarien und noch unberührte Interaktionsumgebungen ein, verlassen den Bildschirm und erobern die physische Welt und verändern so den bisherigen werkzeugartigen Charakter von Computern zu sozialen, selbstlernenden und proaktiven Entitäten. Infolgedessen gibt es keine bereits etablierten Interaktionsparadigmen, Metaphern und Designstrategien, auf die für die Gestaltung solcher Systeme zurückgegriffen werden könnte. Für die HCI stellen sich daher Fragen zur Interaktionsgestaltung im Kontext eingebetteter Systeme und neuartiger Materialien, zur Konzeptualisierung intelligenter Maschinen in Alltagsumgebungen und folglich zu den Auswirkungen auf die Beziehung, in der Mensch und Technologie zueinander stehen. Um sich solchen Fragen zu nähern, ist in der HCI der Human-Centered Design Prozess eine verbreitete Vorgehensweise, die Erkenntnisse über die Bedürfnisse der Benutzer:innen schafft und deren Perspektive bei Designentscheidungen berücksichtigt. Beobachtungen und Interviews werden genutzt, um den Kontext, die Arbeitsabläufe oder Werkzeuge zu verstehen, bevor Ideen und Konzepte für technische Lösungen entwickelt werden. Dieses Vorgehen hat sich als effektiv erwiesen und führt zu Verbesserungen von Technologien, solange die betreffenden Situationen den Nutzer:innen vertraut sind. Offen bleibt jedoch, worauf Design-Konzepte bauen können, wenn zukünftige Technologien zu ungewohnten Situationen führen. Worauf basieren neuartige Interaktionskonzepte, Paradigmen und Entwürfe, wenn Menschen nicht mehr mit ihrem Fachwissen und Erfahrungen beitragen können? Um diese Problematik anzugehen, präsentiere und diskutiere ich in dieser Dissertation einen programmatischen Ansatz, um neuartige Designideen und -konzepte zu entwickeln. Dieser baut auf den Praktiken des spekulativen und kritischen Designs im HCI-Kontext auf. Im Mittelpunkt steht die Entwicklung von Designs unter Verwendung von Mustern aus der realen Welt, da diese das Vorwissen der Nutzer:innen sowie soziale und natürliche Prinzipien einbeziehen. Bei solchen Real-World Patterns (RWP) kann es sich um vertraute Metaphern, Morphologien oder mentale Modelle handeln, bspw. um das Verständnis von Kausalität in der physischen Welt oder um die Kenntnis der grundlegenden Funktionsprinzipien von Musikinstrumenten. Potentielle Muster werden ausgewählt und derart in Entwürfe übertragen, dass die Erwartungen der Nutzer:innen an die Technologie erfüllt oder ihnen widersprochen wird. So werden konfrontative Situationen geschaffen, in denen sich neue Perspektiven eröffnen. Aus der Konfrontation mit den spekulativen Artefakten werden Implikationen und Anforderungen abgeleitet, die sich im Umkehrschluss auf zukünftige Technologien anwenden lassen. Die vorgestellten Projekte wenden diesen Ansatz in verschiedenen HCI-Forschungsbereichen an, unter anderem in der Mensch-Roboter-Interaktion, im Kontext neuer Schnittstellen für die musikalische Interaktion, sowie für deformierbare und flexible Schnittstellen. In dieser Dissertation reflektiere ich den Ansatz anhand dreier Fragen: (Q1) Wie ergänzt der Einsatz von RWPs die Research through Design Praxis?; (Q2) Wie können RWPs für das Design von HCI-Systemen instrumentalisiert werden?; und (Q3) Wie beeinflusst der Einsatz von RWPs die Beziehung zwischen Mensch und Technologie? Der Beitrag dieser Arbeit liegt in (1) einem Überblick über den Ansatz sowie in den drei Perspektiven, die für das Verständnis und die Anwendung von RWPs in der Gestaltung von HCI maßgeblich sind, (2) den acht spekulativen Artefakten, die den Ansatz beispielhaft darstellen, (3) den Theorien und Konzepten, die von den verwendeten RWPs inspiriert sind, und (4) den empirischen Erkenntnissen, die aus den zugehörigen Studien und Umfragen abgeleitet wurden

Cantor Digitalis: chironomic parametric synthesis of singing

Cantor Digitalis is a performative singing synthesizer that is composed of two main parts: a chironomic control interface and a parametric voice synthesizer. The control interface is based on a pen/touch graphic tablet equipped with a template representing vocalic and melodic spaces. Hand and pen positions, pen pressure, and a graphical user interface are assigned to specific vocal controls. This interface allows for real-time accurate control over high-level singing synthesis parameters. The sound generation system is based on a parametric synthesizer that features a spectral voice source model, a vocal tract model consisting of parallel filters for vocalic formants and cascaded with anti-resonance for the spectral effect of hypo-pharynx cavities, and rules for parameter settings and source/filter dependencies between fundamental frequency, vocal effort, and formants. Because Cantor Digitalis is a parametric system, every aspect of voice quality can be controlled (e.g., vocal tract size, aperiodicities in the voice source, vowels, and so forth). It offers several presets for different voice types. Cantor Digitalis has been played on stage in several public concerts, and it has also been proven to be useful as a tool for voice pedagogy. The aim of this article is to provide a comprehensive technical overview of Cantor Digitalis

Gesture elicitation to improve DJ to audience communication

When DJs perform they struggle to communicate their performance actions to their audience. DJs use turntables, CDJs, DJ mixers and/or computers with or without hardware DJ controllers. This equipment has small controls that are difficult to view from any distance. DJs bend over their equipment while performing small hand movements that are difficult for the audience to see. This research aims to investigate whether this visual communication problem can be solved by using full body gestures. The underlying motivation was to enhance DJ performance and the overall audience experience. Following a review of the relevant literature, this thesis begins by identifying common DJ techniques. Then gestures were elicited for each common DJ technique using the Gesture Elicitation Study (GES) methodology with the aim of creating a universally understood gesture set. The GES resulted in mainly low consensus, conflicting and inconsistent gestures which prevented an end-user gesture set from being directly produced. Therefore, three further gesture set creation studies were performed to try to create a conflict and inconsistency free gesture set. This project successfully created an end-user gesture set from the results of all four experiments. However, the inconsistencies and conflicts from these experiments suggest that that there is not a universal language that both DJs and audience members understand. Therefore, the strict GES method is deemed inappropriate for producing a DJ-audience communication focused gesture set; the author suggests adapting this methodology to involve subjective ratings to select the most suitable gestures

Imagining & Sensing: Understanding and Extending the Vocalist-Voice Relationship Through Biosignal Feedback

The voice is body and instrument. Third-person interpretation of the voice by listeners, vocal teachers, and digital agents is centred largely around audio feedback. For a vocalist, physical feedback from within the body provides an additional interaction. The vocalist’s understanding of their multi-sensory experiences is through tacit knowledge of the body. This knowledge is difficult to articulate, yet awareness and control of the body are innate. In the ever-increasing emergence of technology which quantifies or interprets physiological processes, we must remain conscious also of embodiment and human perception of these processes. Focusing on the vocalist-voice relationship, this thesis expands knowledge of human interaction and how technology influences our perception of our bodies. To unite these different perspectives in the vocal context, I draw on mixed methods from cog- nitive science, psychology, music information retrieval, and interactive system design. Objective methods such as vocal audio analysis provide a third-person observation. Subjective practices such as micro-phenomenology capture the experiential, first-person perspectives of the vocalists them- selves. Quantitative-qualitative blend provides details not only on novel interaction, but also an understanding of how technology influences existing understanding of the body. I worked with vocalists to understand how they use their voice through abstract representations, use mental imagery to adapt to altered auditory feedback, and teach fundamental practice to others. Vocalists use multi-modal imagery, for instance understanding physical sensations through auditory sensations. The understanding of the voice exists in a pre-linguistic representation which draws on embodied knowledge and lived experience from outside contexts. I developed a novel vocal interaction method which uses measurement of laryngeal muscular activations through surface electromyography. Biofeedback was presented to vocalists through soni- fication. Acting as an indicator of vocal activity for both conscious and unconscious gestures, this feedback allowed vocalists to explore their movement through sound. This formed new perceptions but also questioned existing understanding of the body. The thesis also uncovers ways in which vocalists are in control and controlled by, work with and against their bodies, and feel as a single entity at times and totally separate entities at others. I conclude this thesis by demonstrating a nuanced account of human interaction and perception of the body through vocal practice, as an example of how technological intervention enables exploration and influence over embodied understanding. This further highlights the need for understanding of the human experience in embodied interaction, rather than solely on digital interpretation, when introducing technology into these relationships