Spring School on Language, Music, and Cognition: Organizing Events in Time

The interdisciplinary spring school “Language, music, and cognition: Organizing events in time” was held from February 26 to March 2, 2018 at the Institute of Musicology of the University of Cologne. Language, speech, and music as events in time were explored from different perspectives including evolutionary biology, social cognition, developmental psychology, cognitive neuroscience of speech, language, and communication, as well as computational and biological approaches to language and music. There were 10 lectures, 4 workshops, and 1 student poster session. Overall, the spring school investigated language and music as neurocognitive systems and focused on a mechanistic approach exploring the neural substrates underlying musical, linguistic, social, and emotional processes and behaviors. In particular, researchers approached questions concerning cognitive processes, computational procedures, and neural mechanisms underlying the temporal organization of language and music, mainly from two perspectives: one was concerned with syntax or structural representations of language and music as neurocognitive systems (i.e., an intrapersonal perspective), while the other emphasized social interaction and emotions in their communicative function (i.e., an interpersonal perspective). The spring school not only acted as a platform for knowledge transfer and exchange but also generated a number of important research questions as challenges for future investigations

Listening and remembering: networked off-line improvisation for four commuters

This paper analyses the experience of the networked off-line improvisation 'Listening and Remembering', a performance for four commuters using voices and sounds from the Mexico City and Paris metros. It addresses the question: how can an act of collective remembering, inspired by listening to metro soundscapes, lead to the creation of networked voice- and sound-based narratives about the urban commuting experience? The networked experience is seen here from the structural perspective (telematic setting), the sonic underground context, the ethnographic process that led to the performance, the narratives that are created in the electro-acoustic setting, the shared acoustic environments that those creations suggest, and the technical features and participants' responses that prevent or facilitate interaction. Emphasis is placed on the participants' status as non-performers, and on their familiarity with the sonic environment, as a context that allows the participation of non-musicians in the making of music through telematically shared interfaces, using soundscape and real-time voice. Participants re-enact their routine experience through a dialogical relation- ship with the sounds, the other participants, themselves, and the experience of sharing: a collective memory

Evaluation of live human-computer music-making: Quantitative and qualitative approaches

NOTICE: this is the author’s version of a work that was accepted for publication in International Journal of Human-Computer Studies. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in International Journal of Human-Computer Studies, [VOL 67,ISS 11(2009)] DOI: 10.1016/j.ijhcs.2009.05.00

Puntos de Fuga (Vanishing Points): Performance Writing for the Digital Theater

Examining the contemporary challenges of "writing" or composing for the digital stage and new forms of real-time programmable or augment­ed multimedia theater, the authors delve into some theoretical concerns that are characteristic of such compositional practices—discussing no­tions of interactivity, wearable space, participatory design, postdramatic textuality and choreography, bilinguality and translation, and political content—before using their new play, Puntos de Fuga (Vanishing Points), as a case study. The essay contextualizes the production aesthetic used for Puntos de Fuga, referring amongst others to the Wooster Group's "dancing with technology," and proceeds to delineate the performer techniques and rehearsal methods used in composing an integrated dig­ital performance and multimediated mise en scene closely and passion­ately dedicated to the poetic testimony of harsh political/economic real­ities at the US-Mexican border

Paralinguistic vocal control of interactive media: how untapped elements of voice might enhance the role of non-speech voice input in the user's experience of multimedia.

Much interactive media development, especially commercial development, implies the dominance of the visual modality, with sound as a limited supporting channel. The development of multimedia technologies such as augmented reality and virtual reality has further revealed a distinct partiality to visual media. Sound, however, and particularly voice, have many aspects which have yet to be adequately investigated. Exploration of these aspects may show that sound can, in some respects, be superior to graphics in creating immersive and expressive interactive experiences. With this in mind, this thesis investigates the use of non-speech voice characteristics as a complementary input mechanism in controlling multimedia applications. It presents a number of projects that employ the paralinguistic elements of voice as input to interactive media including both screen-based and physical systems. These projects are used as a means of exploring the factors that seem likely to affect users’ preferences and interaction patterns during non-speech voice control. This exploration forms the basis for an examination of potential roles for paralinguistic voice input. The research includes the conceptual and practical development of the projects and a set of evaluative studies. The work submitted for Ph.D. comprises practical projects (50 percent) and a written dissertation (50 percent). The thesis aims to advance understanding of how voice can be used both on its own and in combination with other input mechanisms in controlling multimedia applications. It offers a step forward in the attempts to integrate the paralinguistic components of voice as a complementary input mode to speech input applications in order to create a synergistic combination that might let the strengths of each mode overcome the weaknesses of the other