A multi-modal dance corpus for research into real-time interaction between humans in online virtual environments

We present a new, freely available, multimodal corpus for research into, amongst other areas, real-time realistic interaction between humans in online virtual environments. The specific corpus scenario focuses on an online dance class application scenario where students, with avatars driven by whatever 3D capture technology are locally available to them, can learn choerographies with teacher guidance in an online virtual ballet studio. As the data corpus is focused on this scenario, it consists of student/teacher dance choreographies concurrently captured at two different sites using a variety of media modalities, including synchronised audio rigs, multiple cameras, wearable inertial measurement devices and depth sensors. In the corpus, each of the several dancers perform a number of fixed choreographies, which are both graded according to a number of specific evaluation criteria. In addition, ground-truth dance choreography annotations are provided. Furthermore, for unsynchronised sensor modalities, the corpus also includes distinctive events for data stream synchronisation. Although the data corpus is tailored specifically for an online dance class application scenario, the data is free to download and used for any research and development purposes

"Knowing is Seeing:" The Digital Audio Workstation and the Visualization of Sound

The computers visual representation of sound has revolutionized the creation of music through the interface of the Digital Audio Workstation software (DAW). With the rise of DAW-based composition in popular music styles, many artists sole experience of musical creation is through the computer screen. I assert that the particular sonic visualizations of the DAW propagate certain assumptions about music, influencing aesthetics and adding new visually-based parameters to the creative process. I believe many of these new parameters are greatly indebted to the visual structures, interactional dictates and standardizations (such as the office metaphor depicted by operating systems such as Apples OS and Microsofts Windows) of the Graphical User Interface (GUI). Whether manipulating text, video or audio, a users interaction with the GUI is usually structured in the same mannerclicking on windows, icons and menus with a mouse-driven cursor. Focussing on the dialogs from the Reddit communities of Making hip-hop and EDM production, DAW user manuals, as well as interface design guidebooks, this dissertation will address the ways these visualizations and methods of working affect the workflow, composition style and musical conceptions of DAW-based producers

Compositional Mixing: An exploration into sonic ownership in contemporary music production.

This thesis examines a presumption wherein mixing within contemporary practices exists as a process with compositional implications, a term coined as compositional mixing. This presupposition is examined through two lines of inquiry: 1) Are mixing and composition existing interdependently in contemporary music production? and 2) What implications do mixing ideologies have on contemporary music production? To answer these questions, an extensive literature review, eight interviews, and five detailed compositional mixing examples were conducted. The accumulation of the results concluded that: 1) There is evidence that supports interdependency between composition and mixing in contemporary music production within specific parameters. 2) Mixing-related practices can be observed as having a supplementary role on a creationary level within contemporary music production

Implementation and development of interfaces for music performance through analysis of improvised dance movements

Electronic music, even when designed to be interactive, can lack performance interest and is frequently musically unsophisticated. This is unfortunate because there are many aspects of electronic music that can be interesting, elegant, demonstrative and musically informative. The use of dancers to interact with prototypical interfaces comprising clusters of sensors generating music algorithmically provides a method of investigating human actions in this environment. This is achieved through collaborative work involving software and hardware designers, composers, sculptors and choreographers who examine aesthetically and practically the interstices of these disciplines. The proposed paper investigates these interstices

Composition portfolio : producing techno grooves

PhD Thesis Additional music to be consulted in Robinson Library only.PhD submission consisting of a portfolio of recordings presented both in their published form on five twelve-inch vinyl records and on two compact discs. The portfolio is accompanied by a commentary intended to facilitate access to the aesthetic statement presented in the portfolio. Following Charles Keil’s distinction between ‘embodied meaning’ and ‘engendered feeling’ this project investigates approaches to the creation of Techno music in terms of the significance of specific sounds, techniques and technologies in generating subsyntactic value. The concept of the groove and the importance of microtiming as they operate in my practice are discussed in the commentary and demonstrated through the production of a series of Techno records including both original compositions and remixes

Large group musical interaction using disposable wireless motion sensors

Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2003.Includes bibliographical references (p. 113-114).One of the difficulties in interactive music and entertainment is creating environments that reflect and react to the collective activity of groups with tens, hundreds, or even thousands of participants. Generating content on this scale involves many challenges. For example, how is the individual granted low latency control and a sense of causality, while still allowing for information retrieval from all participants so that the environment responds to the behavior of the entire group? These issues are particularly pertinent in the area of interactive dance. To address these issues, a low-cost, wireless motion sensor has been developed. The sensor is inexpensive enough to be considered disposable, allowing it to be given away to participants at large dance events, enabling the dancers to participate concurrently in a realtime, interactive musical performance. The sensors are either worn or held by participants and transmit a short RF pulse when accelerated past a certain threshold. The RF pulses are received by a base station and analyzed to detect rhythmic features and estimate the general activity level of the group. These data are then used to generate music that can either lead or follow the participants' actions, thereby tightening the feedback loop between music and dancer. Multiple tests of the system have been conducted, with groups ranging from fifteen to 200 participants. Results of these tests show the viability of the sensors as a large group interaction tool. Participants found the interface intuitive to use, effectively controlling such aspects of the music as style, tempo, voicing, and filter parameters. These tests also demonstrate the system's ability to detect both the activity level and dominant tempo of the participants' motions, and give considerable insight into methods of mapping these data to musical parameters that give participants direct feedback as to their current state. Furthermore, it is shown that participants, if given this direct feedback, will synchronize their actions and increase in activity level, creating a mutually coherent and pleasing outcome.by Mark Christopher Feldmeier.S.M

Electrifying Opera, Amplifying Agency: Designing a performer-controlled interactive audio system for opera singers

This artistic research project examines the artistic, technical, and pedagogical challenges of developing a performer-controlled interactive technology for real-time vocal processing of the operatic voice. As a classically trained singer-composer, I have explored ways to merge the compositional aspects of transforming electronic sound with the performative aspects of embodied singing. I set out to design, develop, and test a prototype for an interactive vocal processing system using sampling and audio processing methods. The aim was to foreground and accommodate an unamplified operatic voice interacting with the room's acoustics and the extended disembodied voices of the same performer. The iterative prototyping explored the performer's relationship to the acoustic space, the relationship between the embodied acoustic voice and disembodied processed voice(s), and the relationship to memory and time. One of the core challenges was to design a system that would accommodate mobility and allow interaction based on auditory and haptic cues rather than visual. In other words, a system allowing the singer to control their sonic output without standing behind a laptop. I wished to highlight and amplify the performer's agency with a system that would enable nuanced and variable vocal processing, be robust, teachable, and suitable for use in various settings: solo performances, various types and sizes of ensembles, and opera. This entailed mediating different needs, training, and working methods of both electronic music and opera practitioners. One key finding was that even simple audio processing could achieve complex musical results. The audio processes used were primarily combinations of feedback and delay lines. However, performers could get complex musical results quickly through continuous gestural control and the ability to route signals to four channels. This complexity sometimes led to surprising results, eliciting improvisatory responses also from singers without musical improvisation experience. The project has resulted in numerous vocal solo, chamber, and operatic performances in Norway, the Netherlands, Belgium, and the United States. The research contributes to developing emerging technologies for live electronic vocal processing in opera, developing the improvisational performance skills needed to engage with those technologies, and exploring alternatives for sound diffusion conducive to working with unamplified operatic voices. Links: Exposition and documentation of PhD research in Research Catalogue: Electrifying Opera, Amplifying Agency. Artistic results. Reflection and Public Presentations (PhD) (2023): https://www.researchcatalogue.net/profile/show-exposition?exposition=2222429 Home/Reflections: https://www.researchcatalogue.net/view/2222429/2222460 Mapping & Prototyping: https://www.researchcatalogue.net/view/2222429/2247120 Space & Speakers: https://www.researchcatalogue.net/view/2222429/2222430 Presentations: https://www.researchcatalogue.net/view/2222429/2247155 Artistic Results: https://www.researchcatalogue.net/view/2222429/222248

Join the Holy Spirit on Zoom: African Pentecostal churches and their liturgical practices during COVID-19

The COVID-19 global pandemic impacted all social relations, including the way religious communities engage in worship services. Due to strict social distancing protocols, the only viable solution for many congregations was online worship. This article investigates how platforms in cyberspace, such as Zoom, can provide a substitute for the core religious practices found in physical worship services, particularly for African Pentecostal believers who rely heavily on the aesthetic and sensory experience of their religious environment. Drawing on the theoretical concept of affordance, it is argued that digital affordances such as the chat box and emojis are used by believers to communicate affective moments arising from the sensory experience of worship. Members of the congregation become ‘digital spiritual hype people’ who render support to leaders in order to create and regenerate an affective environment where the presence of the Holy Spirit can be felt. The Holy Spirit, a fundamental pillar for Evangelical Christians, is understood as an embedded presence within the digital infrastructure. The internet connection, the phone and computers and screens are all re-appropriated as spiritual tools through which miraculous healing can be dispensed to believers in need. This research stands at a critical juncture between what might be termed the ‘pre-COVID era’ and the ‘post-COVID era’. As vaccination plans continue to roll out and social distancing measures are slowly being lifted, a ‘post-COVID era’ for African Pentecostals means negotiating the boundaries between online and offline spaces to fulfil core religious practices

Enhancing fan experience during live sports broadcasts through second screen applications

When sports fans attend live sports events, they usually engage in social experiences with friends, family members and other fans at the venue sharing the same affiliation. However, fans watching the same event through a live television broadcast end up not feeling so emotionally connected with the athletes and other fans as they would if they were watching it live, together with thousands of other fans. With this in mind, we seek to create mobile applications that deliver engaging social experiences involving remote fans watching live broadcasted sports events. Taking into account the growing use of mobile devices when watching TV broadcasts, these mobile applications explore the second screen concept, which allows users to interact with content that complements the TV broadcast. Within this context, we present a set of second screen application prototypes developed to test our concepts, the corresponding user studies and results, as well as suggestions on how to apply the prototypes’ concepts not only in different sports, but also during TV shows and electronic sports. Finally, we also present the challenges we faced and the guidelines we followed during the development and evaluation phases, which may give a considerable contribution to the development of future second screen applications for live broadcasted events