Innovative Applications of Natural Language Processing and Digital Media in Theatre and Performing Arts

The objective of our research is to investigate new digital techniques and tools, offering the audience innovative, attractive, enhanced and accessible experiences. The project focuses on performing arts, particularly theatre, aiming at designing, implementing, experimenting and evaluating technologies and tools that expand the semiotic code of a performance by offering new opportunities and aesthetic means in stage art and by introducing parallel accessible narrative flows. In our novel paradigm, modern technologies emphasize the stage elements providing a multilevel, intense and immersive theatrical experience. Moreover, lighting, video projections, audio clips and digital characters are incorporated, bringing unique aesthetic features. We also attempt to remove sensory and language barriers faced by some audiences. Accessibility features consist of subtitles, sign language and audio description. The project emphasises on natural language processing technologies, embedded communication and multimodal interaction to monitor automatically the time flow of a performance. Based on this, pre-designed and directed stage elements are being mapped to appropriate parts of the script and activated automatically by using the virtual "world" and appropriate sensors, while accessibility flows are dynamically synchronized with the stage action. The tools above are currently adapted within two experimental theatrical plays for validation purposes. This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.</p

Improvisation as Online Planning

Peer reviewe

Beyond the Electronic Connection: The Technologically Manufactured Cyber-Human and Its Physical Human Counterpart in Performance: A Theory Related to Convergence Identities

This thesis is an investigation of the complex processes and relationships between the physical human performer and the technologically manufactured cyber-human counterpart. I acted as both researcher and the physical human performer, deeply engaged in the moment-to-moment creation of events unfolding within a shared virtual reality environment. As the primary instigator and activator of the cyber-human partner, I maintained a balance between the live and technological performance elements, prioritizing the production of content and meaning. By way of using practice as research, this thesis argues that in considering interactions between cyber-human and human performers, it is crucial to move beyond discussions of technology when considering interactions between cyber-humans and human performers to an analysis of emotional content, the powers of poetic imagery, the trust that is developed through sensory perception and the evocation of complex relationships. A theoretical model is constructed to describe the relationship between a cyber-human and a human performer in the five works created specifically for this thesis, which is not substantially different from that between human performers. Technological exploration allows for the observation and analysis of various relationships, furthering an expanded understanding of ‘movement as content’ beyond the electronic connection. Each of the works created for this research used new and innovative technologies, including virtual reality, multiple interactive systems, six generations of wearable computers, motion capture technology, high-end digital lighting projectors, various projection screens, smart electronically charged fabrics, multiple sensory sensitive devices and intelligent sensory charged alternative performance spaces. They were most often collaboratively created in order to augment all aspects of the performance and create the sense of community found in digital live dance performances/events. These works are identified as one continuous line of energy and discovery, each representing a slight variation on the premise that a working, caring, visceral and poetic content occurs beyond the technological tools. Consequently, a shift in the physical human’s psyche overwhelms the act of performance. Scholarship and reflection on the works have been integral to my creative process throughout. The goals of this thesis, the works created and the resulting methodologies are to investigate performance to heighten the multiple ways we experience and interact with the world. This maximizes connection and results in a highly interactive, improvisational, dynamic, non-linear, immediate, accessible, agential, reciprocal, emotional, visceral and transformative experience without boundaries between the virtual and physical for physical humans, cyborgs and cyber-humans alike.College of Fine Arts at the University of Texas at Austin, Department of Theatre & Dance at the University of Texas at Austi

A Spectrum of Audience Interactivity for entertainment domains

The concept of audience interactivity has been rediscovered across many domains of storytelling and entertainment—e.g. digital games, in-person role-playing, film, theater performance, music, and theme parks—that enrich the form with new idioms, language, and practices. In this paper, we introduce a Spectrum of Audience Interactivity that establishes a common vocabulary for the design space across entertainment domains. Our spectrum expands on an early vocabulary conceptualized through co-design sessions for interactive musical performances. We conduct a cross-disciplinary literature review to evaluate and iterate upon this vocabulary, using our findings to develop our validated spectrum

Performance interfaces and destabilisation

Interaction with technology is occurring increasingly in public and semi-public settings and as a result the roles of spectator and performer are frequently being challenged by the deployment of computing systems. In this paper we discuss how the spectator, performer and interface feature in what we class as performance, how we might analyse their interrelation-ships and how traditional roles have become destabilised historically and technologically. In studying these relationships, we examine technological and non-technological examples from art, performance and exhibition design

Heritage Role Playing - History as an Interactive Digital Game

Creating virtual heritage environments that intend to be both engaging and educational is a challenging process. Digital archaeological reconstruction has been concerned with exact replication of facts rather than with understanding, for the latter raises the annoying dilemma of how to present scientific uncertainty. A computer model almost invariably implies certitude, and archaeologists are still not sure how to convey the murky battle of historical interpretation. Yet games are quite happy to allow users to "muddy" historical settings. And while the bulk of computer game design may be justly considered a-cultural or even anti-cultural, the underlying techniques of engaging interactively with the audience offer new ways of increasing the popularity and immersive learning of virtual environments. However there are some serious issues in heritage projects adopting a game-style approach. Would using interactive game techniques and technologies create a more engaging user experience? If we can animate the past in this way, will the entertainment factor help or impede learning, and how will we know how effective the interactivity is? And would our results help bridge the gap between the industry (be it virtual exhibitions or interactive game design) and academia

16th Sound and Music Computing Conference SMC 2019 (28&#8211;31 May 2019, Malaga, Spain)

The 16th Sound and Music Computing Conference (SMC 2019) took place in Malaga, Spain, 28-31 May 2019 and it was organized by the Application of Information and Communication Technologies Research group (ATIC) of the University of Malaga (UMA). The SMC 2019 associated Summer School took place 25-28 May 2019. The First International Day of Women in Inclusive Engineering, Sound and Music Computing Research (WiSMC 2019) took place on 28 May 2019. The SMC 2019 TOPICS OF INTEREST included a wide selection of topics related to acoustics, psychoacoustics, music, technology for music, audio analysis, musicology, sonification, music games, machine learning, serious games, immersive audio, sound synthesis, etc