Interactive Sound Texture Synthesis through Semi-Automatic User Annotations

We present a way to make environmental recordings controllable again by the use of continuous annotations of the high-level semantic parameter one wishes to control, e.g. wind strength or crowd excitation level. A partial annotation can be propagated to cover the entire recording via cross-modal analysis between gesture and sound by canonical time warping (CTW). The annotations serve as a descriptor for lookup in corpus-based concatenative synthesis in order to invert the sound/annotation relationship. The workflow has been evaluated by a preliminary subject test and results on canonical correlation analysis (CCA) show high consistency between annotations and a small set of audio descriptors being well correlated with them. An experiment of the propagation of annotations shows the superior performance of CTW over CCA with as little as 20 s of annotated material

TRANSEUNTIS MUNDI, A NOMADIC ARTISTIC PRACTICE

In this practice-led Ph.D. research, I investigate how an artistic practice can respond to the migration phenomena performed by human beings across the planet over millennia ¬– what I refer to as the millennial global human journey. Based on the idea of mobility, I chose to frame this research in the articulation of concepts deriving from the prefix trans: transculture, transhumance and transmediality. This research contributes to studies in art composition by developing the processes and concept of transmedial composition, mainly contributing to the field of New Media Art. This investigation resulted in the work Transeuntis Mundi (TM) Project – a nomadic artistic practice that encompasses: the TM Derive and manual, the TM Archive, the TM VR work Derive 01 and two forms for its notation. Transeuntis mundi (TM), from the Latin language, means the ‘passersby of the world’ and metaphorically personify in this work the millennial migrants and their global journeys. Based on proposals from the Realism art movement and the walking-based methodologies of Walkscapes and Dérive, the TM Derive was created as a nomadic methodology of composition in response to the ideas of migration and ancestry. It is framed by the minimal stories ¬– the form of narrative of this work, captured from field recordings with 3D technology of everyday life worldwide. This material formed the TM Archive, presented in the TM VR work. The TM VR work Transeuntis Mundi Derive 01 is an immersive and interactive performative experience for virtual reality, that artistically brings together stories, sounds, images, people, and places worldwide, ¬as a metaphor of the millennial global human migration. This work happens as a VR application using 3D technology with 360º image and ambisonic sound, in order to promote an engaged experience through the immersion and interactivity of the participant. This thesis presents and contextualizes these creations: the scope, references, concepts, origin, collaborations, methodology, technologies, and results of this work. It is informed and accompanied by reflexive and critical writing, including an articulation with references of works across different artistic media and fields.UNIRIO Federal University of the State of Rio de Janeir

Learning and Teaching Languages in Technology-Mediated Environments: Why Modes and Meaning Making Matter

The developing argument presented in this thesis is based on seven articles, eight book chapters and one set of conference proceedings, some single-authored and some co-authored, on language learning and teaching in technology-mediated environments, published between 2004 and 2018. The publications chart my evolution as a researcher and practitioner at the Open University UK. There are several threads which weave themselves through my scholarly journey and which are reflected in the selected work: Thread 1: multimodal competence and language learning and teaching with technology Thread 2: task-based approaches to language learning and teaching with technology Thread 3: teacher (and learner) preparation for language learning and teaching with technology Thread 4: learner (and teacher) autonomy in language learning and teaching with technology. The narrative cloth in the presentation of the publications draws these threads together and illustrates how they interconnect across my work. They are linked by my concern for online language learners’ awareness of the opportunities and demands of the learning environment and the impact that such awareness, or lack thereof, has on the learning process. The empirical studies presented and discussed in my work use mostly qualitative research instruments. They take forward the knowledge in the field by offering original insights into the interrelationship between language learner awareness and control over the learning context understood as awareness of online modes and their potential for meaning-making, communication, interaction and collaboration. This interrelationship is not only relevant for language learning in virtual environments per se, it also has repercussions on online learners’ development of intercultural communicative competence, their digital participatory competence and social presence online, and on their autonomy. Underpinning my work is a view shared by a growing number of researchers and practitioners in online learning and teaching of languages and cultures that a radical pedagogical shift is required: it is not sufficient to see the new technology-infused learning spaces as replicates of conventional face-to-face settings. Such a shift has to be informed by new learning theories which capture the dynamic nature of the enterprise in the wake of unabating technological advancements (e.g. Guichon, 2009; Hampel & Stickler, 2005; Hubbard & Levy, 2006; Hubbard, 2009; Kern, 2015; Sun, 2011). Moreover, I argue, it will need to include the systematic raising of learner awareness of learning context. The presentation of the selected articles, book chapters and conference proceedings in Chapter 4 is divided into five parts in line with the thematic foci of the publications: (1) contextual knowledge, (2) multimodal competence (3) multiliteracies (4) digital literacies (5) participatory literacy and social presence. The publications in Chapter 4, section 4.1 – Hauck (2004), Hampel and Hauck (2004), Hauck (2005), and Hauck and Hurd (2005) – focus on the concept of contextual knowledge and are informed by two studies: one carried out with students when the former Department of Languages (DoL) at the Open University offered learners a choice between face-to-face and online tutorials via an audiographic conferencing application (Lyceum); the other one carried out with OU tutors, most of whom were at the time unfamiliar with using Internet-based conferencing for language learning and teaching purposes. The work presented in section 4.2 – Hampel and Hauck (2006), Hauck (2007), Hauck and Youngs (2008) and Hauck and Hampel (2008) – concentrates on multimodal competence as well as the interface between multimodal communicative competence and intercultural communicative competence online. While Hampel and Hauck (2006) is a theoretical contribution, the other three publications are based on a telecollaborative exchange linking participants from three different parts of the world (the Tridem project). The former helped frame the empirical study at the center of the two articles and the chapter that followed. The publications in section 4.3 – Hauck (2010a) and Hauck (2010b), and Fuchs, Hauck and Müller-Hartmann (2012) – explore multiliteracies with multimodal competence understood as a core element of multiliteracies. They draw on data from a four-way telecollaborative exchange between teacher trainees and language learners in order to illustrate why telecollaboration provides the ideal set-up for fostering such competence development and therefore also online learner and teacher autonomy. This leads to an examination of digital literacies in Kurek and Hauck (2014) and Hauck and Kurek (2017) in section 4.4. Both chapters are, again, theoretical contributions to the field of technology-mediated language learning and teaching. We conceptualise digital proficiency as mastery of modes and meaning-making – in other words multimodal competence – and as a precondition for autonomy. In Kurek and Hauck (2014) we present a task framework for instructed learner reflection to this effect, ideally in telecollaborative settings. Finally, in section 4.5 – Hauck and Warnecke (2012), Hauck, Galley and Warnecke (2016), also a theoretical contribution, and Hauck and Satar (2018) – my co-authors and I explore a subset of digital literacies, namely participatory literacy as reflected in multimodal competence, and its relevance for social presence in online language learning and teaching contexts. An example of how these themes interlink with the aforementioned threads is the task-based approach to multimodal competence development (Threads 1 and 2) in telecollaborative settings which is advocated in all three publications in section 4.3. Together, the publications make a substantial contribution to the field of language learning and teaching in technology-mediated environments, through the centrality they grant to the learning context, and increasingly also to multimodality (Kress & van Leeuwen, 2001; Kress, 2009) as an overarching approach to conceptualising context-related challenges for both students and teachers

A telepresence environment for concurrent life-cycle design and construction

Construction projects normally involve transient 'virtual organisations', where a multidisciplinary project team works together on the design and construction of a facility. Many of these participants often work independently while taking decisions that inevitably affect others. The research described in the thesis involved examining the adoption of concurrent engineering (CE) principles by the construction industry as a way to reduce the problems posed by the fragmentation of the industry, and to enhance its competitiveness. An important aspect of concurrent engineering in construction is the need for effective communication of design information between all members of the project team and across all stages of the constructed facility's life-cycle. The thesis describes the development of a communications infrastructure for Concurrent Life-Cycle Design and Construction. [Continues.

Sonic Interactions in Virtual Environments

This open access book tackles the design of 3D spatial interactions in an audio-centered and audio-first perspective, providing the fundamental notions related to the creation and evaluation of immersive sonic experiences. The key elements that enhance the sensation of place in a virtual environment (VE) are: Immersive audio: the computational aspects of the acoustical-space properties of Virutal Reality (VR) technologies Sonic interaction: the human-computer interplay through auditory feedback in VE VR systems: naturally support multimodal integration, impacting different application domains Sonic Interactions in Virtual Environments will feature state-of-the-art research on real-time auralization, sonic interaction design in VR, quality of the experience in multimodal scenarios, and applications. Contributors and editors include interdisciplinary experts from the fields of computer science, engineering, acoustics, psychology, design, humanities, and beyond. Their mission is to shape an emerging new field of study at the intersection of sonic interaction design and immersive media, embracing an archipelago of existing research spread in different audio communities and to increase among the VR communities, researchers, and practitioners, the awareness of the importance of sonic elements when designing immersive environments

Sonic Interactions in Virtual Environments

This open access book tackles the design of 3D spatial interactions in an audio-centered and audio-first perspective, providing the fundamental notions related to the creation and evaluation of immersive sonic experiences. The key elements that enhance the sensation of place in a virtual environment (VE) are: Immersive audio: the computational aspects of the acoustical-space properties of Virutal Reality (VR) technologies Sonic interaction: the human-computer interplay through auditory feedback in VE VR systems: naturally support multimodal integration, impacting different application domains Sonic Interactions in Virtual Environments will feature state-of-the-art research on real-time auralization, sonic interaction design in VR, quality of the experience in multimodal scenarios, and applications. Contributors and editors include interdisciplinary experts from the fields of computer science, engineering, acoustics, psychology, design, humanities, and beyond. Their mission is to shape an emerging new field of study at the intersection of sonic interaction design and immersive media, embracing an archipelago of existing research spread in different audio communities and to increase among the VR communities, researchers, and practitioners, the awareness of the importance of sonic elements when designing immersive environments

Investigating conditions for higher order thinking in telematics environments

Telematics, or audiographic conferencing, enables synchronous communication via telecommunications. A telephone link and computer communications allow students in distributed classrooms to participate in an extended, or virtual classroom. Telematics is widely used as an instructional medium for the delivery of curricula to students in rural and remote parts of Western Australia. Previous studies of learning meditated by this technology have shown limitations on the forms of interactivity, tasks and learner engagement. Students typically have been found to assume a passive role, often listening to a distant and invisible teacher, but not engaging in cognitively demanding tasks. The tendency for teachers to display didactic forms of teaching has been noted in a number of studies

Sonic interactions in virtual environments

This book tackles the design of 3D spatial interactions in an audio-centered and audio-first perspective, providing the fundamental notions related to the creation and evaluation of immersive sonic experiences. The key elements that enhance the sensation of place in a virtual environment (VE) are: Immersive audio: the computational aspects of the acoustical-space properties of Virutal Reality (VR) technologies Sonic interaction: the human-computer interplay through auditory feedback in VE VR systems: naturally support multimodal integration, impacting different application domains Sonic Interactions in Virtual Environments will feature state-of-the-art research on real-time auralization, sonic interaction design in VR, quality of the experience in multimodal scenarios, and applications. Contributors and editors include interdisciplinary experts from the fields of computer science, engineering, acoustics, psychology, design, humanities, and beyond. Their mission is to shape an emerging new field of study at the intersection of sonic interaction design and immersive media, embracing an archipelago of existing research spread in different audio communities and to increase among the VR communities, researchers, and practitioners, the awareness of the importance of sonic elements when designing immersive environments