Supporting mobile mixed-reality experiences

Mobile mixed-reality experiences mix physical and digital spaces, enabling participants to simultaneously inhabit a shared environment online and on the streets. These experiences take the form of games, educational applications and new forms of performance and art, and engender new opportunities for interaction, collaboration and play. As mobile mixed-reality experiences move out of the laboratory and into more public settings they raise new challenges concerning how to support these experiences in the wild. This thesis argues that mobile mixed-reality experiences in which artists retain creative control over the content and operation of each experience, particularly those that are deployed as theatrical performances, require dedicated support for content authoring and reactive orchestration tools and paradigms in order to be successfully and robustly operated in public settings. These requirements are examined in detail, drawing on the experience of supporting four publicly toured mobile mixed-reality experiences; Can You See Me Now?, Uncle Roy All Around You, I Like Frank in Adelaide and Savannah, which have provided a platform to practically develop, refine and evaluate new solutions to answer these challenges in the face of presenting the experiences to many thousands of participants over a four year period. This thesis presents two significant supporting frameworks. The ColourMaps system enables designers to author location-based content by directly colouring over maps; providing a simple, familiar and yet highly flexible approach to matching location-triggers to complex physical spaces. It provides support for multiple and specialised content layers, and the ability to configure and manage other aspects of an experience, including filtering inaccurate position data and underpinning orchestration tools. Second, the Orchestration framework supports the day-to-day operation of public experiences; providing dedicated control-room tools for monitoring that reveal the content landscape and historical events, intervention and improvisation techniques for steering and shaping each participant's experience as it unfolds both physically and virtually, and processes to manage a constant flow of participants

Supporting mobile mixed-reality experiences

Mobile mixed-reality experiences mix physical and digital spaces, enabling participants to simultaneously inhabit a shared environment online and on the streets. These experiences take the form of games, educational applications and new forms of performance and art, and engender new opportunities for interaction, collaboration and play. As mobile mixed-reality experiences move out of the laboratory and into more public settings they raise new challenges concerning how to support these experiences in the wild. This thesis argues that mobile mixed-reality experiences in which artists retain creative control over the content and operation of each experience, particularly those that are deployed as theatrical performances, require dedicated support for content authoring and reactive orchestration tools and paradigms in order to be successfully and robustly operated in public settings. These requirements are examined in detail, drawing on the experience of supporting four publicly toured mobile mixed-reality experiences; Can You See Me Now?, Uncle Roy All Around You, I Like Frank in Adelaide and Savannah, which have provided a platform to practically develop, refine and evaluate new solutions to answer these challenges in the face of presenting the experiences to many thousands of participants over a four year period. This thesis presents two significant supporting frameworks. The ColourMaps system enables designers to author location-based content by directly colouring over maps; providing a simple, familiar and yet highly flexible approach to matching location-triggers to complex physical spaces. It provides support for multiple and specialised content layers, and the ability to configure and manage other aspects of an experience, including filtering inaccurate position data and underpinning orchestration tools. Second, the Orchestration framework supports the day-to-day operation of public experiences; providing dedicated control-room tools for monitoring that reveal the content landscape and historical events, intervention and improvisation techniques for steering and shaping each participant's experience as it unfolds both physically and virtually, and processes to manage a constant flow of participants

Out there and in here: design for blended scientific inquiry learning

One of the benefits of mobile technologies is to combine ‘the digital’ (e.g., data, information, photos) with ‘field’ experiences in novel ways that are contextualized by people’s current located activities. However, often cost, mobility disabilities and time exclude students from engaging in such peripatetic experiences. The Out There and In Here project, is exploring a combination of mobile and tabletop technologies in support for collaborative learning. A system is being developed for synchronous collaboration between geology students in the field and peers at an indoor location. The overarching goal of this research is to develop technologies that support people working together in a suitable manner for their locations. There are two OTIH project research threads. The first deals with disabled learner access issues: these complex issues are being reviewed in subsequent evaluations and publications. This paper will deal with issues of technology supported learning design for remote and co-located science learners. Several stakeholder evaluations and two field trials have reviewed two research questions: 1. What will enhance the learning experience for those in the field and laboratory? 2. How can learning trajectories and appropriate technologies be designed to support equitable co-located and remote learning collaboration? This paper focuses on describing the iterative linked development of technologies and scientific inquiry pedagogy. Two stages within the research project are presented. The 1st stage details several pilot studies over 3 years with 21 student participants in synchronous collaborations with traditional technology and pedagogical models. Findings revealed that this was an engaging and useful experience although issues of equity in collaboration needed further research. The 2nd stage, in this project, has been to evaluate data from over 25 stakeholders (academics, learning and technology designers) to develop pervasive ambient technological solutions supporting orchestration of mixed levels of pedagogy (i.e. abstract synthesis to specific investigation). Middleware between tabletop ‘surface’ technologies and mobile devices are being designed with Microsoft and OOKL (a mobile software company) to support these developments. Initial findings reveal issues around equity, ownership and professional identity

Examining the role of smart TVs and VR HMDs in synchronous at-a-distance media consumption

This article examines synchronous at-a-distance media consumption from two perspectives: How it can be facilitated using existing consumer displays (through TVs combined with smartphones), and imminently available consumer displays (through virtual reality (VR) HMDs combined with RGBD sensing). First, we discuss results from an initial evaluation of a synchronous shared at-a-distance smart TV system, CastAway. Through week-long in-home deployments with five couples, we gain formative insights into the adoption and usage of at-a-distance media consumption and how couples communicated during said consumption. We then examine how the imminent availability and potential adoption of consumer VR HMDs could affect preferences toward how synchronous at-a-distance media consumption is conducted, in a laboratory study of 12 pairs, by enhancing media immersion and supporting embodied telepresence for communication. Finally, we discuss the implications these studies have for the near-future of consumer synchronous at-a-distance media consumption. When combined, these studies begin to explore a design space regarding the varying ways in which at-a-distance media consumption can be supported and experienced (through music, TV content, augmenting existing TV content for immersion, and immersive VR content), what factors might influence usage and adoption and the implications for supporting communication and telepresence during media consumption

Visualising mixed reality simulation for multiple users

Cowling, MA ORCiD: 0000-0003-1444-1563Blended reality seeks to encourage co-presence in the classroom, blending student experience across virtual and physical worlds. In a similar way, Mixed Reality, a continuum between virtual and real environments, is now allowing learners to work in both the physical and the digital world simultaneously, especially when combined with an immersive headset experience. This experience provides innovative new experiences for learning, but faces the challenge that most of these experiences are single user, leaving others outside the new environment. The question therefore becomes, how can a mixed reality simulation be experienced by multiple users, and how can we present that simulation effectively to users to create a true blended reality environment? This paper proposes a study that uses existing screen production research into the user and spectator to produce a mixed reality simulation suitable for multiple users. A research method using Design Based Research is also presented to assess the usability of the approach

Teaching Cultural Heritage using Mobile Augmented Reality

open2noThe relationship between augmented reality, mobile learning, gamification and non-formal education methods provide a great potential. The AR-CIMUVE Augmented Reality for the Walled Cities of the Veneto is an original project in collaboration with Italia Nostra and other associations which deal with transmitting our cultural heritage and which teach primary and middle school children the cultural and historical importance of the Veneto’s and the surrounding territories’ walled cities. In this learning experience students will explore how our environment has developed across the ages using the mobile devices with the technical back-up of the AR App. This will allow them to see maps, examine data, 3D models and will enable them to judge and improve their skills. From a pedagogical and educational point of view the emphasis is on a constructivist social-cultural approach which helps students to become active citizens more aware of their historical identity.openPetrucco, Corrado; Agostini, DanielePetrucco, Corrado; Agostini, Daniel