Immersive storytelling in mixed reality environments

This is an accepted manuscript of an article published by IEEE in Proceedings of the 2017 23rd International Conference on Virtual Systems and Multimedia (VSMM) on 26/04/2018, available online: https://ieeexplore.ieee.org/document/8346299 The accepted version of the publication may differ from the final published version.How will we adapt to a future that may see humans as an interplanetary species? The proposed project uses themes of outer space, future worlds and space travel to examine ways in which our future identities may be formed from these new environments, the role/s we may have in future societies; and the relationships that we will form with the people we will meet. The utilization of virtual and mixed reality (AR/VR) technologies can be a powerful tool in which to place the audience in different scenarios, to experience it from different viewpoints, and to allow them to anticipate what the future may look, feel like, and indeed be like, by being placed into a set of future space scenarios. This paper presents ideas from an interdisciplinary team of artists, scientists, and technologists of methodological approaches for art-science-technology and the prototypes anticipated through these dialogues.Published versio

Smart glasses for 3D multimodal composition

Extended reality technologies – mixed, augmented, and virtual reality, and future-related technologies – are rapidly expanding in many fields, with underexplored potentials for multimodal composition in digital media environments. This research generates new knowledge about the novel wearable technology – smart glasses – to support elementary students’ multimodal story authoring with 3D virtual objects or holograms. The researchers and teachers implemented learning experiences with upper elementary students from three classrooms to compose and illustrate written narratives before retelling the story with Microsoft HoloLens 2 smart glasses, selecting 3D holograms to illustrate the settings, characters, and events from the 3D Viewer software. The findings analyse how smart glasses supported students’ multimodal composition, and relatedly, the new modal resources available to students wearing smart glasses to compose 3D stories. The findings have significance for educators and researchers to understand and utilise the multimodal affordances of augmented and mixed reality environments for composing and storytelling

Mixed reality for cross-cultural integration. Using positive technology to share experiences and promote communication

The opinion article highlights some innovative resources to deal with the challenges of migrations, relying in the field of positive technologies and, more specifically, in the concept of mixed reality. In the contemporary society, migrations are a common phenomenon that rises cultural and psycho-social issues, as well as political and economic challenges. People move from their place of origin for educational or professional purposes or because they are forced to leave due to political, economic and social conditions, and also natural disasters which produce population flows. Whatever the push and pull factors are, when people move permanently or temporarily they tend to maintain close ties with their place of origin (with people, places, culture, practices etc.), while trying to develop attachment with the place of residence. Immigrants construct their identities in the context of a negotiation between old and new homes’ contexts. However, such a process is not free from issues and relevant consequences on immigrants’ well-being. Some psychosocial issues can be identified regarding identity re-negotiation while moving to a different place, and cultural integration: immigrants could experience feeling of isolation, estrangement and alienation, related to the difficulty to create strong social ties in the new place; the “acculturation stress” associated to adaptation to new culture, language and practices. Positive technologies offer innovative resources to deal with these challenges, by considering the human health and well-being as the main objective for technological advancement. In a broad sense, Positive technology may be used to structure, augment or replace user experience with digital content; also, positive devices may be used to promote positive emotions (hedonic technology), to support the user in the achievement of engaging and self-actualizing experiences (eudaimonic technology), and to enhance connectedness among individuals, groups and societies (social-interpersonal technologies). In such perspective, the mixed reality technology provides resources for intervention in that it is based on the addition of digital elements in the physical environment, instead of its substitution with an immersive experience which, in this case, may act as a palliative care for sadness but does not help to integrate oneself in a new, “real” physical environment and social context. Specifically, mixed reality based Positive technologies can help in maintaining the relation with the home country, and also in fostering the inclusion in and attachment to the receiving society, by providing users with sources of identification that stretch beyond the national and local contexts of their old and new homes. Addressing the social connectedness, the mixed reality can provide the medium to share the meanings that people attach to places, people and cultures, and creating belonging in the receiving society. Indeed, people can better approach the receiving society by understanding the cultural meanings connected with places, history and activities. The concept expressed in the opinion article is still in its infancy. However, it provides an innovative idea for positive technology (at the social-interpersonal level), which may guide the development of future devices and applications for enhancing health and well-being in the growing population looking for a new life in places distant from home

Inchcolm project

Inchcolm Project is part of an interdisciplinary research project which develops new ways of designing for the moving body across media, by combining aesthetics and design methods from contemporary performance practice and video games. As such, it brought a video game (Dear Esther, The Chinese Room, 2012) to life on a Scottish island (Inchcolm island in the Firth of Forth). During the two hour experience on Inchcolm the audience/players wander freely on the island encountering geo-tagged audio, live music, performers and installation spaces evocative of the game world, a playthrough of the game projected in the 12th century Inchcolm abbey, and an orchestral performance of the video game’s soundtrack (composed by Jessica Curry, arranged by Luci Holland and David Jamieson, performed by Mantra Collective)

Bringing tabletop technologies to kindergarten children

Taking computer technology away from the desktop and into a more physical, manipulative space, is known that provide many benefits and is generally considered to result in a system that is easier to learn and more natural to use. This paper describes a design solution that allows kindergarten children to take the benefits of the new pedagogical possibilities that tangible interaction and tabletop technologies offer for manipulative learning. After analysis of children's cognitive and psychomotor skills, we have designed and tuned a prototype game that is suitable for children aged 3 to 4 years old. Our prototype uniquely combines low cost tangible interaction and tabletop technology with tutored learning. The design has been based on the observation of children using the technology, letting them freely play with the application during three play sessions. These observational sessions informed the design decisions for the game whilst also confirming the children's enjoyment of the prototype

Authoring and Living Next-Generation Location-Based Experiences

Authoring location-based experiences involving multiple participants, collaborating or competing in both indoor and outdoor mixed realities, is extremely complex and bound to serious technical challenges. In this work, we present the first results of the MAGELLAN European project and how these greatly simplify this creative process using novel authoring, augmented reality (AR) and indoor geolocalisation techniques

Viewing the Future? Virtual Reality In Journalism

Journalism underwent a flurry of virtual reality content creation, production and distribution starting in the final months of 2015. The New York Times distributed more than 1 million cardboard virtual reality viewers and released an app showing a spherical video short about displaced refugees. The Los Angeles Times landed people next to a crater on Mars. USA TODAY took visitors on a ride-along in the "Back to the Future" car on the Universal Studios lot and on a spin through Old Havana in a bright pink '57 Ford. ABC News went to North Korea for a spherical view of a military parade and to Syria to see artifacts threatened by war. The Emblematic Group, a company that creates virtual reality content, followed a woman navigating a gauntlet of anti- abortion demonstrators at a family planning clinic and allowed people to witness a murder-suicide stemming from domestic violence.In short, the period from October 2015 through February 2016 was one of significant experimentation with virtual reality (VR) storytelling. These efforts are part of an initial foray into determining whether VR is a feasible way to present news. The year 2016 is shaping up as a period of further testing and careful monitoring of potential growth in the use of virtual reality among consumers

Exploring heritage through time and space : Supporting community reflection on the highland clearances

On the two hundredth anniversary of the Kildonan clearances, when people were forcibly removed from their homes, the Timespan Heritage centre has created a program of community centred work aimed at challenging pre conceptions and encouraging reflection on this important historical process. This paper explores the innovative ways in which virtual world technology has facilitated community engagement, enhanced visualisation and encouraged reflection as part of this program. An installation where users navigate through a reconstruction of pre clearance Caen township is controlled through natural gestures and presented on a 300 inch six megapixel screen. This environment allows users to experience the past in new ways. The platform has value as an effective way for an educator, artist or hobbyist to create large scale virtual environments using off the shelf hardware and open source software. The result is an exhibit that also serves as a platform for experimentation into innovative ways of community co-creation and co-curation.Postprin