Immersive learning: Enhancing student engagement using 360° photography and unity simulations in undergraduate medical science courses

Student engagement is a crucial factor that can influence both the student learning experience and student success. However, in response to the recent COVID-19 pandemic, learning for tertiary students had been affected as many universities introduced fully online learning. This shift to online learning has had an adverse effect on engagement for many students. Immersive online learning modules, including interactive simulations, have potential to enhance motivation and engagement. This study aimed to compare existing online standard module lessons (2D virtual laboratory spaces) with platforms and conditions that apply immersive virtual learning environments (360o photography and desktop Unity 3D immersive simulations) and evaluate the effects on the student learning experience and performance. Novel virtual learning environments were created to pilot within a third-year undergraduate pathology course, at the University of New South Wales. The study was conducted using a multimodal approach with two different pathology undergraduate cohorts from 2022 to 2023. Each cohort was randomly divided into two groups to trial a different learning environment. In 2022, we compared an existing online standard module (developed using a HTML platform, H5P) with a 360o laboratory space. In 2023, we compared the existing standard online module with an immersive 3D Unity laboratory simulation. Student engagement and performance was assessed across all learning environments using pre- and post-simulation knowledge/transfer tests, and Qualtrics feedback surveys. Qualitative and quantitative data obtained were used to compare factors such as student motivation, engagement, and confidence within the different learning environments. In feedback surveys, students reported being engaged and immersed in both the 360o environments and in the 3D Unity simulations. However, students also reported navigation issues within the virtual learning environments in addition to reporting cognitive overload. Quantitative data revealed an increase in performance on knowledge/transfer tests regardless of the learning environment type, but the level of improvement between each group was not significantly different. Further data revealed an overall improvement in understanding of content for all learning environments, but there was a greater increase with the standard module groups. This could be attributed to possible cognitive overload experienced within the new virtual learning environments. As reported in this study, despite the engaging/immersive properties of the 360o/Unity environments, newly developed learning simulations may overload or distract the learner. Therefore, further work is required on immersive learning environment factors that promote student engagement and motivation. These promoting factors could also be incorporated into face-to-face learning so that key elements for student engagement are aligned in all learning environments

Laying the groundwork for socialisation and knowledge construction within 3D virtual worlds

The paper reports the theoretical underpinnings for the pedagogical role and rationale for adopting 3D virtual worlds for socialisation and knowledge creation in distance education. Socialisation or 'knowing one another' in remote distributed environments can be achieved through synchronous technologies such as instant messaging, audio and video-conferencing. However, a 3D virtual world can provide an immersive experience where there is a visual presence and virtual proximity of the group members in terms of their 3D selves (avatars). We discuss the affordances of a 3D virtual world and its role in providing a platform for pedagogical design that engenders socialisation, synchronous communication and collaboration. We propose the use of a knowledge construction model as a framework for guiding the design of collaborative activities in a 3D virtual world for blended learning environments. We believe that this framework will also be useful for integrating 2D environments such as blogs, wikis and forums with a 3D learning environment. We consider the implications of this in the context of blended learning in distance education. This paper would be of interest to course designers, researchers, teachers, staff developers and policy-makers who are involved in integrating 3D virtual worlds within the curriculum of their programmes and institutions

3D virtual geology field trips: opportunities and limitations

As a part of The OpenScience Laboratory (http://www.open.ac.uk/researchprojects/open-science/), we have developed a virtual Geology field trip in a three-dimensional (3D) environment (https://learn5.open.ac.uk/course/format/sciencelab/section.php?name=skiddaw_1), using the Unity 3D software (https://unity3d.com/). The learning activities within this 3D application are designed to mirror the experience of a field trip in the Skiddaw mountains of UK’s Lake District. The primary objective of developing an authentic 3D avatar-based interactive environment has been to provide an immersive experience to the users through sense of space. The application is based around a 10km x 10km low to medium detail model of the terrain around Skiddaw with overlaid photogrammetry-derived mesh and textual imagery, and augmented with in-built Unity terrain and flora. The virtual embodiment in the form of avatars and the multi-user environment give a sense of co-presence and provides opportunities for collaborative learning. In this presentation, we will reflect on the design and development process of the 3D App: how the pedagogical aspects of Geology and fieldwork were integrated with the affordances of 3D environments and the capabilities of Unity 3D, the development platform. We will discuss the interaction design of the 3D App including its navigation and wayfinding aspects that provide a sense of presence, sense of place, and an immersive learning experience for the students. We will analyse the pedagogical design of the 3D App to highlight the spatial and contextual learning opportunities for the students. With this Virtual Skiddaw App as an example, we will reflect on the following: a) opportunities offered by 3D virtual field trips for science education in both schools and higher education; b) comparisons (advantages and obstacles) of the 3D experience with 2D field trips (for example, via the websites or DVDs) – for example, the real-time multi-user interactions offered by a 3D environment and their impact on learning and student engagement; and c) limitations of 3D virtual field trips with respect to real world field trips

Identifying immersive environments’ most relevant research topics: an instrument to query researchers and practitioners

This paper provides an instrument for ascertaining researchers’ perspectives on the relative relevance of technological challenges facing immersive environments in view of their adoption in learning contexts, along three dimensions: access, content production, and deployment. It described its theoretical grounding and expert-review process, from a set of previously-identified challenges and expert feedback cycles. The paper details the motivation, setup, and methods employed, as well as the issues detected in the cycles and how they were addressed while developing the instrument. As a research instrument, it aims to be employed across diverse communities of research and practice, helping direct research efforts and hence contribute to wider use of immersive environments in learning, and possibly contribute towards the development of news and more adequate systems.The work presented herein has been partially funded under the European H2020 program H2020-ICT-2015, BEACONING project, grant agreement nr. 687676.info:eu-repo/semantics/publishedVersio

An Immersive Telepresence System using RGB-D Sensors and Head Mounted Display

We present a tele-immersive system that enables people to interact with each other in a virtual world using body gestures in addition to verbal communication. Beyond the obvious applications, including general online conversations and gaming, we hypothesize that our proposed system would be particularly beneficial to education by offering rich visual contents and interactivity. One distinct feature is the integration of egocentric pose recognition that allows participants to use their gestures to demonstrate and manipulate virtual objects simultaneously. This functionality enables the instructor to ef- fectively and efficiently explain and illustrate complex concepts or sophisticated problems in an intuitive manner. The highly interactive and flexible environment can capture and sustain more student attention than the traditional classroom setting and, thus, delivers a compelling experience to the students. Our main focus here is to investigate possible solutions for the system design and implementation and devise strategies for fast, efficient computation suitable for visual data processing and network transmission. We describe the technique and experiments in details and provide quantitative performance results, demonstrating our system can be run comfortably and reliably for different application scenarios. Our preliminary results are promising and demonstrate the potential for more compelling directions in cyberlearning.Comment: IEEE International Symposium on Multimedia 201

From ‘hands up’ to ‘hands on’: harnessing the kinaesthetic potential of educational gaming

Traditional approaches to distance learning and the student learning journey have focused on closing the gap between the experience of off-campus students and their on-campus peers. While many initiatives have sought to embed a sense of community, create virtual learning environments and even build collaborative spaces for team-based assessment and presentations, they are limited by technological innovation in terms of the types of learning styles they support and develop. Mainstream gaming development – such as with the Xbox Kinect and Nintendo Wii – have a strong element of kinaesthetic learning from early attempts to simulate impact, recoil, velocity and other environmental factors to the more sophisticated movement-based games which create a sense of almost total immersion and allow untethered (in a technical sense) interaction with the games’ objects, characters and other players. Likewise, gamification of learning has become a critical focus for the engagement of learners and its commercialisation, especially through products such as the Wii Fit. As this technology matures, there are strong opportunities for universities to utilise gaming consoles to embed levels of kinaesthetic learning into the student experience – a learning style which has been largely neglected in the distance education sector. This paper will explore the potential impact of these technologies, to broadly imagine the possibilities for future innovation in higher education