Teaching clinical reasoning and decision-making skills to nursing students: Design, development, and usability evaluation of a serious game

Background Serious games (SGs) are a type of simulation technology that may provide nursing students with the opportunity to practice their clinical reasoning and decision-making skills in a safe and authentic environment. Despite the growing number of SGs developed for healthcare professionals, few SGs are video based or address the domain of home health care. Aims This paper aims to describe the design, development, and usability evaluation of a video based SG for teaching clinical reasoning and decision-making skills to nursing students who care for patients with chronic obstructive pulmonary disease (COPD) in home healthcare settings. Methods A prototype SG was developed. A unified framework of usability called TURF (Task, User, Representation, and Function) and SG theory were employed to ensure a user-centered design. The educational content was based on the clinical decision-making model, Bloom’s taxonomy, and a Bachelor of Nursing curriculum. A purposeful sample of six participants evaluated the SG prototype in a usability laboratory. Cognitive walkthrough evaluations, a questionnaire, and individual interviews were used for the usability evaluation. The data were analyzed using qualitative deductive content analysis based on the TURF framework elements and related usability heuristics. Results The SG was perceived as being realistic, clinically relevant, and at an adequate level of complexity for the intended users. Usability issues regarding functionality and the user–computer interface design were identified. However, the SG was perceived as being easy to learn, and participants suggested that the SG could serve as a supplement to traditional training in laboratory and clinical settings. Conclusions Using video based scenarios with an authentic COPD patient and a home healthcare registered nurse as actors contributed to increased realism. Using different theoretical approaches in the SG design was considered an advantage of the design process. The SG was perceived as being useful, usable, and satisfying. The achievement of the desired functionality and the minimization of user–computer interface issues emphasize the importance of conducting a usability evaluation during the SG development process

Applicability of Immersive Analytics in Mixed Reality: Usability Study

Nowadays, visual analytics is mainly performed by programming approaches and viewing the results on a desktop monitor. However, due to the capabilities of smart glasses, new user interactions and representation possibilities become possible. This refers especially to 3D visualizations in the medical field, as well as, the industry domain, as valuable depth information can be related to the complex real-world structures and related data, which is also denoted as immersive analytics. However, the applicability of immersive analytics and its drawbacks, especially in the context of mixed reality, are quite unexplored. In order to validate the feasibility of immersive analytics for the aforementioned purposes, we designed and conducted a usability study with 60 participants. More specifically, we evaluated the effects of spatial sounds, performance changes from one analytics task to another, expert status, and compared an immersive analytics approach (i.e., a mixed-reality application) with a desktop-based solution. Participants had to solve several data analytics tasks (outlier’s detection and cluster recognition) with the developed mixed-reality application. Thereby, the performance measures regarding time, errors, and movement patterns were evaluated. The separation into groups (low performer vs. high performer) was performed using a mental rotation pretest. When solving analytic tasks in mixed reality, participants changed their movement patterns in the mixed reality setting significantly, while the use of spatial sounds reduced the handling time significantly, but did not affect the movement patterns. Furthermore, the usage of mixed reality for cluster recognition is significantly faster than the desktop-based solution (i.e., a 2D approach). Moreover, the results obtained with self-developed questionnaires indicate 1) that wearing smart glasses are perceived as a potential stressor and 2) that the utilization of sounds is perceived very differently by the participants. Altogether, industry and researchers should consider immersive analytics as a suitable alternative compared to the traditional approaches

A comprehensive method to design and assess mixed reality simulations

AbstractThe scientific literature highlights how Mixed Reality (MR) simulations allow obtaining several benefits in healthcare education. Simulation-based training, boosted by MR, offers an exciting and immersive learning experience that helps health professionals to acquire knowledge and skills, without exposing patients to unnecessary risks. High engagement, informational overload, and unfamiliarity with virtual elements could expose students to cognitive overload and acute stress. The implementation of effective simulation design strategies able to preserve the psychological safety of learners and the investigation of the impacts and effects of simulations are two open challenges to be faced. In this context, the present study proposes a method to design a medical simulation and evaluate its effectiveness, with the final aim to achieve the learning outcomes and do not compromise the students' psychological safety. The method has been applied in the design and development of an MR application to simulate the rachicentesis procedure for diagnostic purposes in adults. The MR application has been tested by involving twenty students of the 6th year of Medicine and Surgery of Università Politecnica delle Marche. Multiple measurement techniques such as self-report, physiological indices, and observer ratings of performance, cognitive and emotional states of learners have been implemented to improve the rigour of the study. Also, a user-experience analysis has been accomplished to discriminate between two different devices: Vox Gear Plus® and Microsoft Hololens®. To compare the results with a reference, students performed the simulation also without using the MR application. The use of MR resulted in increased stress measured by physiological parameters without a high increase in perceived workload. It satisfies the objective to enhance the realism of the simulation without generating cognitive overload, which favours productive learning. The user experience (UX) has found greater benefits in involvement, immersion, and realism; however, it has emphasized the technological limitations of devices such as obstruction, loss of depth (Vox Gear Plus), and narrow FOV (Microsoft Hololens)

Investigating Real-time Touchless Hand Interaction and Machine Learning Agents in Immersive Learning Environments

The recent surge in the adoption of new technologies and innovations in connectivity, interaction technology, and artificial realities can fundamentally change the digital world. eXtended Reality (XR), with its potential to bridge the virtual and real environments, creates new possibilities to develop more engaging and productive learning experiences. Evidence is emerging that thissophisticated technology offers new ways to improve the learning process for better student interaction and engagement. Recently, immersive technology has garnered much attention as an interactive technology that facilitates direct interaction with virtual objects in the real world. Furthermore, these virtual objects can be surrogates for real-world teaching resources, allowing for virtual labs. Thus XR could enable learning experiences that would not bepossible in impoverished educational systems worldwide. Interestingly, concepts such as virtual hand interaction and techniques such as machine learning are still not widely investigated in immersive learning. Hand interaction technologies in virtual environments can support the kinesthetic learning pedagogical approach, and the need for its touchless interaction nature hasincreased exceptionally in the post-COVID world. By implementing and evaluating real-time hand interaction technology for kinesthetic learning and machine learning agents for self-guided learning, this research has addressed these underutilized technologies to demonstrate the efficiency of immersive learning. This thesis has explored different hand-tracking APIs and devices to integrate real-time hand interaction techniques. These hand interaction techniques and integrated machine learning agents using reinforcement learning are evaluated with different display devices to test compatibility. The proposed approach aims to provide self-guided, more productive, and interactive learning experiences. Further, this research has investigated ethics, privacy, and security issues in XR and covered the future of immersive learning in the Metaverse.<br/

Investigating Real-time Touchless Hand Interaction and Machine Learning Agents in Immersive Learning Environments

The recent surge in the adoption of new technologies and innovations in connectivity, interaction technology, and artificial realities can fundamentally change the digital world. eXtended Reality (XR), with its potential to bridge the virtual and real environments, creates new possibilities to develop more engaging and productive learning experiences. Evidence is emerging that thissophisticated technology offers new ways to improve the learning process for better student interaction and engagement. Recently, immersive technology has garnered much attention as an interactive technology that facilitates direct interaction with virtual objects in the real world. Furthermore, these virtual objects can be surrogates for real-world teaching resources, allowing for virtual labs. Thus XR could enable learning experiences that would not bepossible in impoverished educational systems worldwide. Interestingly, concepts such as virtual hand interaction and techniques such as machine learning are still not widely investigated in immersive learning. Hand interaction technologies in virtual environments can support the kinesthetic learning pedagogical approach, and the need for its touchless interaction nature hasincreased exceptionally in the post-COVID world. By implementing and evaluating real-time hand interaction technology for kinesthetic learning and machine learning agents for self-guided learning, this research has addressed these underutilized technologies to demonstrate the efficiency of immersive learning. This thesis has explored different hand-tracking APIs and devices to integrate real-time hand interaction techniques. These hand interaction techniques and integrated machine learning agents using reinforcement learning are evaluated with different display devices to test compatibility. The proposed approach aims to provide self-guided, more productive, and interactive learning experiences. Further, this research has investigated ethics, privacy, and security issues in XR and covered the future of immersive learning in the Metaverse.<br/

The HoloLens in Medicine: A systematic Review and Taxonomy

The HoloLens (Microsoft Corp., Redmond, WA), a head-worn, optically see-through augmented reality display, is the main player in the recent boost in medical augmented reality research. In medical settings, the HoloLens enables the physician to obtain immediate insight into patient information, directly overlaid with their view of the clinical scenario, the medical student to gain a better understanding of complex anatomies or procedures, and even the patient to execute therapeutic tasks with improved, immersive guidance. In this systematic review, we provide a comprehensive overview of the usage of the first-generation HoloLens within the medical domain, from its release in March 2016, until the year of 2021, were attention is shifting towards it's successor, the HoloLens 2. We identified 171 relevant publications through a systematic search of the PubMed and Scopus databases. We analyze these publications in regard to their intended use case, technical methodology for registration and tracking, data sources, visualization as well as validation and evaluation. We find that, although the feasibility of using the HoloLens in various medical scenarios has been shown, increased efforts in the areas of precision, reliability, usability, workflow and perception are necessary to establish AR in clinical practice.Comment: 35 pages, 11 figure

An Evaluation of Game Usability in Shared Mixed and Virtual Environments

Augmented reality (AR) and virtual reality (VR) technologies are increasingly becoming more pervasive and important in the information technology area. Thanks to the technological improvements, desktop interfaces are being replaced by immersive VR devices that offer a more compelling game experience. AR games have started to draw attention of researchers and companies for their ability to exploit both the real and virtual environments. New fascinating challenges are generated by the possibility of designing hybrid games that allow several users to access shared environments exploiting the features of both AR and VR devices. However, the user experience and usability can be affected by several parameters, such as the field of view (FoV) of the employed devices or the realism of the scene. The work presented in this chapter aims to assess the impact of the FoV on the usability of the interfaces in a first-person shooter game. Two players, interacting with AR (first player) and VR (second player) devices, can fight each other in a large game environment. Although we cannot ascertain that different FoVs have affected the game usability, users considered the narrow FoV interfaces to be less usable, even though they could freely move around the real environment