Human Computer Interaction Opportunities in Hand Tracking and Finger Recognition in Ship Engine Room VR Training

The research conducted for this paper is an extension of the continuous efforts at the Turku University of applied Sciences to optimize the Maritime Immersive Ocean Technology (MarsISOT) by integrating advanced immersive technologies. This paper reports the integration of hand tracking and finger recognition in the ShipSEVR learning episode. ShipSEVR, part of MarISOT, is a VR Training next generation learning environment focused specifically on ship engines and engine rooms safety procedures. The technology has been designed and developed at Turku University of Applied Sciences for Wärtsilä Land and Sea Academy utilizing latest VR technologies. The delivered learning episode consists of a 3D ship engine room space where trainees are expected to find certain devices and equipment by utilizing the available technical drawings. This enhanced human computer interaction environment reflects to industry requirements the derived after the first version of the technology and its test with industry experts

Eye tracking in maritime immersive safe oceans technology

This paper presents the integration of eye tracking in the MarSEVR (Maritime Safety Education with VR) technology to increase the precision of the trainee focus on delivering the learning episodes of the technology with enhanced impressiveness and user engagement. MarSEVR is part of the Safe Oceans concept, a green ocean technology that integrates several VR safety training applications to reduce maritime accidents that result into human casualties, sea pollution and other environmental damages. The paper indicates the research delivery architecture driven by Hevner's design science in information systems Research for usability, use experience (UX) and effectiveness. Furthermore, this technology integration is approached from a game design perspective for user engagement but also from a cognitive and neuroscience perspective for pedagogical use and purposes. The paper addressees the impact of the eye tracking technology in the maritime sector operations, training market, and competitive research. Lastly areas of further research are presented and the efforts to link and align finger tracking and hand recognitions technologies with eye tracking for a more complete VR training environment

Avatar Based Multiplayer Functionalities in Next Generation Communication and Learning in Virtual Reality Social Platforms – Case MarISOT Room

Virtual Reality Social Platforms are increasingly in use in conferences, and exhibitions. COVID19 pandemic is the main reason for a rapid growth but there are reasons such as climate change to believe these technologies will be used widely once we will enter to the post pandemic era. In this paper, we will introduce our vision to combine social functioning and hands-on-training features in one package. We have participated in and organized various events using existing virtual reality social platforms. We will analyze in this paper their pros and cons plus introduce our own technology trials. As a result, we have better understanding what development tools and how much resources would be needed to be able to create the first minimal viable product combining social functioning and hands-on-training features

Neural Network Driven Eye Tracking Metrics and Data Visualization in Metaverse and Virtual Reality Maritime Safety Training

Understand the human brain, predict human performance, and proactively plan, strategize and act based on such information initiated a scientific multidisciplinary alliance to address modern management challenges. This paper integrates numerous advanced information technologies such as eye tracking, virtual reality and neural networks for cognitive task analysis leading to behavioral analysis on humans that perform specific activities. The technology developed and presented in this paper has been tested on a maritime safety training application for command bridge communication and procedures for collision avoidance. The technology integrates metaverse and virtual reality environments with eye tracking for the collection of behavioral data which are analyzed by a neural network to indicate the mental and physical state, attention and readiness of a seafarer to perform such a critical task. The paper demonstrates the technology architecture, data collection process, indicative results, and areas for further research

Learning Impact of a Virtual Brain Electrical Activity Simulator Among Neurophysiology Students: Mixed-Methods Intervention Study

Background:Virtual simulation is the re-creation of reality depicted on a computer screen. It offers the possibility to exercise motor and psychomotor skills. In biomedical and medical education, there is an attempt to find new ways to support students’ learning in neurophysiology. Traditionally, recording electroencephalography (EEG) has been learned through practical hands-on exercises. To date, virtual simulations of EEG measurements have not been used.Objective:This study aimed to examine the development of students’ theoretical knowledge and practical skills in the EEG measurement when using a virtual EEG simulator in biomedical laboratory science in the context of a neurophysiology course.Methods:A computer-based EEG simulator was created. The simulator allowed virtual electrode placement and EEG graph interpretation. The usefulness of the simulator for learning EEG measurement was tested with 35 participants randomly divided into three equal groups. Group 1 (experimental group 1) used the simulator with fuzzy feedback, group 2 (experimental group 2) used the simulator with exact feedback, and group 3 (control group) did not use a simulator. The study comprised pre- and posttests on theoretical knowledge and practical hands-on evaluation of EEG electrode placement.Results:The Wilcoxon signed-rank test indicated that the two groups that utilized a computer-based electrode placement simulator showed significant improvement in both theoretical knowledge (Z=1.79, P=.074) and observed practical skills compared with the group that studied without a simulator.Conclusions:Learning electrode placement using a simulator enhances students’ ability to place electrodes and, in combination with practical hands-on training, increases their understanding of EEG measurement.</p

Rethinking Serious Games Design in the Age of COVID-19: Setting the Focus on Wicked Problems

We live in a complex world, in which our existence is defined by forces that we cannot fully comprehend, predict, nor control. This is the world of wicked problems, of which the situation triggered by the COVID-19 pandemic is a notable example. Wicked problems are complex scenarios defined by the interplay of multiple environmental, social and economic factors. They are everchanging, and largely unpredictable and uncontrollable. As a consequence, wicked problems cannot be definitively solved through traditional problem-solving approaches. Instead, they should be iteratively managed, recognizing and valuing our connectedness with each other and the environment, and engaging in joint thinking and action to identify and pursue the common good. Serious games can be key to foster wicked problem management abilities. To this end, they should engage players in collective activities set in contexts simulating real-world wicked problem scenarios. These should require the continuous interpretation of changing circumstances to identify and pursue shared goals, promoting the development of knowledge, attitudes and skill sets relevant to tackle real-world situations. In this paper we outline the nature, implications and challenges of wicked problems, highlighting why games should be leveraged to foster wicked problem management abilities. Then, we propose a theory-based framework to support the design of games for this purpose

Learning impact of a virtual brain electrical activity simulator among neurophysiology students:mixed-methods intervention study

Abstract Background: Virtual simulation is the re-creation of reality depicted on a computer screen. It offers the possibility to exercise motor and psychomotor skills. In biomedical and medical education, there is an attempt to find new ways to support students’ learning in neurophysiology. Traditionally, recording electroencephalography (EEG) has been learned through practical hands-on exercises. To date, virtual simulations of EEG measurements have not been used. Objective: This study aimed to examine the development of students’ theoretical knowledge and practical skills in the EEG measurement when using a virtual EEG simulator in biomedical laboratory science in the context of a neurophysiology course. Methods: A computer-based EEG simulator was created. The simulator allowed virtual electrode placement and EEG graph interpretation. The usefulness of the simulator for learning EEG measurement was tested with 35 participants randomly divided into three equal groups. Group 1 (experimental group 1) used the simulator with fuzzy feedback, group 2 (experimental group 2) used the simulator with exact feedback, and group 3 (control group) did not use a simulator. The study comprised pre- and posttests on theoretical knowledge and practical hands-on evaluation of EEG electrode placement. Results: The Wilcoxon signed-rank test indicated that the two groups that utilized a computer-based electrode placement simulator showed significant improvement in both theoretical knowledge (Z=1.79, P=.074) and observed practical skills compared with the group that studied without a simulator. Conclusions: Learning electrode placement using a simulator enhances students’ ability to place electrodes and, in combination with practical hands-on training, increases their understanding of EEG measurement