Neural Network Driven Eye Tracking Metrics and Data Visualization in Metaverse and Virtual Reality Maritime Safety Training

Understand the human brain, predict human performance, and proactively plan, strategize and act based on such information initiated a scientific multidisciplinary alliance to address modern management challenges. This paper integrates numerous advanced information technologies such as eye tracking, virtual reality and neural networks for cognitive task analysis leading to behavioral analysis on humans that perform specific activities. The technology developed and presented in this paper has been tested on a maritime safety training application for command bridge communication and procedures for collision avoidance. The technology integrates metaverse and virtual reality environments with eye tracking for the collection of behavioral data which are analyzed by a neural network to indicate the mental and physical state, attention and readiness of a seafarer to perform such a critical task. The paper demonstrates the technology architecture, data collection process, indicative results, and areas for further research

Detecting Soldiers' Fatigue Using Eye-Tracking Glasses: Practical Field Applications and Research Opportunities.

Objectively determining soldiers' fatigue levels could help prevent injuries or accidents resulting from inattention or decreased alertness. Eye-tracking technologies, such as optical eye tracking (OET) and electrooculography (EOG), are often used to monitor fatigue. Eyeblinks-especially blink frequency and blink duration-are known as easily observable and valid biomarkers of fatigue. Currently, various eye trackers (i.e., eye-tracking glasses) are available on the market using either OET or EOG technologies. These wearable eye trackers offer several advantages, including unobtrusive functionality, practicality, and low costs. However, several challenges and limitations must be considered when implementing these technologies in the field to monitor fatigue levels. This review investigates the feasibility of eye tracking in the field focusing on the practical applications in military operational environments.; This paper summarizes the existing literature about eyeblink dynamics and available wearable eye-tracking technologies, exposing challenges and limitations, as well as discussing practical recommendations on how to improve the feasibility of eye tracking in the field.; So far, no eye-tracking glasses can be recommended for use in a demanding work environment. First, eyeblink dynamics are influenced by multiple factors; therefore, environments, situations, and individual behavior must be taken into account. Second, the glasses' placement, sunlight, facial or body movements, vibrations, and sweat can drastically decrease measurement accuracy. The placement of the eye cameras for the OET and the placement of the electrodes for the EOG must be chosen consciously, the sampling rate must be minimal 200 Hz, and software and hardware must be robust to resist any factors influencing eye tracking.; Monitoring physiological and psychological readiness of soldiers, as well as other civil professionals that face higher risks when their attention is impaired or reduced, is necessary. However, improvements to eye-tracking devices' hardware, calibration method, sampling rate, and algorithm are needed in order to accurately monitor fatigue levels in the field

Using natural head movements to continually calibrate EOG signals

Electrooculography (EOG) is the measurement of eye movements using surface electrodes adhered around the eye. EOG systems can be designed to have an unobtrusive form-factor that is ideal for eye tracking in free-living over long durations, but the relationship between voltage and gaze direction requires frequent re-calibration as the skin-electrode impedance and retinal adaptation vary over time. Here we propose a method for automatically calibrating the EOG-gaze relationship by fusing EOG signals with gyroscopic measurements of head movement whenever the vestibulo-ocular reflex (VOR) is active. The fusion is executed as recursive inference on a hidden Markov model that accounts for all rotational degrees-of-freedom and uncertainties simultaneously. This enables continual calibration using natural eye and head movements while minimizing the impact of sensor noise. No external devices like monitors or cameras are needed. On average, our method’s gaze estimates deviate by 3.54° from those of an industry-standard desktop video-based eye tracker. Such discrepancy is on par with the latest mobile video eye trackers. Future work is focused on automatically detecting moments of VOR in free-living

Tracking team mental workload by multimodal measurements in the operating room

Mental workload and its effects on surgical performance are underexplored topics, despite their importance for operating room (OR) efficiency and patient safety. We developed a multimodal platform that can simultaneously collect data from EEG, heart rate and breathing rate, tool handle pressure, and eye tracker from mobile subjects. We performed experiments using the Fundamentals of Laparoscopic Surgery model, with 22 subjects of varying skill levels ranging from nonsurgeon to expert. The results indicated significant modulations of the measurements depending on pupil size, heart rate variability, P300 response, tool pressure, task difficulty, time-on-task, and skill level. These provide evidence that physiology based metrics can be used in automated classification of fine gradations of skill, the assessment and certification of surgery trainees, developing real-time flags and warnings for the OR, and validating new OR technology

Unobtrusive and pervasive video-based eye-gaze tracking

Eye-gaze tracking has long been considered a desktop technology that finds its use inside the traditional office setting, where the operating conditions may be controlled. Nonetheless, recent advancements in mobile technology and a growing interest in capturing natural human behaviour have motivated an emerging interest in tracking eye movements within unconstrained real-life conditions, referred to as pervasive eye-gaze tracking. This critical review focuses on emerging passive and unobtrusive video-based eye-gaze tracking methods in recent literature, with the aim to identify different research avenues that are being followed in response to the challenges of pervasive eye-gaze tracking. Different eye-gaze tracking approaches are discussed in order to bring out their strengths and weaknesses, and to identify any limitations, within the context of pervasive eye-gaze tracking, that have yet to be considered by the computer vision community.peer-reviewe

Ocular attention-sensing interface system

The purpose of the research was to develop an innovative human-computer interface based on eye movement and voice control. By eliminating a manual interface (keyboard, joystick, etc.), OASIS provides a control mechanism that is natural, efficient, accurate, and low in workload

Eye movement analysis and cognitive assessment: the use of comparative visual search tasks in a non-immersive vr application

Background: An adequate behavioral response depends on attentional and mnesic processes. When these basic cognitive functions are impaired, the use of non-immersive Virtual Reality Applications (VRAs) can be a reliable technique for assessing the level of impairment. However, most non-immersive VRAs use indirect measures to make inferences about visual attention and mnesic processes (e.g., time to task completion, error rate). Objectives: To examine whether the eye movement analysis through eye tracking (ET) can be a reliable method to probe more effectively where and how attention is deployed and how it is linked with visual working memory during comparative visual search tasks (CVSTs) in non-immersive VRAs. Methods: The eye movements of 50 healthy participants were continuously recorded while CVSTs, selected from a set of cognitive tasks in the Systemic Lisbon Battery (SLB). Then a VRA designed to assess of cognitive impairments were randomly presented. Results: The total fixation duration, the number of visits in the areas of interest and in the interstimulus space, along with the total execution time was significantly different as a function of the Mini Mental State Examination (MMSE) scores. Conclusions: The present study demonstrates that CVSTs in SLB, when combined with ET, can be a reliable and unobtrusive method for assessing cognitive abilities in healthy individuals, opening it to potential use in clinical samples.info:eu-repo/semantics/submittedVersio

Trial application of pupillometry for a maritime usability study in field conditions

Eye-tracking is a tool employed in usability testing. It is primarily intended as a means for tracking the visual attention patterns of an observer on a continuous basis. Eye-tracking can also capture certain physiological data, such as pupil dilation. Pupil diameter is a validated metric of cognitive workload, meaning the pupil dilates with increasing workload. This research evaluates the fitness, in field conditions, of an eye-tracking based method for accurate measurement of cognitive workload. This implies evaluating the fitness of this tool in changing light conditions such as in coastal navigation. This methodology thus accounts for the effect of light on pupil dilation. This means we are able to account for the effect of only cognitive workload on the pupil dilation even in changing light conditions. This method was applied as a part of an analysis of a navigational exercise involving the navigator and the navigator’s assistant on board a training vessel of the Royal Norwegian Naval Academy. Pupillometry is used alongside egocentric video recordings and Geo-positioning systems (GPS) recordings to allow for multi-faceted evaluation of the activity. Subjective data was recorded as well to evaluate the quality of the eye-tracking data. Subjective data was recorded using NASA-TLX self-report of mental workload, self-report of mental workload (on three levels) using maps and an expert assessment of the mental workload was obtained for the navigational course. The analysis concluded that pupillometry (through eye-trackers) can have a substantial role in the evaluation of field operations and provide a good and objective estimate of the perceived workload. The eye-tracking technology has substantial limitations, for example sometimes strong infra-red sources of light can impede data collection as such with an eye-tracker, meaning the analysis is labour intensive as it relies on the ability of the operator to filter out low quality data and retain the rest