End-to-End Eye Movement Detection Using Convolutional Neural Networks

Common computational methods for automated eye movement detection - i.e. the task of detecting different types of eye movement in a continuous stream of gaze data - are limited in that they either involve thresholding on hand-crafted signal features, require individual detectors each only detecting a single movement, or require pre-segmented data. We propose a novel approach for eye movement detection that only involves learning a single detector end-to-end, i.e. directly from the continuous gaze data stream and simultaneously for different eye movements without any manual feature crafting or segmentation. Our method is based on convolutional neural networks (CNN) that recently demonstrated superior performance in a variety of tasks in computer vision, signal processing, and machine learning. We further introduce a novel multi-participant dataset that contains scripted and free-viewing sequences of ground-truth annotated saccades, fixations, and smooth pursuits. We show that our CNN-based method outperforms state-of-the-art baselines by a large margin on this challenging dataset, thereby underlining the significant potential of this approach for holistic, robust, and accurate eye movement protocol analysis

A standardized protocol for quantification of saccadic eye movements: DEMoNS

OBJECTIVE: Quantitative saccadic testing is a non-invasive method of evaluating the neural networks involved in the control of eye movements. The aim of this study is to provide a standardized and reproducible protocol for infrared oculography measurements of eye movements and analysis, which can be applied for various diseases in a multicenter setting. METHODS: Development of a protocol to Demonstrate Eye Movement Networks with Saccades (DEMoNS) using infrared oculography. Automated analysis methods were used to calculate parameters describing the characteristics of the saccadic eye movements. The two measurements of the subjects were compared with descriptive and reproducibility statistics. RESULTS: Infrared oculography measurements of all subjects were performed using the DEMoNS protocol and various saccadic parameters were calculated automatically from 28 subjects. Saccadic parameters such as: peak velocity, latency and saccade pair ratios showed excellent reproducibility (intra-class correlation coefficients > 0.9). Parameters describing performance of more complex tasks showed moderate to good reproducibility (intra-class correlation coefficients 0.63-0.78). CONCLUSIONS: This study provides a standardized and transparent protocol for measuring and analyzing saccadic eye movements in a multicenter setting. The DEMoNS protocol details outcome measures for treatment trial which are of excellent reproducibility. The DEMoNS protocol can be applied to the study of saccadic eye movements in various neurodegenerative and motor diseases

Improving the economic value of photographic screening for optical coherence tomography-detectable macular oedema : a prospective, multicentre, UK study

Peer reviewedPublisher PD

The evaluation of pilots performance and mental workload by eye movement

Pilots make important decisions often using ambiguous information, while under stresses and with very little time. During flight operations detecting the warning light of system failure is a task with real-world application relates to measurement of pilot's performance and eye movement. The demand for a pilot’s visual and situational awareness in multiple tasks can be detrimental during pilots’ mental overload conditions. The purpose of this research is to evaluate the relationship between pilot’s mental workload and operational performance by eye tracking. Collecting eye movement data during flight operations in a virtual reality of flight simulator provided useful information to analysis participants’ cognitive processes. There were 36 pilots participated in this research, the experience of flight hours between 320 and 2,920, the range of age between 26 and 51 years old. The apparatus included Applied Science Laboratories (ASL) eye tracking, IDF flight simulator and NASA_TLX for data collection. The results show that pilots with high SA detecting hydraulic malfunction have shorter total fixation duration on Air Speed Indicator and longer total fixation duration on Altitude Indicator, Vertical Speed Indicator, Right multi-display and Left multi-display compared with pilots without detecting the signal of hydraulic malfunction. Pilots’ total fixation time on Integration Control Panel, Altitude Indicator, Attitude Indicator and Right Multi-display, and pilots’ subjective rating on NASA-TLX effort dimension for the mission of close pattern have significant relationship with pilots’ performance on the operational time for completing the tactic mission. Experienced pilots operate aircraft familiar with monitoring Airspeed Indicator and kinetic maneuvering result in less fuel consumption. This study could provide guidelines for future training design to reduce pilots mental workload and improve situational awareness for enhancing flight safety

Visual search and VDUS

This wide-ranging study explored various parameters of visual search in relation to computer screen displays. Its ultimate goal was to help identify factors which could result in improvements in commercially available displays within the 'real world’. Those improvements are generally reflected in suggestions for enhancing efficiency of locatabolity of information through an acknowledgement of the visual and cognitive factors involved. The thesis commenced by introducing an ergonomics approach to the presentation of information on VDUs. Memory load and attention were discussed. In the second chapter, literature on general and theoretical aspects of visual search (with particular regard for VDUs) was reviewed. As an experimental starting point, three studies were conducted involving locating a target within arrays of varying configurations. A model concerning visual lobes was proposed. Two text-editing studies were then detailed showing superior user performances where conspicuity and the potential for peripheral vision are enhanced. Relevant eye movement data was combined with a keystroke analysis derived from an automated protocol analyser. Results of a further search task showed icons to be more quickly located within an array than textual material. Precise scan paths were then recorded and analyses suggested greater systematicity of search strategies for complex items. This led on to a relatively 'pure' search study involving materials of varying spatial frequencies. Results were discussed in terms of verbal material generally being of higher spatial frequencies and how the ease of resolution and greater cues available in peripheral vision can result in items being accessed more directly. In the final (relatively applied) study, differences in eye movement indices were found across various fonts used. One main conclusion was that eye movement monitoring was a valuable technique within the visual search/VDU research area in illuminating precise details of performance which otherwise, at best, could only be inferred

Automated design analysis, assembly planning and motion study analysis using immersive virtual reality

Previous research work at Heriot-Watt University using immersive virtual reality (VR) for cable harness design showed that VR provided substantial productivity gains over traditional computer-aided design (CAD) systems. This follow-on work was aimed at understanding the degree to which aspects of this technology were contributed to these benefits and to determine if engineering design and planning processes could be analysed in detail by nonintrusively monitoring and logging engineering tasks. This involved using a CAD-equivalent VR system for cable harness routing design, harness assembly and installation planning that can be functionally evaluated using a set of creative design-tasks to measure the system and users' performance. A novel design task categorisation scheme was created and formalised which broke down the cable harness design process and associated activities. The system was also used to demonstrate the automatic generation of usable bulkhead connector, cable harness assembly and cable harness installation plans from non-intrusive user logging. Finally, the data generated from the user-logging allowed the automated activity categorisation of the user actions, automated generation of process flow diagrams and chronocyclegraphs