Human-in-the-Loop Configurations in Process and Energy Industries: A Systematic Review

The human-in-the-loop performance evaluation is an area of growing interest in industries where safety-critical systems are in place. Concerns here are due to the increasing complexity of automation, new technologies for control, and safety. Because, unlike a more traditional approach to evaluating the human and the system they work with, human-in-the-loop gives a holistic view of their interaction (human, automation or artificial intelligence) and dynamics. It also emphasizes adapting the technology or automation to the human, being central, considering certain factors like risk. Therefore, there is a need to identify the relevant factors, novel measures and methods or improvements on existing methods that can be adapted for this field of research. This paper intends to present an overview of human-in-the-loop in the process and energy industries by presenting a literature summary highlighting current factors and measures, methods, gaps, solutions and future work. Experimental (13) and observational (11) studies have been reviewed for results. It was observed that new factors, measures and techniques are currently being explored to fill some of the current gaps for the human-in-the-loop, for example, during performance assessment new methods and modalities have been adopted such as eye tracking and electroencephalography methods. The results and open questions from the papers reviewed and possible future research opportunities are presented and discussed in this paper

An eye tracking based framework for safety improvement of offshore operations

Offshore drilling operations consist of complex and high-risk processes. Lack of situational awareness in drilling operations has become an important human factor issue that causes safety accidents. Prolonged work shifts and fatigue are some of the crucial issues that impact performance. Eye tracking technology can be used to distinguish the degree of awareness or alertness of participants that might be related to fatigue or onsite distractions. Oculomotor activity can be used to obtain visual cues that can quantify the drilling operators’ situational awareness that might enable us to develop warning alarms to alert the driller. Such systems can help reduce accidents and save non-productive time. In this paper, eye movement characteristics were investigated to differentiate the situational awareness between a representative expert and a group of novices using a scenario-based Virtual Reality Drilling Simulator. Significant visual oculomotor activity differences were identified between the expert and the novices that indicate an eye-tracking based system can detect the distraction and alertness exhibited by the workers. Results show promise on developing a framework which implements a real-time eye tracking technology in various drilling operations at drilling rigs and Real Time Operation Centers to improve process safety

Multimodal Neuroergonomic Approaches to Human Behavior and Cognitive Workload in Complex High-Risk Semantically Rich Environments: A Case Study of Local & En-Route Air Traffic Controllers

Fast-paced technology advancements have enabled us to create ecologically valid simulations of high risk, complex, and semantically rich environments in which human interaction and decision-making are the keys to increase system performance. These advances have improved our capabilities of exploring, quantifying, and measuring the underlying mechanisms that guide human behavior using sophisticated neuroergonomic devices; and in turn, improve human performance and reduce human errors. In this thesis, multimodal approaches consisted of a self-report analysis, eye-tracking analysis, and functional near-infrared spectroscopy analysis were used to investigate how veteran local & en-route air traffic controllers carry out their operational tasks. Furthermore, the correlations among the cognitive workload and physiological measures (i.e. eye movement characteristics and brain activities) were investigated. Combining the results of these experiments, we can observe that the multimodal approaches show promise on exploring the underlying mechanisms of workload and human interaction in a complex, high-risk, and semantically rich environment. This is because cognitive workload can be considered as a multidimensional construct and different devices or approaches might be more effective in sensing changes in either the task difficulty or complexity. The results can be used to find ways to better train the novices