Mesure et analyse de l’activité cérébrale par des techniques d’imagerie en proche infrarouge

La mesure de charge attentionnelle est un enjeu d’avenir dans la mise en place d’interfaces adaptatives dans le milieu aéronautique, pour améliorer les conditions d’utilisation et la sécurité (en particulier dans la détection de phénomènes attentionnels extrêmes tels que la tunnélisation ou la divagation attentionnelles). Les mesures physiologiques, par leur utilisabilité en temps réel et leur estimation objective du niveau de charge attentionnelle, sont d’un intérêt croissant pour les chercheurs. Dans ce rapport, nous étudions les différentes métriques associées à la mesure de charge attentionnelle, et expérimentons l’utilisabilité de l’imagerie par spectroscopie en proche infrarouge (une méthode d’imagerie récente) dans la détection de niveaux de charge attentionnelle en situation de pilotage. Dans ce cadre, nous avons mis en place une expérience faisant intervenir les effets de la mémoire et de la difficulté de la tâche sur la charge attentionnelle. Les résultats prometteurs de cette méthode d’imagerie sur cette expérience nous permettent d’envisager son utilisation dans le cadre d’une interface adaptative. To improve security and usability conditions in the aeronautic field, being able of monitoring user’s attentional load and using its measure in an adaptive interface is becoming a widely considered solution (especially concerning the detection of extreme attentional states such as attentional tunneling or low alertness). In this regard, physiological measures bring a real-time and objective measure of attentional load. In this report, we consider different metrics of the attentional load, and study more specifically (through experimentation) the usability of functional Near Infrared Spectroscopy (NIRS) as an assessment of pilot’s attentional workload. An experiment was settled to look at the effect of memory and task difficulty on attentional workload. The promising results of NIRS on this experiment encourage us in using it in an adaptive interface

Using near infrared spectroscopy and heart rate variability to detect mental overload

Mental workload is a key factor influencing the occurrence of human error, especially during piloting and remotely operated vehicle (ROV) operations, where safety depends on the ability of pilots to act appropriately. In particular, excessively high or low mental workload can lead operators to neglect critical information. The objective of the present study is to investigate the potential of functional Near Infrared Spectroscopy (fNIRS) – a non-invasive method of measuring prefrontal cortex activity – in combination with measurements of heart rate variability (HRV), to predict mental workload during a simulated piloting task, with particular regard to task engagement and disengagement. Twelve volunteers performed a computer-based piloting task in which they were asked to follow a dynamic target with their aircraft, a task designed to replicate key cognitive demands associated with real life ROV operating tasks. In order to cover a wide range of mental workload levels, task difficulty was manipulated in terms of processing load and difficulty of control – two critical sources of workload associated with piloting and remotely operating a vehicle. Results show that both fNIRS and HRV are sensitive to different levels of mental workload; notably, lower prefrontal activation as well as a lower LF/HF ratio at the highest level of difficulty, suggest that these measures are suitable for mental overload detection. Moreover, these latter measurements point towards the existence of a quadratic model of mental workload

Real-Time State Estimation in a Flight Simulator Using fNIRS

Working memory is a key executive function for flying an aircraft. This function is particularly critical when pilots have to recall series of air traffic control instructions. However, working memory limitations may jeopardize flight safety. Since the functional near-infrared spectroscopy (fNIRS) method seems promising for assessing working memory load, our objective is to implement an on-line fNIRS-based inference system that integrates two complementary estimators. The first estimator is a real-time state estimation MACD-based algorithm dedicated to identifying the pilot’s instantaneous mental state (not-on-task vs. on-task). It does not require a calibration process to perform its estimation. The second estimator is an on-line SVM-based classifier that is able to discriminate task difficulty (low working memory load vs. high working memory load). These two estimators were tested with 19 pilots who were placed in a realistic flight simulator and were asked to recall air traffic control instructions. We found that the estimated pilot’s mental state matched significantly better than chance with the pilot’s real state (62% global accuracy, 58% specificity, and 72% sensitivity). The second estimator, dedicated to assessing single trial working memory loads, led to 80% classification accuracy, 72% specificity, and 89% sensitivity. These two estimators establish reusable blocks for further fNIRS-based passive brain computer interface development

Neural signature of inattentional deafness

Inattentional deafness is the failure to hear otherwise audible sounds (usually alarms) that may occur under high workload conditions. One potential cause for its occurrence could be an atten- tional bottleneck that occurs when task demands are high, resulting in lack of resources for processing of additional tasks. In this fMRI experiment, we explore the brain regions active during the occurrence of inattentional deafness using a difficult perceptual-motor task in which the participants fly through a simulated Red Bull air race course and at the same time push a button on the joystick to the presence of audio alarms. Participants were instructed to focus on the difficult piloting task and to press the button on the joystick quickly when they noticed an audio alarm. The fMRI results revealed that audio misses relative to hits had significantly greater activity in the right inferior frontal gyrus IFG and the superior medial frontal cortex. Consistent with an attentional bottleneck, activity in these regions was also present for poor flying performance (contrast of gates missed versus gates passed for the flying task). A psychophysiological interaction analysis from the IFG identified reduced effective connectivity to auditory processing regions in the right superior temporal gyrus for missed audio alarms relative to audio alarms that were heard. This study identifies a neural signature of inattentional deafness in an ecologically valid situation by directly measuring differences in brain activity and effective connectivity between audio alarms that were not heard compared to those that were heard

EEG-engagement index and auditory alarm misperception: an inattentional deafness study in actual flight condition

The inability to detect auditory alarms is a critical issue in many do- mains such as aviation. An interesting prospect for flight safety is to understand the neural mechanisms underpinning auditory alarm misperception under actual flight condition. We conducted an experiment in which four pilots were to re- spond by button press when they heard an auditory alarm. The 64 channel Cognionics dry-wireless EEG system was used to measure brain activity in a 4 seat light aircraft. An instructor was present on all flights and in charge of initi- ating the various scenarios to induce two levels of task engagement (simple navigation task vs. complex maneuvering task). Our experiment revealed that inattentional deafness to single auditory alarms could take place as the pilots missed a mean number of 12.5 alarms occurring mostly during the complex maneuvering condition, when the EEG engagement index was high

Anticipating human error before it happens: Towards a psychophysiological model for online prediction of mental workload

Mental workload is a key factor influencing the occurrence of human error; specifically in remotely-operated vehicle operations. Both low and high mental workload has been found to disrupt performance in a nonlinear fashion at a given task; however, research that has attempted to predict individual mental workload has met with little success. The objective of the present study is to investigate the potential of the dual-task paradigm and prefrontal cortex oxygenation as online measures of mental workload. Subjects performed a computerized object tracking task in which they had to follow a dynamic target with their aircraft. Task difficulty was manipulated in terms of processing load and difficulty of control: two critical sources of workload associated with remotely operating a vehicle. Mental workload was assessed by a secondary concurrent time production task and a functional near infrared spectrometer. Results show that the effects of task difficulty differ across measures of mental workload. This pattern of behavioural and neurophysiologic results suggests that the empirically-based selection of an appropriate secondary task for the measure of mental workload is critical as its sensitivity may vary considerably depending on task factors

Concevoir un assistant conversationnel de manière itérative et semi-supervisée avec le clustering interactif

National audienceThe design of a dataset needed to train a chatbot is most often the result of manual and tedious step. To guarantee the efficiency of the annotation, we propose the interactive clustering method, an active learning method based on constraints annotation. It’s an iterative approach, relying on a constrained clustering algorithm and using annotator knowledge to lead clustering. In this paper, we expose the process to design a chatbot with the interactive clustering method.La création d'un jeu de données nécessaire à la conception d'un assistant conversationnel résulte le plus souvent d'une étape manuelle et fastidieuse qui manque de techniques destinées à l'assister. Pour accélérer cette étape d'annotation, nous proposons une méthode de clustering interactif : il s'agit d'une approche itérative inspirée de l'apprentissage actif, reposant sur un algorithme de clustering et tirant parti d'une annotation de contraintes pour guider le regroupement des questions en une structure d'intentions. Dans cet article, nous exposons la méthodologie à mettre en oeuvre pour concevoir un assistant conversationnel opérationnel à l'aide du clustering interactif

Conception itérative et semi-supervisée d'assistants conversationnels par regroupement interactif des questions

National audienceThe design of a dataset needed to train a chatbot is most often the result of manual and tedious step. To guarantee the efficiency and objectivity of the annotation, we propose an active learning method based on constraints annotation. It’s an iterative approach, relying on a clustering algorithm to segment data and using annotator knowledge to lead clustering from unlabeled question to relevant intents structure. In this paper, we study the optimal modeling parameters to get an exploitable dataset with a minimum of annotations, and show that this approach allows to make a coherent structure for the training of a chatbot.La création d’un jeu de données pour l’entrainement d’un chatbot repose sur un a priori de connaissance du domaine. En conséquence, cette étape est le plus souvent manuelle, fastidieuse et soumise aux biais. Pour garantir l’efficacité et l’objectivité de l’annotation, nous proposons une méthodologie d’apprentissage actif par annotation de contraintes. Il s’agit d’une approche itérative, reposant sur un algorithme de clustering pour segmenter les données et tirant parti de la connaissance de l’annotateur pour guider le regroupement des questions en une structure d’intentions. Dans cet article, nous étudions les paramètres optimaux de modélisation pour réaliser une segmentation exploitable en un minimum d’annotations, et montrons que cette approche permet d’aboutir à une structure cohérente pour l’entrainement d’un assistant conversationnel

Biocybernetic Adaptation Strategies: Machine awareness of human state for improved operational performance

Human operators interacting with machines or computers continually adapt to the needs of the system ideally resulting in optimal performance. In some cases, however, deteriorated performance is an outcome. Adaptation to the situation is a strength expected of the human operator which is often accomplished by the human through self-regulation of mental state. Adaptation is at the core of the human operator’s activity, and research has demonstrated that the implementation of a feedback loop can enhance this natural skill to improve training and human/machine interaction. Biocybernetic adaptation involves a “loop upon a loop,” which may be visualized as a superimposed loop which senses a physiological signal and influences the operator’s task at some point. Biocybernetic adaptation in, for example, physiologically adaptive automation employs the “steering” sense of “cybernetic,” and serves a transitory adaptive purpose – to better serve the human operator by more fully representing their responses to the system. The adaptation process usually makes use of an assessment of transient cognitive state to steer a functional aspect of a system that is external to the operator’s physiology from which the state assessment is derived. Therefore, the objective of this paper is to detail the structure of biocybernetic systems regarding the level of engagement of interest for adaptive systems, their processing pipeline, and the adaptation strategies employed for training purposes, in an effort to pave the way towards machine awareness of human state for self-regulation and improved operational performance