Probing ECG-based mental state monitoring on short time segments

Electrocardiography is used to provide features for mental state monitoring systems. There is a need for quick mental state assessment in some applications such as attentive user interfaces. We analyzed how heart rate and heart rate variability features are influenced by working memory load (WKL) and time-on-task (TOT) on very short time segments (5s) with both statistical significance and classification performance results. It is shown that classification of such mental states can be performed on very short time segments and that heart rate is more predictive of TOT level than heart rate variability. However, both features are efficient for WKL level classification. What's more, interesting interaction effects are uncovered: TOT influences WKL level classification either favorably when based on HR, or adversely when based on HRV. Implications for mental state monitoring are discussed

Eye blink characterization from frontal EEG electrodes using source separation and pattern recognition algorithms

Due to its major safety applications, including safe driving, mental fatigue estimation is a rapidly growing research topic in the engineering field. Most current mental fatigue monitoring systems analyze brain activity through electroencephalography (EEG). Yet eye blink analysis can also be added to help characterize fatigue states. It usually requires the use of additional devices, such as EOG electrodes, uncomfortable to wear, or more expensive eye trackers. However, in this article, a method is proposed to evaluate eye blink parameters using frontal EEG electrodes only. EEG signals, which are generally corrupted by ocular artifacts, are decomposed into sources by means of a source separation algorithm. Sources are then automatically classified into ocular or non-ocular sources using temporal, spatial and frequency features. The selected ocular source is back propagated in the signal space and used to localize blinks by means of an adaptive threshold, and then to characterize detected blinks. The method, validated on 11 different subjects, does not require any prior tuning when applied to a new subject, which makes it subject-independent. The vertical EOG signal was recorded during an experiment lasting 90 min in which the participants’ mental fatigue increased. The blinks extracted from this signal were compared to those extracted using frontal EEG electrodes. Very good performances were obtained with a true detection rate of 89% and a false alarm rate of 3%. The correlation between the blink parameters extracted from both recording modalities was 0.81 in average

Mental fatigue and working memory load estimation: Interaction and implications for EEG-based passive BCI

Current mental state monitoring systems, a.k.a. passive brain-computer interfaces (pBCI), allow one to perform a real-time assessment of an operator's cognitive state. In EEG-based systems, typical measurements for workload level assessment are band power estimates in several frequency bands. Mental fatigue, arising from growing time-on-task (TOT), can significantly affect the distribution of these band power features. However, the impact of mental fatigue on workload (WKL) assessment has not yet been evaluated. With this paper we intend to help fill in this lack of knowledge by analyzing the influence of WKL and TOT on EEG band power features, as well as their interaction and its impact on classification performance. Twenty participants underwent an experiment that modulated both their WKL (low/high) and time spent on the task (short/long). Statistical analyses were performed on the EEG signals, behavioral and subjective data. They revealed opposite changes in alpha power distribution between WKL and TOT conditions, as well as a decrease in WKL level discriminability with increasing TOT in both number of statistical differences in band power and classification performance. Implications for pBCI systems and experimental protocol design are discussed

Efficient mental workload estimation using task-independent EEG features

Objective. Mental workload is frequently estimated by EEG-based mental state monitoring systems. Usually, these systems use spectral markers and event-related potentials ( ERPs ) . To our knowledge, no study has directly compared their performance for mental workload assessment, nor evaluated the stability in time of these markers and of the performance of the associated mental workload estimators. This study proposes a comparison of two processing chains, one based on the power in fi ve frequency bands, and one based on ERPs, both including a spatial filtering step ( respectively CSP and CCA ) , an FLDA classification and a 10-fold cross-validation. Approach. To get closer to a real life implementation, spectral markers were extracted from a short window ( i.e. towards reactive systems ) that did not include any motor activity and the analyzed ERPs were elicited by a task-independent probe that required a re fl ex-like answer ( i.e. close to the ones required by dead man ’ s vigilance devices ) . The data were acquired from 20 participants who performed a Sternberg memory task for 90 min ( i.e. 2 / 6 digits to memorize) inside which a simple detection task was inserted. The results were compared both when the testing was performed at the beginning and end of the session. Main results. Both chains performed significantly better than random; however the one based on the spectral markers had a low performance ( 60% ) and was not stable in time. Conversely, the ERP-based chain gave very high results ( 91% ) and was stable in time. Significance. This study demonstrates that an efficient and stable in time workload estimation can be achieved using task-independent spatially filtered ERPs elicited in a minimally intrusive manner

Efficient Workload Classification based on Ignored Auditory Probes: A Proof of Concept

Mental workload is a mental state that is currently one of the main research focuses in neuroergonomics. It can notably be estimated using measurements in electroencephalography (EEG), a method that allows for direct mental state assessment. Auditory probes can be used to elicit event-related potentials (ERPs) that are modulated by workload. Although, some papers do report ERP modulations due to workload using attended or ignored probes, to our knowledge there is no literature regarding effective workload classification based on ignored auditory probes. In this paper, in order to efficiently estimate workload, we advocate for the use of such ignored auditory probes in a single-stimulus paradigm and a signal processing chain that includes a spatial filtering step. The effectiveness of this approach is demonstrated on data acquired from participants that performed the Multi-Attribute Task Battery – II. They carried out this task during two 10-min blocks. Each block corresponded to a workload condition that was pseudorandomly assigned. The easy condition consisted of two monitoring tasks performed in parallel, and the difficult one consisted of those two tasks with an additional plane driving task. Infrequent auditory probes were presented during the tasks and the participants were asked to ignore them. The EEG data were denoised and the probes’ ERPs were extracted and spatially filtered using a canonical correlation analysis. Next, binary classification was performed using a Fisher LDA and a fivefold cross-validation procedure. Our method allowed for a very high estimation performance with a classification accuracy above 80% for every participant, and minimal intrusiveness thanks to the use of a single-stimulus paradigm. Therefore, this study paves the way to the efficient use of ERPs for mental state monitoring in close to real-life settings and contributes toward the development of adaptive user interfaces

EEG index for control operators’ mental fatigue monitoring using interactions between brain regions

Mental fatigue is a gradual and cumulative phenomenon induced by the time spent on a tedious but mentally demanding task, which is associated with a decrease in vigilance. It may be dangerous for operators controlling air traffic or monitoring plants. An index that estimates this state on-line from EEG signals recorded in 6 brain regions is proposed. It makes use of the Frobenius distance between the EEG spatial covariance matrices of each of the 6 regions calculated on 20s epochs to a mean covariance matrix learned during an initial reference state. The index is automatically tuned from the learning set for each subject. Its performance is analyzed on data from a group of 15 subjects who performed for 90 min an experiment that modulates mental workload. It is shown that the index based on the alpha band is well correlated with an ocular index that measures external signs of mental fatigue and can accurately assess mental fatigue over long periods of time

Estimation of Working Memory Load using EEG Connectivity Measures

Working memory load can be estimated using features extracted from the electroencephalogram (EEG). Connectivity measures, that evaluate the interaction between signals, can be used to extract such features and therefore provide information about the interconnection of brain areas and electrode sites. To our knowledge, there is no literature regarding a direct comparison of the relevance of several connectivity measures for working memory load estimation. This study intends to overcome this lack of literature by proposing a direct comparison of four connectivity measures on data extracted from a working memory load experiment performed by 20 participants. These features are extracted using pattern-based or vector-based methods, and classified using an FLDA classifier and a 10-fold cross-validation procedure. The relevance of the connectivity measures was assessed by statistically comparing the obtained classification accuracy. Additional investigations were performed regarding the best set of electrodes and the best frequency band. The main results are that covariance seems to be the best connectivity measure to estimate working memory load from EEG signals, even more so with signals filtered in the beta band. point

Influence of Workload on Auditory Evoked Potentials in a Single-stimulus Paradigm

Mental workload can be assessed via neurophysiological markers. Temporal features such as event related potentials (ERPs) are one of those which are very often described in the literature. However, most of the studies that evaluate their sensitivity to workload use secondary tasks. Yet potentials elicited by ignored stimuli could provide mental state monitoring systems with less intrusive probing methods. For instance, auditory probing systems could be used in adaptive driving or e-learning applications. This study evaluates how workload influences auditory evoked potentials (AEPs) elicited by a single-stimulus paradigm when probes are to be ignored. Ten participants performed a Sternberg memory task on a touchpad with three levels of difficulty plus a view-only condition. In addition, they performed two ecological tasks of their choice, one deemed easy (e.g. reading novels), and the other difficult (e.g. programming). AEPs were elicited thanks to pure tones presented during the memory task retention period, and during the whole extent of the external tasks. Performance and AEPs were recorded and analyzed. Participants’ accuracy decreased linearly with increasing workload, whereas the difference in amplitude between the P3 and its adjacent components, N2 and SW, increased. This reveals the relevance of this triphasic sequence for mental workload assessment

SlideImages: A Dataset for Educational Image Classification

In the past few years, convolutional neural networks (CNNs) have achieved impressive results in computer vision tasks, which however mainly focus on photos with natural scene content. Besides, non-sensor derived images such as illustrations, data visualizations, figures, etc. are typically used to convey complex information or to explore large datasets. However, this kind of images has received little attention in computer vision. CNNs and similar techniques use large volumes of training data. Currently, many document analysis systems are trained in part on scene images due to the lack of large datasets of educational image data. In this paper, we address this issue and present SlideImages, a dataset for the task of classifying educational illustrations. SlideImages contains training data collected from various sources, e.g., Wikimedia Commons and the AI2D dataset, and test data collected from educational slides. We have reserved all the actual educational images as a test dataset in order to ensure that the approaches using this dataset generalize well to new educational images, and potentially other domains. Furthermore, we present a baseline system using a standard deep neural architecture and discuss dealing with the challenge of limited training data.Comment: 8 pages, 2 figures, to be presented at ECIR 202

Biodiversity conservation, ecosystem services and organic viticulture: A glass half-full

Organic farming is a promising but still debated option to ensure sustainable agriculture. However, whether organic farming fosters synergies or mitigates tradeoffs between biodiversity, ecosystem services and crop production has rarely been quantified. Here, we investigate relationships between multitrophic diversity (14 taxa above and belowground), yield, natural pest control and soil quality (14 proxies of ecosystem services) in organic and conventional vineyards along a landscape gradient. Organic farming enhanced biodiversity and pest control, but decreased wine production. Compared to conventional systems, multitrophic diversity was 15 % higher, and pest control services were 9 % higher in organic systems, while wine production was 11 % lower. Regardless of management type, we found a strong tradeoff between wine production and pest control, but not between wine production and biodiversity. The landscape context was not a strong moderator of organic farming effects across taxa groups and ecosystem services, but affected specific taxa and ecosystem services, especially natural pest control. Our study reveals that wine production and biodiversity conservation do not necessarily exclude each other, which implies the existence of a safe operating space where biodiversity and wine production can be combined. We conclude that organic farming can contribute to improve the sustainability of viticulture, but needs to be complemented by management options at the local and landscape scales in order to fully balance biodiversity conservation with the simultaneous provision of multiple ecosystem services.This research was funded by the research project SECBIVIT, which was funded through the 2017–2018 Belmont Forum and BiodivERsA joint call for research proposals, under the BiodivScen ERA-Net COFUND program, with the funding organizations: Agencia Estatal de Investigación (Ministerio de Ciencia e Innovación/Spain, grant PCI2018-092938; MCIN/AEI/10.13039/501100011033); Austrian Science Fund (FWF) (grant number I 4025-B32); Federal Ministry of Education and Research (BMBF/Germany) (grant number 031A349I); French National Research Agency (ANR); Netherlands Organization for Scientific Research (NWO); National Science Foundation (grant #1850943); and Romanian Executive Agency for Higher Education, Research, Development, and Innovation Funding (UEFISCDI). The authors also acknowledge the support of the ECOPHYTO 2+ Plan under the grant X4IN33VI (OPERA project) as well as the support the French National Research Agency (ANR) under the grant 20-PCPA-0010 (PPR Vitae, Cultivating the grapevine without pesticides: towards agroecological wine-producing socio-ecosystems). We thank Evelyne Thys and Hugo Hernandez for their help in field sampling, Lionel Delbac for the Lobesia botrana rearing, Alexis Saintilan for identifying pollinators, and Edith Gruber for identifying earthworms