Attention, concentration, and distraction measure using EEG and eye tracking in virtual reality

Attention is important in learning, Attention-deficit/hyperactivity disorder, Driving, and many other fields. Hence, intelligent tutoring systems, Attention-deficit/hyperactivity disorder diagnosis systems, and distraction detection of driver systems should be able to correctly monitor the attention levels of individuals in real time in order to estimate their attentional state. We study the feasibility of detecting distraction and concentration by monitoring participants' attention levels while they complete cognitive tasks using Electroencephalography and Eye Tracking in a virtual reality environment. Furthermore, we investigate the possibility of improving the concentration of participants using relaxation in virtual reality. We developed an indicator that estimates levels of attention with a real value using EEG data. The participant-independent indicator based on EEG data we used to assess the concentration levels of participants correctly predicts the concentration state with an accuracy (F1 = 73%). Furthermore, the participant-independent distraction model based on Eye Tracking data correctly predicted the distraction state of participants with an accuracy (F1 = 89%) in a participant-independent validation setting.La concentration est importante dans l’apprentissage, Le trouble du déficit de l’attention avec ou sans hyperactivité, la conduite automobile et dans de nombreux autres domaines. Par conséquent, les systèmes de tutorat intelligents, les systèmes de diagnostic du trouble du déficit de l’attention avec ou sans hyperactivité et les systèmes de détection de la distraction au volant devraient être capables de surveiller correctement les niveaux d’attention des individus en temps réel afin de déduire correctement leur état attentionnel. Nous étudions la faisabilité de la détection de la distraction et de la concentration en surveillant les niveaux d’attention des participants pendant qu’ils effectuent des tâches cognitives en utilisant l’Électroencéphalographie et l’Eye Tracking dans un environnement de réalité virtuelle. En outre, nous étudions la possibilité d’améliorer la concentration des participants en utilisant la relaxation en réalité virtuelle. Nous avons mis au point un indicateur qui estime les niveaux d’attention avec une valeur réelle en utilisant les données EEG. L’indicateur indépendant du participant basé sur les données EEG que nous avons utilisé pour évaluer les niveaux de concentration des participants prédit correctement l’état de concentration avec une précision (F1 = 73%). De plus, le modèle de distraction indépendant des participants, basé sur les données d’Eye Tracking, a correctement prédit l’état de distraction des participants avec une précision (F1 = 89%) dans un cadre de validation indépendant des participants

Emotions in context: examining pervasive affective sensing systems, applications, and analyses

Pervasive sensing has opened up new opportunities for measuring our feelings and understanding our behavior by monitoring our affective states while mobile. This review paper surveys pervasive affect sensing by examining and considering three major elements of affective pervasive systems, namely; “sensing”, “analysis”, and “application”. Sensing investigates the different sensing modalities that are used in existing real-time affective applications, Analysis explores different approaches to emotion recognition and visualization based on different types of collected data, and Application investigates different leading areas of affective applications. For each of the three aspects, the paper includes an extensive survey of the literature and finally outlines some of challenges and future research opportunities of affective sensing in the context of pervasive computing

Machine Learning Models for Mental Stress Classification based on Multimodal Biosignal Input

Mental stress is a largely prevalent condition directly or indirectly responsible for almost half of all work-related diseases. Work-Related Stress is the second most impactful occupational health problem in Europe, behind musculoskeletal diseases. When mental health is adequately handled, a worker’s well-being, performance, and productivity can be considerably improved. This thesis presents machine learning models to classify mental stress experienced by computer users using physiological signals including heart rate, acquired using a smart- watch; respiration, derived from a smartphone’s acc placed on the chest; and trapezius electromyography, using proprietary electromyography sensors. Two interactive proto- cols were implemented to collect data from 12 individuals. Time and frequency domain features were extracted from the heart rate and electromyography signals, and statistical and temporal features were extracted from the derived respiration signal. Three algorithms: Support Vector Machine, Random Forest, and K-Nearest-Neighbor were employed for mental stress classification. Different input modalities were tested for the machine learning models: one for each physiological signal and a multimodal one, combining all of them. Random Forest obtained the best mean accuracy (98.5%) for the respiration model whereas K-Nearest-Neighbor attained higher mean accuracies for the heart rate (89.0%) left, right and total electromyography (98.9%, 99.2%, and 99.3%) models. KNN algorithm was also able to achieve 100% mean accuracy for the multimodal model. A possible future approach would be to validate these models in real-time.O stress mental é uma condição amplamente prevalente direta ou indiretamente responsável por quase metade de todas doenças relacionadas com trabalho. O stress expe- rienciado no trabalho é o segundo problema de saúde ocupacional com maior impacto na Europa, depois das doenças músculo-esqueléticas. Quando a saúde mental é adequada- mente cuidada, o bem-estar, o desempenho e a produtividade de um trabalhador podem ser consideravelmente melhorados. Esta tese apresenta modelos de aprendizagem automática que classificam o stress mental experienciado por utilizadores de computadores recorrendo a sinais fisiológi- cos, incluindo a frequência cardíaca, adquirida pelo sensor de fotopletismografia de um smartwatch; a respiração, derivada de um acelerómetro incorporado no smartphone po- sicionado no peito; e electromiografia de cada um dos músculos trapézios, utilizando sensores electromiográficos proprietários. Foram implementados dois protocolos inte- ractivos para recolha de dados de 12 indivíduos. Características do domínio temporal e de frequência foram extraídas dos sinais de frequência cardíaca e electromiografia, e características estatísticas e temporais foram extraídas do sinal respiratório. Três algoritmos entitulados K-Nearest-Neighbor, Random Forest, e Support Vector Machine foram utilizados para a classificação do stress mental. Foram testadas diferentes modalidades de dados para os modelos de aprendizagem automática: uma para cada sinal fisiológico e uma multimodal, combinando os três. O Random Forest obteve a melhor precisão média (98,5%) para o modelo de respiração enquanto que o K-Nearest-Neighbor atingiu uma maior precisão média nos modelos de frequência cardíaca (89,0%) e electro- miografia esquerda, direita e total (98,9%, 99,2%, e 99,3%). O algoritmo KNN conseguiu ainda atingir uma precisão média de 100% para o modelo multimodal. Uma possível abordagem futura seria efetuar uma validação destes modelos em tempo real