Unobtrusive Health Monitoring in Private Spaces: The Smart Vehicle

Unobtrusive in-vehicle health monitoring has the potential to use the driving time to perform regular medical check-ups. This work intends to provide a guide to currently proposed sensor systems for in-vehicle monitoring and to answer, in particular, the questions: (1) Which sensors are suitable for in-vehicle data collection? (2) Where should the sensors be placed? (3) Which biosignals or vital signs can be monitored in the vehicle? (4) Which purposes can be supported with the health data? We reviewed retrospective literature systematically and summarized the up-to-date research on leveraging sensor technology for unobtrusive in-vehicle health monitoring. PubMed, IEEE Xplore, and Scopus delivered 959 articles. We firstly screened titles and abstracts for relevance. Thereafter, we assessed the entire articles. Finally, 46 papers were included and analyzed. A guide is provided to the currently proposed sensor systems. Through this guide, potential sensor information can be derived from the biomedical data needed for respective purposes. The suggested locations for the corresponding sensors are also linked. Fifteen types of sensors were found. Driver-centered locations, such as steering wheel, car seat, and windscreen, are frequently used for mounting unobtrusive sensors, through which some typical biosignals like heart rate and respiration rate are measured. To date, most research focuses on sensor technology development, and most application-driven research aims at driving safety. Health-oriented research on the medical use of sensor-derived physiological parameters is still of interest

運転者の心理状態に関する研究 ～センサーネットワークを用いた運転者の疲労度推定～

宇都宮大学博士（工学）2023Text学位論文 / Thesis or Dissertationdoctoral thesi

Multimodal Features for Detection of Driver Stress and Fatigue: Review

Driver fatigue and stress significantly contribute to higher number of car accidents worldwide. Although, different detection approaches have been already commercialized and used by car producers (and third party companies), research activities in this field are still needed in order to increase the reliability of these alert systems. Also, in the context of automated driving, the driver mental state assessment will be an important part of cars in future. This paper presents state-of-the-art review of different approaches for driver fatigue and stress detection and evaluation. We describe in details various signals (biological, car and video) and derived features used for these tasks and we discuss their relevance and advantages. In order to make this review complete, we also describe different datasets, acquisition systems and experiment scenarios

Applying Functional Data Analysis to Estimate the Mental Workload of the Human Driver

Recent studies have focused on accurately estimating mental workload using machine learning algorithms and extracting features from psycho-physiological measures. However, feature extraction leads to the loss of valuable information and often results in binary classifications that lack specificity in the identification of optimum mental workload levels. This study investigates the feasibility of using raw psycho-physiological data (EEG, facial EMG, ECG, EDA, pupillometry) combined with Functional Data Analysis (FDA) to estimate mental workload of human drivers. A driving scenario with five tasks was employed, and subjective ratings were collected using the modified Bedford workload scale. Results demonstrate that the FDA applied to nine different combinations of raw psycho-physiological signals achieved a maximum 90% accuracy, outperforming extracted features by 73%. This study shows that mental workload of human drivers can be accurately estimated without utilizing the burdensome feature extraction. The approach proposed in this study offers promise for mental workload assessment in real-world applications

Brain Tumour Classification Using Deep Features from Structural MRI

C

Human-Computer Interaction and Biosignals: Evaluating Human Learning Processes

In recent years, the widespread adoption of e-learning has led to a surge in solutions for delivering and accessing educational content online. However, the challenge of monitoring learners’ cognitive states in real-time becomes pronounced in distance-learning scenarios where tutors are absent. Suboptimal cognitive states can impede the learning process, prompting the need for innovative approaches to incorporate cognitive monitoring in e-learning environments. In this sense, this work developed 3 different studies: 1) attention classification using biosignals and machine learning; 2) cognitive fatigue detection using functional near- infrared spectroscopy (fNIRS); 3) complex learning states (Neutral, Interest/Flow, Surprise, Boredom, Distraction, Confusion, Eureka, and Frustration) classification using both a myriad of biosignals and Human-Computer Interaction (HCI) metrics. In each of those, different cognitive tasks to monitor the distinct states were applied: 1) N-Back, Mental Subtraction, and a programming language lesson to detect attention; 2) Corsi-Block task, a concentration task, and a lesson about the electrocardiogram (ECG) that helped to elicit cognitive fatigue; 3) a modified version of the ECG lesson combined with a self-reporting stage to classify the complex learning states reported by the participants. In the three studies, six different biosignals were monitored - fNIRS, electroencephalogram (EEG), respiratory inductance plethysmography (RIP), ECG, electrodermal activity (EDA), and an accelerometer - as well as HCI metrics, namely, mouse-tracking. Attention and cognitive fatigue detection using machine learning was possible with some restrictions. Personalized user-tuned classifiers were required to properly detect those cognitive states due to individual differences between participants. Furthermore, different biosignals and combinations were revealed to be suited differently for each participant. Analogously, the classification of the complex learning states detection in the third study was studied under various conditions, where the user-tuned classification of the gathered similar states was revealed to be the best approach. In this case, the mouse-tracking features were undoubtedly the best-performing features, reaching an F1-Score of 0.87 in this task.This work and the presented results demonstrate that cognitive and learning monitoring is possible with minimal intrusion in e-learning contexts, allowing the development of solutions that integrate them and adjust to users in real-time.Nos últimos anos, a ampla adoção do ensino eletrónico (e-learning) tem gerado um au- mento do número de soluções para a divulgação e acesso de conteúdo educacional online. No entanto, o desafio de monitorizar em tempo real os estados cognitivos dos alunos torna-se evidente em cenários de ensino à distância, onde os tutores estão ausentes. Esta- dos cognitivos subótimos podem prejudicar o processo de aprendizagem, evidenciando a necessidade de abordagens inovadoras para incorporar monitorização cognitiva em ambientes de e-learning. Desta forma, neste trabalho desenvolvemos três estudos distintos: 1) classificação de atenção utilizando biossinais e aprendizagem automática; 2) deteção de fadiga cognitiva usando espectroscopia funcional de infravermelho próximo (fNIRS); 3) classificação de estados complexos de aprendizagem (Neutro, Interesse, Surpresa, Aborrecimento, Distra- cão, Confusão, Eureca e Frustração) utilizando uma variedade de biossinais e métricas de Interação Humano-Computador (HCI). Em cada um deles, aplicámos diferentes tarefas cognitivas para monitorizar os estados distintos: 1) Tarefa N-Back, Subtração Mental e uma lição sobre uma linguagem de programação para detetar atenção; 2) tarefa Corsi-Block, uma tarefa de concentração e uma lição sobre o sinal de eletrocardiograma (ECG) que ajudou a elicitar fadiga cognitiva; 3) uma versão modificada da lição de ECG combinada com uma etapa de auto-análise para classificar os estados complexos de aprendizagem relatados pelos participantes. Nos três estudos, monitorizámos seis biossinais diferen- tes - fNIRS, eletroencefalograma (EEG), pletismografia de impedância respiratória (RIP), ECG, atividade eletrodérmica (EDA) e um acelerometria - além de métricas de HCI, nomeadamente, movimentos do rato. A deteção de atenção e fadiga cognitiva utilizando aprendizagem automática foi pos- sível com algumas restrições. Verificámos que classificadores personalizados ajustados ao utilizador eram necessários para detetar adequadamente esses estados cognitivos de- vido às diferenças individuais entre os participantes. Além disso, diferentes biossinais e combinações mostraram ser mais adequados de maneiras distintas para cada participante. Analogamente, estudámos a classificação de estados complexos de aprendizagem no ter- ceiro estudo sob várias condições, onde a classificação ajustada ao utilizador da agregação de estados semelhantes mostrou ser a melhor abordagem. Nesse caso, as características do movimento do rato foram indiscutivelmente as mais eficazes, alcançando um F1-Score de 0.87 nessa tarefa. Este trabalho e os resultados apresentados demonstram que a monitorização cognitiva e de aprendizagem é possível com um mínimo de intrusão em contextos de e-learning, permitindo o desenvolvimento de soluções que os integrem e se ajustem aos utilizadores em tempo real