Mobile Thermography-based Physiological Computing for Automatic Recognition of a Person’s Mental Stress

This thesis explores the use of Mobile Thermography1, a significantly less investigated sensing capability, with the aim of reliably extracting a person’s multiple physiological signatures and recognising mental stress in an automatic, contactless manner. Mobile thermography has greater potentials for real-world applications because of its light-weight, low computation-cost characteristics. In addition, thermography itself does not necessarily require the sensors to be worn directly on the skin. It raises less privacy concerns and is less sensitive to ambient lighting conditions. The work presented in this thesis is structured through a three-stage approach that aims to address the following challenges: i) thermal image processing for mobile thermography in variable thermal range scenes; ii) creation of rich and robust physiology measurements; and iii) automated stress recognition based on such measurements. Through the first stage (Chapter 4), this thesis contributes new processing techniques to address negative effects of environmental temperature changes upon automatic tracking of regions-of-interest and measuring of surface temperature patterns. In the second stage (Chapters 5,6,7), the main contributions are: robustness in tracking respiratory and cardiovascular thermal signatures both in constrained and unconstrained settings (e.g. respiration: strong correlation with ground truth, r=0.9987), and investigation of novel cortical thermal signatures associated with mental stress. The final stage (Chapters 8,9) contributes automatic stress inference systems that focus on capturing richer dynamic information of physiological variability: firstly, a novel respiration representation-based system (which has achieved state-of-the-art performance: 84.59% accuracy, two stress levels), and secondly, a novel cardiovascular representation-based system using short-term measurements of nasal thermal variability and heartrate variability from another sensing channel (78.33% accuracy achieved from 20seconds measurements). Finally, this thesis contributes software libraries and incrementally built labelled datasets of thermal images in both constrained and everyday ubiquitous settings. These are used to evaluate performance of our proposed computational methods across the three-stages

Socio-Cognitive and Affective Computing

Social cognition focuses on how people process, store, and apply information about other people and social situations. It focuses on the role that cognitive processes play in social interactions. On the other hand, the term cognitive computing is generally used to refer to new hardware and/or software that mimics the functioning of the human brain and helps to improve human decision-making. In this sense, it is a type of computing with the goal of discovering more accurate models of how the human brain/mind senses, reasons, and responds to stimuli. Socio-Cognitive Computing should be understood as a set of theoretical interdisciplinary frameworks, methodologies, methods and hardware/software tools to model how the human brain mediates social interactions. In addition, Affective Computing is the study and development of systems and devices that can recognize, interpret, process, and simulate human affects, a fundamental aspect of socio-cognitive neuroscience. It is an interdisciplinary field spanning computer science, electrical engineering, psychology, and cognitive science. Physiological Computing is a category of technology in which electrophysiological data recorded directly from human activity are used to interface with a computing device. This technology becomes even more relevant when computing can be integrated pervasively in everyday life environments. Thus, Socio-Cognitive and Affective Computing systems should be able to adapt their behavior according to the Physiological Computing paradigm. This book integrates proposals from researchers who use signals from the brain and/or body to infer people's intentions and psychological state in smart computing systems. The design of this kind of systems combines knowledge and methods of ubiquitous and pervasive computing, as well as physiological data measurement and processing, with those of socio-cognitive and affective computing

Human-Computer Interaction

In this book the reader will find a collection of 31 papers presenting different facets of Human Computer Interaction, the result of research projects and experiments as well as new approaches to design user interfaces. The book is organized according to the following main topics in a sequential order: new interaction paradigms, multimodality, usability studies on several interaction mechanisms, human factors, universal design and development methodologies and tools

Machine Learning Models for Mental Stress Classification based on Multimodal Biosignal Input

Mental stress is a largely prevalent condition directly or indirectly responsible for almost half of all work-related diseases. Work-Related Stress is the second most impactful occupational health problem in Europe, behind musculoskeletal diseases. When mental health is adequately handled, a worker’s well-being, performance, and productivity can be considerably improved. This thesis presents machine learning models to classify mental stress experienced by computer users using physiological signals including heart rate, acquired using a smart- watch; respiration, derived from a smartphone’s acc placed on the chest; and trapezius electromyography, using proprietary electromyography sensors. Two interactive proto- cols were implemented to collect data from 12 individuals. Time and frequency domain features were extracted from the heart rate and electromyography signals, and statistical and temporal features were extracted from the derived respiration signal. Three algorithms: Support Vector Machine, Random Forest, and K-Nearest-Neighbor were employed for mental stress classification. Different input modalities were tested for the machine learning models: one for each physiological signal and a multimodal one, combining all of them. Random Forest obtained the best mean accuracy (98.5%) for the respiration model whereas K-Nearest-Neighbor attained higher mean accuracies for the heart rate (89.0%) left, right and total electromyography (98.9%, 99.2%, and 99.3%) models. KNN algorithm was also able to achieve 100% mean accuracy for the multimodal model. A possible future approach would be to validate these models in real-time.O stress mental é uma condição amplamente prevalente direta ou indiretamente responsável por quase metade de todas doenças relacionadas com trabalho. O stress expe- rienciado no trabalho é o segundo problema de saúde ocupacional com maior impacto na Europa, depois das doenças músculo-esqueléticas. Quando a saúde mental é adequada- mente cuidada, o bem-estar, o desempenho e a produtividade de um trabalhador podem ser consideravelmente melhorados. Esta tese apresenta modelos de aprendizagem automática que classificam o stress mental experienciado por utilizadores de computadores recorrendo a sinais fisiológi- cos, incluindo a frequência cardíaca, adquirida pelo sensor de fotopletismografia de um smartwatch; a respiração, derivada de um acelerómetro incorporado no smartphone po- sicionado no peito; e electromiografia de cada um dos músculos trapézios, utilizando sensores electromiográficos proprietários. Foram implementados dois protocolos inte- ractivos para recolha de dados de 12 indivíduos. Características do domínio temporal e de frequência foram extraídas dos sinais de frequência cardíaca e electromiografia, e características estatísticas e temporais foram extraídas do sinal respiratório. Três algoritmos entitulados K-Nearest-Neighbor, Random Forest, e Support Vector Machine foram utilizados para a classificação do stress mental. Foram testadas diferentes modalidades de dados para os modelos de aprendizagem automática: uma para cada sinal fisiológico e uma multimodal, combinando os três. O Random Forest obteve a melhor precisão média (98,5%) para o modelo de respiração enquanto que o K-Nearest-Neighbor atingiu uma maior precisão média nos modelos de frequência cardíaca (89,0%) e electro- miografia esquerda, direita e total (98,9%, 99,2%, e 99,3%). O algoritmo KNN conseguiu ainda atingir uma precisão média de 100% para o modelo multimodal. Uma possível abordagem futura seria efetuar uma validação destes modelos em tempo real

Wearable and BAN Sensors for Physical Rehabilitation and eHealth Architectures

The demographic shift of the population towards an increase in the number of elderly citizens, together with the sedentary lifestyle we are adopting, is reflected in the increasingly debilitated physical health of the population. The resulting physical impairments require rehabilitation therapies which may be assisted by the use of wearable sensors or body area network sensors (BANs). The use of novel technology for medical therapies can also contribute to reducing the costs in healthcare systems and decrease patient overflow in medical centers. Sensors are the primary enablers of any wearable medical device, with a central role in eHealth architectures. The accuracy of the acquired data depends on the sensors; hence, when considering wearable and BAN sensing integration, they must be proven to be accurate and reliable solutions. This book is a collection of works focusing on the current state-of-the-art of BANs and wearable sensing devices for physical rehabilitation of impaired or debilitated citizens. The manuscripts that compose this book report on the advances in the research related to different sensing technologies (optical or electronic) and body area network sensors (BANs), their design and implementation, advanced signal processing techniques, and the application of these technologies in areas such as physical rehabilitation, robotics, medical diagnostics, and therapy

Cumulative trauma disorders in the workplace: bibliography

"This publication provided a compilation of materials describing research conducted by NIOSH on cumulative trauma disorders in the workplace. Selected references, both NIOSH and nonNIOSH, were provided, concentrating on NIOSH activities in preventing work related musculoskeletal disorders, prevention and intervention research at NIOSH for work related musculoskeletal disorders, comments to the Department of Labor on the OSHA proposed rule on ergonomic safety and health management, a manual for musculoskeletal diseases of the upper limbs, a review of physical exercises recommended for video display tube operators, management of upper extremity cumulative trauma disorders, ergonomics and prevention of musculoskeletal injuries, and carpal tunnel syndrome. A bibliography of NIOSH publications on cumulative trauma disorders in the workplace was provided, including numbered publications, testimony, journal articles, grant reports, contract reports, and health hazard evaluations. NonNiosh references were also listed." - NIOSHTIC-2Part I. Selected NIOSH and non-NIOSH References -- A. NIOSH Activities in Preventing Work-Related Musculoskeletal Disorders -- B. Work-Related Musculoskeletal Disorders: Prevention and Intervention Research at NIOSH -- C. Comments to DOL on the Occupational Safety and Health Administration Proposed Role on Ergonomic Safety and Health Management - Part 1 -- C. Comments to DOL on the Occupational Safety and Health Administration Proposed Role on Ergonomic Safety and Health Management - Part 2 -- D. Cumulative Trauma Disorders: A Manual for Musculoskeletal Diseases of the Upper Limbs -- E. A Review of Physical Exercises Recommended for VDT Operators ) -- F. Management of Upper Extremity Cumulative Trauma Disorders -- G. Preventing Illness and Injury in the Workplace: Ergonomics and Prevention of Musculoskeletal Injuries -- H. Carpal Tunnel Syndrome -- -- Part II. Cumulative Trauma Disorders in the Workplace - Bibliography -- A. NIOSH Publications Reports -- 1. Numbered Publications -- 2. Testimony -- 3. Journal Articles -- 4. Grant Reports -- 5. Contract Reports -- 6. Health Hazard Evaluations -- -- B. Selected non-NIOSH ReferencesAlso available via the World Wide Web

Implementing Industry 4.0 in SMEs

This open access book addresses the practical challenges that Industry 4.0 presents for SMEs. While large companies are already responding to the changes resulting from the fourth industrial revolution , small businesses are in danger of falling behind due to the lack of examples, best practices and established methods and tools. Following on from the publication of the previous book ‘Industry 4.0 for SMEs: Challenges, Opportunities and Requirements’, the authors offer in this new book innovative results from research on smart manufacturing, smart logistics and managerial models for SMEs. Based on a large scale EU-funded research project involving seven academic institutions from three continents and a network of over fifty small and medium sized enterprises, the book reveals the methods and tools required to support the successful implementation of Industry 4.0 along with practical examples

Innovative Technologies and Services for Smart Cities

A smart city is a modern technology-driven urban area which uses sensing devices, information, and communication technology connected to the internet of things (IoTs) for the optimum and efficient utilization of infrastructures and services with the goal of improving the living conditions of citizens. Increasing populations, lower budgets, limited resources, and compatibility of the upgraded technologies are some of the few problems affecting the implementation of smart cities. Hence, there is continuous advancement regarding technologies for the implementation of smart cities. The aim of this Special Issue is to report on the design and development of integrated/smart sensors, a universal interfacing platform, along with the IoT framework, extending it to next-generation communication networks for monitoring parameters of interest with the goal of achieving smart cities. The proposed universal interfacing platform with the IoT framework will solve many challenging issues and significantly boost the growth of IoT-related applications, not just in the environmental monitoring domain but in the other key areas, such as smart home, assistive technology for the elderly care, smart city with smart waste management, smart E-metering, smart water supply, intelligent traffic control, smart grid, remote healthcare applications, etc., signifying benefits for all countries