HRMobile: A lightweight, local architecture for heart rate measurement

Heart rate and heart rate variability (HRV) are important metrics in the study of numerous physical and psychiatric conditions. Previously, measurement of heart rate was relegated to clinical settings, and was neither convenient nor captured a patient’s typical resting state. In effect, this made gathering heart rate data costly and introduced noise. The current prevalence of mobile phone technology and Internet access has increased the viability of remote health monitoring, thus presenting an opportunity to substantially improve the speed, convenience, and reliability of heart rate readings. Recent attention has focused on different methods for remote, non-contact heart rate measurement. Of these methods, video presents perhaps the best option for optimizing cost and convenience. This thesis introduces a lightweight architecture for estimating heart rate and HRV using a smartphone camera. The system presented here runs locally on a smartphone, requiring only a phone camera and 15s or more of continuous video of a subject’s face. No Internet connection or networking is necessary. Building the system to run locally in this manner means that this software confers benefits such as greater user privacy, offline availability, reliability, cost effectiveness, and speed. However, it also introduces added constraints on computational complexity. With these tradeoffs in mind, the system presented here is implemented within an Android mobile app. The performance of our approach fell short of that of existing state-of-the-art methods in terms of mean absolute error (MAE) of heart rate estimation, achieving MAE during validation that was over $17x$ greater than other existing approaches. There are a number of factors which may contribute to this performance discrepancy, including limitations in the diversity of the data used with respect to gender, age, skin tone, and heart rate intensity. Further, remote photoplethysmographic (rPPG) signal generated by this architecture contains a large number of noise artifacts which are difficult to consistently remove through signal processing. This noise is the primary reason for the underperformance of this architecture, and could potentially be explained by model and feature engineering decisions which were made to address the risk of overfitting on the limited dataset used in this work

Smart technologies and beyond: exploring how a smart band can assist in monitoring children’s independent mobility & well-being

The problem which is being investigated through this thesis is not having a device(s) or method(s) which are appropriate for monitoring a child’s vital and tracking a child’s location. This aspect is being explored by other researchers which are yet to find a viable solution. This work focuses on providing a solution that would consider using the Internet of Things for measuring and improving children’s health. Additionally, the focus of this research is on the use of technology for health and the needs of parents who are concerned about their child’s physical health and well-being. This work also provides an insight into how technology is used during the pandemic. This thesis will be based on a mixture of quantitative and qualitative research, which will have been used to review the following areas covering key aspects and focuses of this study which are (i) Children’s Independent Mobility (ii) Physical activity for children (iii) Emotions of a child (iv) Smart Technologies and (v) Children’s smart wearables. This will allow a review of the problem in detail and how technology can help the health sector, especially for children. The deliverable of this study is to recommend a suitable smart band device that enables location tracking of the child, activity tracking as well as monitoring the health and wellbeing of the child. The research also includes an element of practical research in the form of (i) Surveys, the use of smart technology and a perspective on the solution from parents. (ii) Focus group, in the form of a survey allowing opinions and collection of information on the child and what the parents think of smart technology and how it could potentially help with their fears. (iii) Observation, which allows the collection of data from children who were given six activities to conduct while wearing the Fitbit Charge HR. The information gained from these elements will help provide guidelines for a proposed solution. In this thesis, there are three frameworks which are about (i) Research process for this study (ii) Key Performance Indicators (KPIs) which are findings from the literature review and (iii) Proposed framework for the solution, all three combined frameworks can help health professionals and many parents who want an efficient and reliable device, also deployment of technologies used in the health industry for children in support of independent mobility. Current frameworks have some considerations within the technology and medical field but were not up to date with the latest elements such as parents fears within today’s world and the advanced features of technology

Heart rates estimation using rPPG methods in challenging imaging conditions

Abstract. The cardiovascular system plays a crucial role in maintaining the body’s equilibrium by regulating blood flow and oxygen supply to different organs and tissues. While contact-based techniques like electrocardiography and photoplethysmography are commonly used in healthcare and clinical monitoring, they are not practical for everyday use due to their skin contact requirements. Therefore, non-contact alternatives like remote photoplethysmography (rPPG) have gained significant attention in recent years. However, extracting accurate heart rate information from rPPG signals under challenging imaging conditions, such as image degradation and occlusion, remains a significant challenge. Therefore, this thesis aims to investigate the effectiveness of rPPG methods in extracting heart rate information from rPPG signals in these imaging conditions. It evaluates the effectiveness of both traditional rPPG approaches and rPPG pre-trained deep learning models in the presence of real-world image transformations, such as occlusion of the faces by sunglasses or facemasks, as well as image degradation caused by noise artifacts and motion blur. The study also explores various image restoration techniques to enhance the performance of the selected rPPG methods and experiments with various fine-tuning methods of the best-performing pre-trained model. The research was conducted on three databases, namely UBFC-rPPG, UCLA-rPPG, and UBFC-Phys, and includes comprehensive experiments. The results of this study offer valuable insights into the efficacy of rPPG in practical scenarios and its potential as a non-contact alternative to traditional cardiovascular monitoring techniques

Remote Bio-Sensing: Open Source Benchmark Framework for Fair Evaluation of rPPG

Remote Photoplethysmography (rPPG) is a technology that utilizes the light absorption properties of hemoglobin, captured via camera, to analyze and measure blood volume pulse (BVP). By analyzing the measured BVP, various physiological signals such as heart rate, stress levels, and blood pressure can be derived, enabling applications such as the early prediction of cardiovascular diseases. rPPG is a rapidly evolving field as it allows the measurement of vital signals using camera-equipped devices without the need for additional devices such as blood pressure monitors or pulse oximeters, and without the assistance of medical experts. Despite extensive efforts and advances in this field, serious challenges remain, including issues related to skin color, camera characteristics, ambient lighting, and other sources of noise, which degrade performance accuracy. We argue that fair and evaluable benchmarking is urgently required to overcome these challenges and make any meaningful progress from both academic and commercial perspectives. In most existing work, models are trained, tested, and validated only on limited datasets. Worse still, some studies lack available code or reproducibility, making it difficult to fairly evaluate and compare performance. Therefore, the purpose of this study is to provide a benchmarking framework to evaluate various rPPG techniques across a wide range of datasets for fair evaluation and comparison, including both conventional non-deep neural network (non-DNN) and deep neural network (DNN) methods. GitHub URL: https://github.com/remotebiosensing/rppg.Comment: 19 pages, 10 figure

Quantitative Multidimensional Stress Assessment from Facial Videos

Stress has a significant impact on the physical and mental health of an individual and is a growing concern for society, especially during the COVID-19 pandemic. Facial video-based stress evaluation from non-invasive cameras has proven to be a significantly more efficient method to evaluate stress in comparison to approaches that use questionnaires or wearable sensors. Plenty of classification models have been built for stress detection. However, most do not consider individual differences. Also, the results for such models are limited by a uni-dimensional definition of stress levels lacking a comprehensive quantitative definition of stress. The dissertation focuses on building a framework that utilizes the multilevel video frame representations from deep learning and the remote photoplethysmography signals extracted from the facial videos for stress assessment. The fusion model takes the inputs of a baseline video and a target video of the subject. The physiological features such as heart rate and heart rate variability are used with the initial stress scores generated from deep learning are used to predict the stress scores in cognitive anxiety, somatic anxiety, and self-confidence. To generate stress scores with better accuracy, the signal extraction method is improved by introducing the CWT-SNR method that uses the signal-to-noise ratio to assist the adaptive bandpass filtering in the post-processing of the signals. A study on phase space reconstruction features is performed and the results show the potential for additional accuracy improvement for the heart rate variability detection. To select the best deep learning architecture, multiple deep learning architectures are tested to build the deep learning model. Support Vector Regression is used to generate the output stress score results. Testing with the data from the UBFC-Phys dataset, the fusion model shows a strong correlation between ground truth and the predicted results

Psychophysiology in games

Psychophysiology is the study of the relationship between psychology and its physiological manifestations. That relationship is of particular importance for both game design and ultimately gameplaying. Players’ psychophysiology offers a gateway towards a better understanding of playing behavior and experience. That knowledge can, in turn, be beneficial for the player as it allows designers to make better games for them; either explicitly by altering the game during play or implicitly during the game design process. This chapter argues for the importance of physiology for the investigation of player affect in games, reviews the current state of the art in sensor technology and outlines the key phases for the application of psychophysiology in games.The work is supported, in part, by the EU-funded FP7 ICT iLearnRWproject (project no: 318803).peer-reviewe

Utilising Emotion Monitoring for Developing Music Interventions for People with Dementia:A State-of-the-Art Review

The demand for smart solutions to support people with dementia (PwD) is increasing. These solutions are expected to assist PwD with their emotional, physical, and social well-being. At the moment, state-of-the-art works allow for the monitoring of physical well-being; however, not much attention is delineated for monitoring the emotional and social well-being of PwD. Research on emotion monitoring can be combined with research on the effects of music on PwD given its promising effects. More specifically, knowledge of the emotional state allows for music intervention to alleviate negative emotions by eliciting positive emotions in PwD. In this direction, the paper conducts a state-of-the-art review on two aspects: (i) the effect of music on PwD and (ii) both wearable and non-wearable sensing systems for emotional state monitoring. After outlining the application of musical interventions for PwD, including emotion monitoring sensors and algorithms, multiple challenges are identified. The main findings include a need for rigorous research approaches for the development of adaptable solutions that can tackle dynamic changes caused by the diminishing cognitive abilities of PwD with a focus on privacy and adoption aspects. By addressing these requirements, advancements can be made in harnessing music and emotion monitoring for PwD, thereby facilitating the creation of more resilient and scalable solutions to aid caregivers and PwD