Multi-hierarchical Convolutional Network for Efficient Remote Photoplethysmograph Signal and Heart Rate Estimation from Face Video Clips

Heart beat rhythm and heart rate (HR) are important physiological parameters of the human body. This study presents an efficient multi-hierarchical spatio-temporal convolutional network that can quickly estimate remote physiological (rPPG) signal and HR from face video clips. First, the facial color distribution characteristics are extracted using a low-level face feature Generation (LFFG) module. Then, the three-dimensional (3D) spatio-temporal stack convolution module (STSC) and multi-hierarchical feature fusion module (MHFF) are used to strengthen the spatio-temporal correlation of multi-channel features. In the MHFF, sparse optical flow is used to capture the tiny motion information of faces between frames and generate a self-adaptive region of interest (ROI) skin mask. Finally, the signal prediction module (SP) is used to extract the estimated rPPG signal. The experimental results on the three datasets show that the proposed network outperforms the state-of-the-art methods.Comment: 33 pages,9 figure

Robust heart-rate estimation from facial videos using Project_ICA.

‘This is an author-created, un-copyedited version of an article published in Physiological Measurement. IOP Publishing Ltd is not responsible for any errors or omissions in this version of the manuscript or any version derived from it. The Version of Record is available online at https://doi.org/10.1088/1361-6579/ab2c9f’OBJECTIVE: Remote photoplethysmography (rPPG) can achieve non-contact measurement of heart rate (HR) from a continuous video sequence by scanning the skin surface. However, practical applications are still limited by factors such as non-rigid facial motion and head movement. In this work, a detailed system framework for remotely estimating heart rate from facial videos under various movement conditions is described. APPROACH: After the rPPG signal has been obtained from a defined region of the facial skin, a method, termed 'Project_ICA', based on a skin reflection model, is employed to extract the pulse signal from the original signal. MAIN RESULTS: To evaluate the performance of the proposed algorithm, a dataset containing 112 videos including the challenges of various skin tones, body motion and HR recovery after exercise was created from 28 participants. SIGNIFICANCE: The results show that Project_ICA, when evaluated by several criteria, provides a more accurate and robust estimate of HR than most existing methods, although problems remain in obtaining reliable measurements from dark-skinned subjects

Automated Remote Pulse Oximetry System (ARPOS)

Funding: This research is funded by the School of Computer Science and by St Leonard’s Postgraduate College Doctoral Scholarship, both at the University of St Andrews for Pireh Pirzada’s PhD. Early work was funded by the Digital Health & Care Innovation Centre (DHI).Current methods of measuring heart rate (HR) and oxygen levels (SPO2) require physical contact, are individualised, and for accurate oxygen levels may also require a blood test. No-touch or non-invasive technologies are not currently commercially available for use in healthcare settings. To date, there has been no assessment of a system that measures HR and SPO2 using commercial off-the-shelf camera technology that utilises R, G, B and IR data. Moreover, no formal remote photoplethysmography studies have been done in real life scenarios with participants at home with different demographic characteristics. This novel study addresses all these objectives by developing, optimising, and evaluating a system that measures the HR and SPO2 of 40 participants. HR and SPO2 are determined by measuring the frequencies from different wavelength band regions using FFT and radiometric measurements after pre-processing face regions of interest (forehead, lips, and cheeks) from Colour, IR and Depth data. Detrending, interpolating, hamming, and normalising the signal with FastICA produced the lowest RMSE of 7.8 for HR with the r-correlation value of 0.85 and RMSE 2.3 for SPO2. This novel system could be used in several critical care settings, including in care homes and in hospitals and prompt clinical intervention as required.Publisher PDFPeer reviewe

Medición de signos vitales mediante técnicas de visión artificial

En esta tesis, se presenta el desarrollo de un sistema de medición de signos vitales, mediante la aplicación de técnicas de visión artificial. Para lo cual, se detalla una revisión sistemática sobre la supervisión de signos vitales. Se analizan los aspectos y conceptos relevantes para la medición de signos vitales a través de técnicas de visión artificial. Además, se identifica los puntos débiles que existen en los sistemas de medición actuales basados en técnicas de visión artificial. En función de esto, se desarrollan mejoras y posibles soluciones para optimizar las mediciones. Finalmente, se detalla un sistema que permite medir mediante técnicas de visión artificial y sin contacto: la frecuencia cardíaca, la frecuencia respiratoria, la saturación de oxígeno y la temperatura corporal. El mismo, es evaluado frente a sistemas clásicos de medición y a sistemas existentes de medición que funcionan por imagen fotopletismografía. Finalmente, se implementa el sistema dentro del framework conocido como ROS, lo que le permite funcionar en robots.In this thesis, we describe the development of a vital sign measurement system through the application of artificial vision techniques. As a first step, a systematic review of the vital signs supervision is detailed. The relevant aspects and concepts for the measurement of vital signs through artificial vision techniques are analyzed as well. In addition, the weak points of current measurement systems based on techniques of artificial vision are identified. Based on these, we developed some solutions and improvements to optimize the measurements. Finally, the thesis presents the description of the proposed system that allows to measure by means of techniques of computer vision and without contact: the heart rate, the respiratory rate, the saturation of oxygen and the corporal temperature. This system is evaluated against classical measurement systems and existing measuring systems that work by photoplethysmography. Finally, the system is implemented within the framework known as ROS which allows it to work in robots.Programa Oficial de Doctorado en Ingeniería Eléctrica, Electrónica y AutomáticaPresidente: José María Sebastián y Zúñiga.- Secretario: Fares Jawad Abu-Dakka.- Vocal: Pedro Guerra Lópe