Harmonic Sum-based Method for Heart Rate Estimation using PPG Signals Affected with Motion Artifacts

Wearable photoplethysmography has recently become a common technology in heart rate (HR) monitoring. General observation is that the motion artifacts change the statistics of the acquired PPG signal. Consequently, estimation of HR from such a corrupted PPG signal is challenging. However, if an accelerometer is also used to acquire the acceleration signal simultaneously, it can provide helpful information that can be used to reduce the motion artifacts in the PPG signal. By dint of repetitive movements of the subjects hands while running, the accelerometer signal is found to be quasi-periodic. Over short-time intervals, it can be modeled by a finite harmonic sum (HSUM). Using the HSUM model, we obtain an estimate of the instantaneous fundamental frequency of the accelerometer signal. Since the PPG signal is a composite of the heart rate information (that is also quasi-periodic) and the motion artifact, we fit a joint HSUM model to the PPG signal. One of the harmonic sums corresponds to the heart-beat component in PPG and the other models the motion artifact. However, the fundamental frequency of the motion artifact has already been determined from the accelerometer signal. Subsequently, the HR is estimated from the joint HSUM model. The mean absolute error in HR estimates was 0.7359 beats per minute (BPM) with a standard deviation of 0.8328 BPM for 2015 IEEE Signal Processing cup data. The ground-truth HR was obtained from the simultaneously acquired ECG for validating the accuracy of the proposed method. The proposed method is compared with four methods that were recently developed and evaluated on the same dataset

Motion-resistant pulse oximetry

The measurement of vital signs ? such as peripheral capillary oxygen saturation (SpO2) and heart rate (HR) levels ? by a pulse oximeter is studied. The pulse oximeter is a non-invasive device that measures photoplethysmography (PPG) signals and extracts vital signs from them. However, the quality of the PPG signal measured by oximetry sensors is known to deteriorate in the presence of substantial human and sensor movements contributing to the measurement noise. Methods to suppress such noise from PPG signals measured by an oximeter and to calculate the associated vital signs with high accuracy even when the wearer is under substantial motion are presented in this study. The spectral components of the PPG waveform are known to appear at a fundamental frequency that corresponds to the participant\u27s HR and at its harmonics. To match this signal, a time-varying comb filter tuned to the participant\u27s HR is employed. The filter captures the HR components and eliminates most other artifacts. A significant improvement in the accuracy of SpO2 calculated from the comb-filtered PPG signals is observed, when tested on data collected from human participants while they are at rest and while they are exercising. In addition, an architecture that integrates SpO2 levels from multiple PPG channels mounted on different parts of the wearer\u27s arm is presented. The SpO2 levels are integrated using a Kalman filter that uses past measurements and modeling of the SpO2 dynamics to attenuate the effect of the motion artifacts. Again, data collected from human participants while they are at rest and while they are exercising are used. The integrated SpO2 levels are shown to be more accurate and reliable than those calculated from individual channels. Motion-resistant algorithms typically require an additional noise reference signal to produce high quality vital signs such as HR. A framework that employs PPG sensors only ? one in the green and one in the infrared spectrum ? to compute high quality HR levels is developed. Our framework is tested on experimental data collected from human participants while at rest and while running at various speeds. Our PPG-only framework generates HR levels with high accuracy and low computational complexity as compared to leading HR calculation methods in the literature that require the availability of a noise reference signal. The methods for SpO2 and HR calculation presented in this study are desirable since (1) they yield high accuracy in estimating vital signs under substantial level of motion artifacts and (2) they are computationally efficient, (and therefore are capable to be implemented in wearable devices)

Characterizing the Noise Associated with Sensor Placement and Motion Artifacts and Overcoming its Effects for Body-worn Physiological Sensors

Wearable sensors for continuous physiological monitoring have the potential to change the paradigm for healthcare by providing information in scenarios not covered by the existing clinical model. One key challenge for wearable physiological sensors is that their signal-to-noise ratios are low compared to those of their medical grade counterparts in hospitals. Two primary sources of noise are the sensor-skin contact interface and motion artifacts due to the user’s daily activities. These are challenging problems because the initial sensor placement by the user may not be ideal, the skin conditions can change over time, and the nature of motion artifacts is not predictable. The objective of this research is twofold. The first is to design sensors with reconfigurable contact to mitigate the effects of misplaced sensors or changing skin conditions. The second is to leverage signal processing techniques for accurate physiological parameter estimation despite the presence of motion artifacts. In this research, the sensor contact problem was specifically addressed for dry-contact electroencephalography (EEG). The proposed novel extension to a popular existing EEG electrode design enabled reconfigurable contact to adjust to variations in sensor placement and skin conditions over time. Experimental results on human subjects showed that reconfiguration of contact can reduce the noise in collected EEG signals without the need for manual intervention. To address the motion artifact problem, a particle filter based approach was employed to track the heart rate in cardiac signals affected by the movements of the user. The algorithm was tested on cardiac signals from human subjects running on a treadmill and showed good performance in accurately tracking heart rate. Moreover, the proposed algorithm enables fusion of multiple modalities and is also computationally more efficient compared to other contemporary approaches

Imaging photoplethysmography: towards effective physiological measurements

Since its conception decades ago, Photoplethysmography (PPG) the non-invasive opto-electronic technique that measures arterial pulsations in-vivo has proven its worth by achieving and maintaining its rank as a compulsory standard of patient monitoring. However successful, conventional contact monitoring mode is not suitable in certain clinical and biomedical situations, e.g., in the case of skin damage, or when unconstrained movement is required. With the advance of computer and photonics technologies, there has been a resurgence of interest in PPG and one potential route to overcome the abovementioned issues has been increasingly explored, i.e., imaging photoplethysmography (iPPG). The emerging field of iPPG offers some nascent opportunities in effective and comprehensive interpretation of the physiological phenomena, indicating a promising alternative to conventional PPG. Heart and respiration rate, perfusion mapping, and pulse rate variability have been accessed using iPPG. To effectively and remotely access physiological information through this emerging technique, a number of key issues are still to be addressed. The engineering issues of iPPG, particularly the influence of motion artefacts on signal quality, are addressed in this thesis, where an engineering model based on the revised Beer-Lambert law was developed and used to describe opto-physiological phenomena relevant to iPPG. An iPPG setup consisting of both hardware and software elements was developed to investigate its reliability and reproducibility in the context of effective remote physiological assessment. Specifically, a first study was conducted for the acquisition of vital physiological signs under various exercise conditions, i.e. resting, light and heavy cardiovascular exercise, in ten healthy subjects. The physiological parameters derived from the images captured by the iPPG system exhibited functional characteristics comparable to conventional contact PPG, i.e., maximum heart rate difference was <3 bpm and a significant (p < 0.05) correlation between both measurements were also revealed. Using a method for attenuation of motion artefacts, the heart rate and respiration rate information was successfully assessed from different anatomical locations even in high-intensity physical exercise situations. This study thereby leads to a new avenue for noncontact sensing of vital signs and remote physiological assessment, showing clear and promising applications in clinical triage and sports training. A second study was conducted to remotely assess pulse rate variability (PRV), which has been considered a valuable indicator of autonomic nervous system (ANS) status. The PRV information was obtained using the iPPG setup to appraise the ANS in ten normal subjects. The performance of the iPPG system in accessing PRV was evaluated via comparison with the readings from a contact PPG sensor. Strong correlation and good agreement between these two techniques verify the effectiveness of iPPG in the remote monitoring of PRV, thereby promoting iPPG as a potential alternative to the interpretation of physiological dynamics related to the ANS. The outcomes revealed in the thesis could present the trend of a robust non-contact technique for cardiovascular monitoring and evaluation

Blind Source Separation for the Processing of Contact-Less Biosignals

(Spatio-temporale) Blind Source Separation (BSS) eignet sich für die Verarbeitung von Multikanal-Messungen im Bereich der kontaktlosen Biosignalerfassung. Ziel der BSS ist dabei die Trennung von (z.B. kardialen) Nutzsignalen und Störsignalen typisch für die kontaktlosen Messtechniken. Das Potential der BSS kann praktisch nur ausgeschöpft werden, wenn (1) ein geeignetes BSS-Modell verwendet wird, welches der Komplexität der Multikanal-Messung gerecht wird und (2) die unbestimmte Permutation unter den BSS-Ausgangssignalen gelöst wird, d.h. das Nutzsignal praktisch automatisiert identifiziert werden kann. Die vorliegende Arbeit entwirft ein Framework, mit dessen Hilfe die Effizienz von BSS-Algorithmen im Kontext des kamera-basierten Photoplethysmogramms bewertet werden kann. Empfehlungen zur Auswahl bestimmter Algorithmen im Zusammenhang mit spezifischen Signal-Charakteristiken werden abgeleitet. Außerdem werden im Rahmen der Arbeit Konzepte für die automatisierte Kanalauswahl nach BSS im Bereich der kontaktlosen Messung des Elektrokardiogramms entwickelt und bewertet. Neuartige Algorithmen basierend auf Sparse Coding erwiesen sich dabei als besonders effizient im Vergleich zu Standard-Methoden.(Spatio-temporal) Blind Source Separation (BSS) provides a large potential to process distorted multichannel biosignal measurements in the context of novel contact-less recording techniques for separating distortions from the cardiac signal of interest. This potential can only be practically utilized (1) if a BSS model is applied that matches the complexity of the measurement, i.e. the signal mixture and (2) if permutation indeterminacy is solved among the BSS output components, i.e the component of interest can be practically selected. The present work, first, designs a framework to assess the efficacy of BSS algorithms in the context of the camera-based photoplethysmogram (cbPPG) and characterizes multiple BSS algorithms, accordingly. Algorithm selection recommendations for certain mixture characteristics are derived. Second, the present work develops and evaluates concepts to solve permutation indeterminacy for BSS outputs of contact-less electrocardiogram (ECG) recordings. The novel approach based on sparse coding is shown to outperform the existing concepts of higher order moments and frequency-domain features

Machine learning algorithm development of SPO2 sensor for improved robustness in wearables

Wearable devices application in the digital measurement of health has gained attention by researchers. These devices allow for data acquisition during real-life activities, resulting in higher data availability. They often include photoplethysmography (PPG) sensors, the sensor behind pulse oximetry. Pulse oximetry is a non-invasive method for continuous oxygen saturation (SpO2) measurements, a standard monitor for anesthesia procedures, an essential tool for managing patients undergoing pulmonary rehabilitation and an effective method for assessing sleep-disordered breathing. However, the current market focuses on heart rate measurements and lacks the robustness of clinical applications for SpO2 assessment. In addition, the most common obstacle in PPG measurements is the signal quality, especially in the form of motion artifacts. Thus, this work aims at increasing the clinical robustness in this devices by evaluating its quality and then extracting relevant metrics. Firstly, a data acquisition protocol was developed, focused on acquiring data during daily activities. This resulted in a dataset with different signal qualities, which was manually annotated to be used as the base for the Machine Learning models. A second protocol was also developed especially designed for the extraction of the SpO2 measurement. Several Machine Learning models were developed to evaluate the signal in three distinct qualities (corrupted, suboptimal, optimal) in real time. A Random Forest classifier achieved accuracies of 79% and 80% for the binary models capable of differentiating between usable and unusable signals, and accuracies of 74% and 80% when distinguishing between optimal and suboptimal signals, for the two utilized channels. The multi-class models achieved accuracies of 66% and 65% for the two utilized channels. Three clinically relevant metrics were also extracted from the PPG signal: heart rate, respiratory rate and SpO2. The heart rate and respiratory rate algorithms resulted in performances similar to the ones found in the literature and in other devices currently on the market. However, while promising, more data is needed to reach statistical significance for the SpO2 measurement.A monitorização do estado de saúde de pacientes em ambulatório utilizando dispositivos wearables tem vindo a ser cada vez mais investigada. Estes dispositivos permitem uma aquisição de dados durante o dia a dia, resultando num maior conjunto de dados. Frequentemente, estes dispositivos incluem fotopletismógrafos (PPG), o sensor por detrás da oximetria de pulso. A oximetria de pulso é um método não invasivo para a medição da saturação de oxigénio no sangue (SpO2) de forma contínua. É um equipamento padrão para procedimentos com anestesia, uma ferramenta essencial para monitorizar pacientes em reabilitação pulmonar e um método eficaz para avaliar respiração desordenada do sono. Ainda assim, o mercado atual foca-se principalmente em medições da frequência cardíaca e carece robustez para aplicações clínicas da medição de SpO2. Para além disso, o obstáculo mais comum em medições com PPG é a qualidade do sinal. Consequentemente, este trabalho procura melhorar a robustez clínica destes dispositivos analisando a qualidade do sinal e, posteriormente, extrair métricas relevantes. Primeiramente, foi desenvolvido um protocolo para aquisição de dados de atividades do dia a dia. Assim, foram adquiridos dados com diferentes qualidades, que foram avaliados manualmente de forma a servir de base para os vários modelos de Machine Learning. Também foi desenvolvido um segundo protocolo para a extração do valor de SpO2. Diferentes modelos de Machine Learning foram desenvolvidos para avaliar em tempo real a qualidade do sinal em três qualidades (corrompido, subótimo, ótimo) . Um classificador baseado em Random Forest atingiu exatidões de 79% e 80% em classificadores binários capazes de distinguir entre sinais úteis e inúteis, e exatidões de 74% e 80% a diferenciar entre um sinal subótimo e ótimo, para os dois canais usados. Os classificadores multi-classe atingiram exatidões de 66% e 65% para os dois canais usados. Três medidas clinicamente relevantes foram também extraídas do sinal de PPG: frequências cardíaca e respiratória, cujos algoritmos atingiram resultados semelhantes aos encontrados na literatura e em aparelhos no mercado, e SpO2 que, ainda que promissores, mais dados seriam necessários para os resultados serem estatisticamente significativo

Comparative evaluation of the applicability of self-organized operational neural networks to remote photoplethysmography

Abstract. Photoplethysmography (PPG) is a widely applied means of obtaining blood volume pulse (BVP) information from subjects which can be used for monitoring numerous physiological signs such as heart rate and respiration. Following observations that blood volume information can also be retrieved from videos recorded of the human face, several approaches for the remote extraction of PPG signals have been proposed in literature. These methods are collectively referred to as remote photoplethysmography (rPPG). The current state of the art of rPPG approaches is represented by deep convolutional neural network (CNN) models, which have been successfully applied in a wide range of computer vision tasks. A novel technology called operational neural networks (ONNs) has recently been proposed in literature as an extension of convolutional neural networks. ONNs attempt to overcome the limitations of conventional CNN models which are primarily caused by exclusively employing the linear neuron model. In addition, to address certain drawbacks of ONNs, a technology called self- organized operational neural networks (Self-ONNs) have recently been proposed as an extension of ONNs. This thesis presents a novel method for rPPG extraction based on self-organized operational neural networks. To comprehensively evaluate the applicability of Self-ONNs as an approach for rPPG extraction, three Self-ONN models with varying number of layers are implemented and evaluated on test data from three data sets representing different distributions. The performance of the proposed models are compared against corresponding CNN architectures as well as a typical unsupervised rPPG pipeline. The performance of the methods is evaluated based on heart rate estimations calculated from the extracted rPPG signals. In the presented experimental setup, Self-ONN models did not result in improved heart rate estimation performance over parameter-equivalent CNN alternatives. However, every Self-ONN model showed superior ability to fit the train target, which both shows promise for the applicability of Self-ONNs as well as suggests inherent problems in the training setup. Additionally, when taking into account the required computational resources in addition to raw HR estimation performance, certain Self-ONN models showcased improved efficiency over CNN alternatives. As such, the experiments nonetheless present a promising proof of concept which can serve as grounds for future research.Vertaileva arviointi itseorganisoituvien operationaalisten neuroverkkojen soveltuvuudesta etäfotopletysmografiaan. Tiivistelmä. Fotopletysmografia on laajasti sovellettu menetelmä veritilavuuspulssi-informaation saamiseksi kohteista, jota voidaan käyttää useiden fysiologisten arvojen, kuten sydämensykkeen ja hengityksen, seurannassa. Seuraten havainnoista, että veritilavuusinformaatiota on mahdollista palauttaa myös ihmiskasvoista kuvatuista videoista, useita menetelmiä fotopletysmografiasignaalien erottamiseksi etänä on esitetty kirjallisuudessa. Yhteisnimitys näille menetelmille on etäfotopletysmografia (remote photoplethysmography, rPPG). Syvät konvolutionaaliset neuroverkkomallit (convolutional neural networks, CNNs), joita on onnistuneesti sovellettu laajaan valikoimaan tietokonenäön tehtäviä, edustavat nykyistä rPPG-lähestymistapojen huippua. Uusi teknologia nimeltään operationaaliset neuroverkot (operational neural networks, ONNs) on hiljattain esitetty kirjallisuudessa konvolutionaalisten neuroverkkojen laajennukseksi. ONN:t pyrkivät eroon tavanomaisten CNN-mallien rajoitteista, jotka johtuvat pääasiassa lineaarisen neuronimallin yksinomaisesta käytöstä. Lisäksi tietyistä ONN-mallien heikkouksista eroon pääsemiseksi, teknologia nimeltään itseorganisoituvat operationaaliset neuroverkot (self-organized operational neural networks, Self-ONNs) on hiljattain esitetty lajeennuksena ONN:ille. Tämä tutkielma esittelee uudenlaisen menetelmän rPPG-erotukselle pohjautuen itseorganisoituviin operationaalisiin neuroverkkoihin. Self-ONN:ien soveltuvuuden rPPG-erotukseen perusteelliseksi arvioimiseksi kolme Self-ONN -mallia vaihtelevalla määrällä kerroksia toteutetaan ja arvioidaan testidatalla kolmesta eri datajoukosta, jotka edustavat eri jakaumia. Esitettyjen mallien suorituskykyä verrataan vastaaviin CNN-arkkitehtuureihin sekä tyypilliseen ohjaamattomaan rPPG-liukuhihnaan. Menetelmien suorituskykyä arvioidaan perustuen rPPG-signaaleista laskettuihin sydämensykearvioihin. Esitellyssä kokeellisessa asetelmassa Self-ONN:t eivät johtaneet parempiin sykearvioihin verrattuna parametrivastaaviin CNN-vaihtoehtoihin. Self-ONN:t kuitenkin osoittivat ylivertaista kykyä sovittaa opetuskohteen, mikä sekä on lupaavaa Self-ONN:ien soveltuvuuden kannalta että viittaa luontaisiin ongelmiin opetusasetelmassa. Lisäksi, kun huomioon otetaan vaaditut laskentaresurssit raa’an sykkeen arvioinnin suorituskyvyn lisäksi, tietyt Self-ONN -mallit osoittivat parempaa tehokkuutta CNN-vaihtoehtoihin verrattuna. Näin ollen kokeet joka tapauksessa tarjoavat lupaavan konseptitodistuksen, joka voi toimia perustana tulevalle tutkimukselle