Recent development of respiratory rate measurement technologies

Respiratory rate (RR) is an important physiological parameter whose abnormity has been regarded as an important indicator of serious illness. In order to make RR monitoring simple to do, reliable and accurate, many different methods have been proposed for such automatic monitoring. According to the theory of respiratory rate extraction, methods are categorized into three modalities: extracting RR from other physiological signals, RR measurement based on respiratory movements, and RR measurement based on airflow. The merits and limitations of each method are highlighted and discussed. In addition, current works are summarized to suggest key directions for the development of future RR monitoring methodologies

Contactless extraction of respiratory rate from depth and thermal sensors

Monitoring of respiration and restless sleep can help detect sleep disturbances that may be indicative of poor health and functional deficits. The current methods of estimating the respiratory rate such as Pneumograph, Capnograph, Photo-plethysmograph (PPG), Respiratory inductance plethysmography (RIP), involve sensors that are in contact with the patient. However, we have a few scenarios such as in hospitals and senior retirement communities where we would like to non-invasively collect the respiration rate and restless body motion where we are not able to place these types of sensors on patients. The initial requirement was to non-invasively monitor vital activity of patients in psychiatric centers. This work investigates a novel approach to estimate the respiratory rate of a person lying on the bed using depth and thermal sensors along with other signal processing algorithms. The initial proof of concept tests were conducted on three subjects. Additional testing on a diverse group of ten participants (ranging in age and body type) was performed to validate the algorithm and the data collection method. The depth and thermal waveforms captured were tested to explore a new approach for detecting individual respiratory rate noninvasively, using various algorithms to detect the region of the bed, common grids where a person is present, best signal selection from grids, and accurately estimate the respiratory rate and amount of body movement during sleep. The performance results at approximately 30 frames per second for the set of 10 participants was a mean error difference of 0.6 breaths per minute for the time domain algorithm and 0.8 breaths per minute for the frequency domain algorithm.Includes bibliographical reference

State of the art of audio- and video based solutions for AAL

Working Group 3. Audio- and Video-based AAL ApplicationsIt is a matter of fact that Europe is facing more and more crucial challenges regarding health and social care due to the demographic change and the current economic context. The recent COVID-19 pandemic has stressed this situation even further, thus highlighting the need for taking action. Active and Assisted Living (AAL) technologies come as a viable approach to help facing these challenges, thanks to the high potential they have in enabling remote care and support. Broadly speaking, AAL can be referred to as the use of innovative and advanced Information and Communication Technologies to create supportive, inclusive and empowering applications and environments that enable older, impaired or frail people to live independently and stay active longer in society. AAL capitalizes on the growing pervasiveness and effectiveness of sensing and computing facilities to supply the persons in need with smart assistance, by responding to their necessities of autonomy, independence, comfort, security and safety. The application scenarios addressed by AAL are complex, due to the inherent heterogeneity of the end-user population, their living arrangements, and their physical conditions or impairment. Despite aiming at diverse goals, AAL systems should share some common characteristics. They are designed to provide support in daily life in an invisible, unobtrusive and user-friendly manner. Moreover, they are conceived to be intelligent, to be able to learn and adapt to the requirements and requests of the assisted people, and to synchronise with their specific needs. Nevertheless, to ensure the uptake of AAL in society, potential users must be willing to use AAL applications and to integrate them in their daily environments and lives. In this respect, video- and audio-based AAL applications have several advantages, in terms of unobtrusiveness and information richness. Indeed, cameras and microphones are far less obtrusive with respect to the hindrance other wearable sensors may cause to one’s activities. In addition, a single camera placed in a room can record most of the activities performed in the room, thus replacing many other non-visual sensors. Currently, video-based applications are effective in recognising and monitoring the activities, the movements, and the overall conditions of the assisted individuals as well as to assess their vital parameters (e.g., heart rate, respiratory rate). Similarly, audio sensors have the potential to become one of the most important modalities for interaction with AAL systems, as they can have a large range of sensing, do not require physical presence at a particular location and are physically intangible. Moreover, relevant information about individuals’ activities and health status can derive from processing audio signals (e.g., speech recordings). Nevertheless, as the other side of the coin, cameras and microphones are often perceived as the most intrusive technologies from the viewpoint of the privacy of the monitored individuals. This is due to the richness of the information these technologies convey and the intimate setting where they may be deployed. Solutions able to ensure privacy preservation by context and by design, as well as to ensure high legal and ethical standards are in high demand. After the review of the current state of play and the discussion in GoodBrother, we may claim that the first solutions in this direction are starting to appear in the literature. A multidisciplinary 4 debate among experts and stakeholders is paving the way towards AAL ensuring ergonomics, usability, acceptance and privacy preservation. The DIANA, PAAL, and VisuAAL projects are examples of this fresh approach. This report provides the reader with a review of the most recent advances in audio- and video-based monitoring technologies for AAL. It has been drafted as a collective effort of WG3 to supply an introduction to AAL, its evolution over time and its main functional and technological underpinnings. In this respect, the report contributes to the field with the outline of a new generation of ethical-aware AAL technologies and a proposal for a novel comprehensive taxonomy of AAL systems and applications. Moreover, the report allows non-technical readers to gather an overview of the main components of an AAL system and how these function and interact with the end-users. The report illustrates the state of the art of the most successful AAL applications and functions based on audio and video data, namely (i) lifelogging and self-monitoring, (ii) remote monitoring of vital signs, (iii) emotional state recognition, (iv) food intake monitoring, activity and behaviour recognition, (v) activity and personal assistance, (vi) gesture recognition, (vii) fall detection and prevention, (viii) mobility assessment and frailty recognition, and (ix) cognitive and motor rehabilitation. For these application scenarios, the report illustrates the state of play in terms of scientific advances, available products and research project. The open challenges are also highlighted. The report ends with an overview of the challenges, the hindrances and the opportunities posed by the uptake in real world settings of AAL technologies. In this respect, the report illustrates the current procedural and technological approaches to cope with acceptability, usability and trust in the AAL technology, by surveying strategies and approaches to co-design, to privacy preservation in video and audio data, to transparency and explainability in data processing, and to data transmission and communication. User acceptance and ethical considerations are also debated. Finally, the potentials coming from the silver economy are overviewed.publishedVersio

Imaging photoplethysmography: towards effective physiological measurements

Since its conception decades ago, Photoplethysmography (PPG) the non-invasive opto-electronic technique that measures arterial pulsations in-vivo has proven its worth by achieving and maintaining its rank as a compulsory standard of patient monitoring. However successful, conventional contact monitoring mode is not suitable in certain clinical and biomedical situations, e.g., in the case of skin damage, or when unconstrained movement is required. With the advance of computer and photonics technologies, there has been a resurgence of interest in PPG and one potential route to overcome the abovementioned issues has been increasingly explored, i.e., imaging photoplethysmography (iPPG). The emerging field of iPPG offers some nascent opportunities in effective and comprehensive interpretation of the physiological phenomena, indicating a promising alternative to conventional PPG. Heart and respiration rate, perfusion mapping, and pulse rate variability have been accessed using iPPG. To effectively and remotely access physiological information through this emerging technique, a number of key issues are still to be addressed. The engineering issues of iPPG, particularly the influence of motion artefacts on signal quality, are addressed in this thesis, where an engineering model based on the revised Beer-Lambert law was developed and used to describe opto-physiological phenomena relevant to iPPG. An iPPG setup consisting of both hardware and software elements was developed to investigate its reliability and reproducibility in the context of effective remote physiological assessment. Specifically, a first study was conducted for the acquisition of vital physiological signs under various exercise conditions, i.e. resting, light and heavy cardiovascular exercise, in ten healthy subjects. The physiological parameters derived from the images captured by the iPPG system exhibited functional characteristics comparable to conventional contact PPG, i.e., maximum heart rate difference was <3 bpm and a significant (p < 0.05) correlation between both measurements were also revealed. Using a method for attenuation of motion artefacts, the heart rate and respiration rate information was successfully assessed from different anatomical locations even in high-intensity physical exercise situations. This study thereby leads to a new avenue for noncontact sensing of vital signs and remote physiological assessment, showing clear and promising applications in clinical triage and sports training. A second study was conducted to remotely assess pulse rate variability (PRV), which has been considered a valuable indicator of autonomic nervous system (ANS) status. The PRV information was obtained using the iPPG setup to appraise the ANS in ten normal subjects. The performance of the iPPG system in accessing PRV was evaluated via comparison with the readings from a contact PPG sensor. Strong correlation and good agreement between these two techniques verify the effectiveness of iPPG in the remote monitoring of PRV, thereby promoting iPPG as a potential alternative to the interpretation of physiological dynamics related to the ANS. The outcomes revealed in the thesis could present the trend of a robust non-contact technique for cardiovascular monitoring and evaluation

The 2023 wearable photoplethysmography roadmap

Photoplethysmography is a key sensing technology which is used in wearable devices such as smartwatches and fitness trackers. Currently, photoplethysmography sensors are used to monitor physiological parameters including heart rate and heart rhythm, and to track activities like sleep and exercise. Yet, wearable photoplethysmography has potential to provide much more information on health and wellbeing, which could inform clinical decision making. This Roadmap outlines directions for research and development to realise the full potential of wearable photoplethysmography. Experts discuss key topics within the areas of sensor design, signal processing, clinical applications, and research directions. Their perspectives provide valuable guidance to researchers developing wearable photoplethysmography technology

Methods for Doppler Radar Monitoring of Physiological Signals

Unobtrusive health monitoring includes advantages such as long-term monitoring of rarely occurring conditions or of slow changes in health, at reasonable costs. In addition, the preparation of electrodes or other sensors is not needed. Currently, the main limitation of remote patient monitoring is not in the existing communication infrastructure but the lack of reliable, easy-to-use, and well-studied sensors.The aim of this thesis was to develop methods for monitoring cardiac and respiratory activity with microwave continuous wave (CW) Doppler radar. When considering cardiac and respiration monitoring, the heart and respiration rates are often the first monitored parameters. The motivation of this thesis, however, is to measure not only rate-related parameters but also the cardiac and respiratory waveforms, including the chest wall displacement information.This dissertation thoroughly explores the signal processing methods for accurate chest wall displacement measurement with a radar sensor. The sensor prototype and measurement setup choices are reported. The contributions of this dissertation encompass an I/Q imbalance estimation method and a nonlinear demodulation method for a quadrature radar sensor. Unlike the previous imbalance estimation methods, the proposed method does not require the use of laboratory equipment. The proposed nonlinear demodulation method, on the other hand, is shown to be more accurate than other methods in low-noise cases. In addition, the separation of the cardiac and respiratory components with independent component analysis (ICA) is discussed. The developed methods were validated with simulations and with simplified measurement setups in an office environment. The performance of the nonlinear demodulation method was also studied with three patients for sleep-time respiration monitoring. This is the first time that whole-night measurements have been analyzed with the method in an uncontrolled environment. Data synchronization between the radar sensor and a commercial polysomnographic (PSG) device was assured with a developed infrared (IR) link, which is reported as a side result.The developed methods enable the extraction of more useful information from a radar sensor and extend its application. This brings Doppler radar sensors one step closer to large-scale commercial use for a wide range of applications, including home health monitoring, sleep-time respiration monitoring, and measuring gating signals for medical imaging

A contribution to unobtrusive video-based measurement of respiratory signals

Due to the growing popularity of video-based methods for physiological signal measurement, and taking into account the technological advancements of these type of devices, this work proposes a series of new novel methods to obtain the respiratory signal from a distance, based on video analysis. This thesis aims to improve the state of the art video methods for respiratory measurement, more specifically, by presenting methods that can be used to obtain respiratory variability or perform respiratory rhythm measurements. Moreover, this thesis also aims to present a new implementation of a time-frequency signal processing technique, to improve its computational efficiency when applied to the respiratory signals. In this document a first approach to video-based methods for respiratory signal measurement is performed, to assert the feasibility of using a consumer-grade camera, not only to measure the mean respiratory rate or frequency, but to assert if this hardware could be used to acquire the raw respiratory signal and the respiratory rhythm as well. In this regard a new video-based method was introduced that measures the respiratory signal of a subject at a distance, with the aid of a custom pattern placed on the thorax of the subject. Given the results from the first video-based method, a more broad approach was taken by comparing three different types of video hardware, with the aim to characterise if they could be used for respiratory signal acquisition and respiratory variability measurements. The comparative analysis was performed in terms of instantaneous frequency, as it allowed to characterise the methods in terms of respiratory variability and to compare them in the same terms with the reference method. Subsequently, and due to the previous obtained results, a new method was proposed using a stereo depth camera with the aim to tackle the limitations of the previous study. The proposed method uses an hybrid architecture were the synchronized infrared frame and depth point-cloud from the same camera are acquired. The infrared frame is used to detect the movements of the subject inside the scene, and to recompute on demand a region of interest to obtain the respiratory signal from the depth point-cloud. Furthermore, in this study an opportunistic approach is taken in order to process all the obtained data, as it is also the aim of this study to verify if using a more realistic approach to respiratory signal analysis in real-life conditions, would influence the respiratory rhythm measurement. Even though the depth camera method proved reliable in terms of respiratory rhythm measurement, the opportunistic approach relied on visual inspection of the obtained respiratory signal to properly define each piece. For this reason, a quality indicator had to be proposed that could objectively identify whenever a respiratory signal contained errors. Furthermore, from the idea to characterise the movements of a subject, and by changing the measuring point from a frontal to a lateral perspective to avoid most of the occlusions, a new method based on obtaining the movement of the thoraco-abdominal region using dense optical flow was proposed. This method makes us of the phase of the optical flow to obtain the respiratory signal of the subject, while using the modulus to compute a quality index. Finally, regarding the different signal processing methods used in this thesis to obtain the instantaneous frequency, there were none that could perform in real-time, making the analysis of the respiratory variability not possible in real-life systems where the signals have to be processed in a sample by sample basis. For this reason, as a final chapter a new implementation of the synchrosqueezing transform for time-frequency analysis in real-time is proposed, with the aim to provide a new tool for non-contact methods to obtain the variability of the respiratory signal in real-time.A causa de la creixent popularitat en la mesura de senyals fisiològics amb mètodes de vídeo, i tenint en compte els avenços tecnològics d'aquests dispositius, aquesta tesi proposa una sèrie de nous mètodes per tal d'obtenir la respiració a distància mitjançant l'anàlisi de vídeo. Aquesta tesi té com a objectiu millorar l'estat de l'art referent a la mesura de senyal respiratòria mitjançant els mètodes que en ella es descriuen, així com presentar mètodes que puguin ser usats per obtenir la variabilitat o el ritme respiratori. A més, aquesta tesi té com a objectiu presentar una nova implementació d'un mètode de processat de senyal temps-freqüencial, per tal de millorar-ne l'eficiència computacional quant s’aplica a senyals respiratoris. En aquest document, es realitza una primera aproximació a la mesura de senyal respiratòria mitjançant mètodes de vídeo per tal de verificar si és factible utilitzar una càmera de consum, no només per mesurar el senyal respiratori, sinó verificar si aquest tipus de hardware també pot ser emprat per obtenir el ritme respiratori. En aquest sentit, es presenta en aquest document un nou mètode d'adquisició de senyal respiratòria a distància basat en vídeo, el qual fa ús d'un patró ubicat al tòrax del subjecte per tal d'obtenir-ne la respiració. Un cop obtinguts els resultats del primers resultats, s'han analitzat tres tipus diferents de càmeres, amb la finalitat de caracteritzar-ne la viabilitat d'obtenir el senyal respiratori i la seva variabilitat. L'estudi comparatiu s'ha realitzat en termes de freqüència instantània, donat que permet caracteritzar els mètodes en termes de variabilitat respiratòria i comparar-los, en les mateixes condicions, amb el mètode de referencia. A continuació, s'ha presentat un nou mètode basat en una càmera de profunditat estèreo amb la finalitat de millorar i corregir les limitacions anteriors. El nou mètode proposat es basa en una arquitectura hibrida la qual utilitza els canals de vídeo infraroig i de profunditat de forma sincronitzada. El canal infraroig s'utilitza per detectar els moviments del subjecte dins l'escena i calcular, sota demanda, una regió d'interès que s'utilitza posteriorment en el canal de profunditat per extreure el senyal respiratori. A més a més, en aquest estudi s'ha utilitzat una aproximació oportunista en el processat del senyal respiratori, donat que també és un dels objectius d'aquest estudi, verificar si el fet d'utilitzar una aproximació més realista en l'adquisició de senyal, pot influir en la mesura del ritme respiratori. Tot i que el mètode anterior es mostra fiable en termes de mesura del ritme respiratori, la selecció oportunista del senyal necessita d’inspecció visual per tal de definir correctament cada fragment. Per aquest motiu, era necessari definir un índex de qualitat el qual permetés identificar de forma objectiva cada tram de senyal, així com detectar si el senyal conté errors. Partint de la idea de caracteritzar el moviment del subjecte de l'estudi anterior, i modificant el punt de mesura frontal cap a un de lateral per tal d'evitar oclusions, es proposa un nou mètode basat en l'obtenció del moviment toràcic-abdominal a partir del flux òptic del senyal de vídeo. Aquest mètode recupera el senyal respiratori del subjecte a partir de la fase del flux òptic, tot calculant un índex de qualitat a partir del mòdul. Finalment, i tenint en compte els diferents mètodes de processat utilitzats en aquesta tesi per tal de obtenir la freqüència instantània, es pot apreciar que cap d'ells és capaç de funcionar en temps real, fent inviable l'anàlisi de la variabilitat respiratòria en sistemes reals amb processat mostra a mostra. Per aquest motiu, en el capítol final d'aquesta tesi, s'ha proposat una nova implementació de la transformació "synchrosqueezing" per tal de realitzar l’anàlisi temporal-freqüencial en temps real, i proveir d'una nova eina per tal d'obtenir la variabilitat respiratòria en temps real, amb mètodes sense contacte

Jatkuva-aikainen vitaalielintoimintojen monitorointi pienillä lapsilla käyttäen rinnalle asetettavaa sensoria

Tiivistelmä. Tausta: Hengitystaajuus, sydämen syketaajuus ja happisaturaatio (SpO₂) ovat verenpaineen lisäksi kliinisesti tärkeitä indikaattoreita vastasyntyneiden ja kriittisesti sairaiden potilaiden tilan arvioimisessa ja ennustamisessa. Kliinisessä käytössä ei kuitenkaan ole yhdellä sensorilla toteutettavaa jatkuva-aikaista monitorointimenetelmää, joka avulla samanaikainen vitaalielintoimintojen seuranta olisi helposti ja nopeasti aloitettavissa. Työn tarkoitus: Tutkimuksen tavoitteena oli testata ja validoida mittausmenetelmää, jolla mitataan rintakehältä ei-invasiivisesti hengitys- ja syketaajuutta sekä happisaturaatiota. Tässä työssä rintakehälle asetettiin kiihtyvyysanturin lisäksi optinen happisaturaatioanturi sekä testattiin mittausmenetelmän soveltuvuutta erityisesti pienten lasten monitorointiin. Toisen kaulalle sijoitettavan kolmiaksiaalisen kiihtyvyysanturin avulla voidaan estimoida myöhemmin pulssin kulkuaikasignaalia. Tutkimuksen avulla myös pilotoidaan osa laajemman kliinisen tutkimuksen mittausasetelmasta. Menetelmät: Tutkimuksen aineisto kerättiin kotioloissa tehdyillä mittauksilla. Tutkimusjoukko koostui seitsemästä perusterveestä pienestä lapsesta (keskiarvo 2,7 v ja keskihajonta 2,1 v). Mittauksissa kerättiin signaalia rintakehältä sekä optisella anturilla että kiihtyvyysanturilla. Lisäksi kiihtyvyyssignaalia mitattiin kaulalle sijoitettavalla kiihtyvyysanturilla. Raakasignaalin laatua ja mittausasetelman käytettävyyttä arvioitiin eri mittaustilanteissa ja signaalinkäsittelyllä mittaussignaaleista estimoitiin hengitys- ja syketaajuus. Pilotoitavaa tutkimusta varten rakennettiin myös käyttöliittymä, johon yhdistettiin videokuvan tallennus synkronoidusti mittaussignaalien kanssa sekä suunniteltiin mittausasetelma. Tulokset: Mittausmenetelmän todettiin soveltuvan pienten lasten monitorointiin ja mittaukset suoritettiin onnistuneesti eri ikäisiltä lapsilta. Signaalien laatu eri mittaustilanteissa oli hyvää signaalinkäsittelyä varten lähes kaikissa mittauksissa, jolloin niistä voitiin laskea hengitys- ja syketaajuus sekä happisaturaatio. Signaalinkäsittely vaatii kuitenkin lisää kehittämistä, jotta vitaalielintoiminnoista kertovat parametrit ovat tarkkoja ja suodattavat liikehdinnästä aiheutuvat artefaktat riittävän hyvin. Tässä tutkimuksessa saadut happisaturaatioarvot eivät myöskään olleet uskottavia. Mittausasetelma kokonaisuudessaan oli käytännöllisesti toteutettavissa pienten lasten jatkuva-aikaiseen monitorointiin kotioloissa. Johtopäätökset: Käytettyä menetelmää on aiemmin testattu onnistuneesti mitattaessa vitaalielintoimintoja aikuisilta (Myllylä et al., 2017). Tähän tutkimukseen perustuen hengitystä, sydämen sykettä ja happisaturaatiota on mahdollista mitata helposti myös pieniltä lapsilta, mutta mittausmenetelmän validoinnille on vielä tarvetta. Jatkotutkimuksessa mittausmenetelmää soveltuvuutta tutkitaan keskosten monitoroinnissa sekä tutkitaan vastasyntyneiden kotimonitoroinnin mahdollisuuksia sähköisiin terveydenhuoltopalveluihin yhdistettynä.Continuous monitoring of vital functions in small children using wearable sensor on the chest. Abstract. Background: Respiratory rate, heart rate, and oxygen saturation (SpO₂) are important parameters for vital functions providing important information to identify a critically ill patient. However, neither in clinical nor in home use, there is a single sensor available that can monitor all these vital signals simultaneously and reliable. Objective: The aim of the study was to test and validate a measurement method for measuring respiratory rate, heart rate and oxygen saturation non-invasively on the chest. In this work, in addition to the accelerometer, an optical oxygen saturation sensor was placed on the chest and the applicability of the measurement method was tested, especially for monitoring of small children. The second tri-axial accelerometer located on the neck can be used to estimate the pulse transit time signal. The study also piloted part of a broader clinical trial measurement setup. Methods: The research material was collected by home measurements. The study group consisted of seven healthy small children (mean age 2,7 y and standard deviation 2,1 y). Measurements were taken from the chest using both an optical sensor and an accelerometer. In addition, the accelerometer was placed on the neck. The quality of the raw signal and the usability of the measurement setup were evaluated in different measurement situations, and the respiratory rate, heart rate and oxygen saturation were estimated from the measurement signals by signal processing. The user interface was also designed for a pilot study, which will combine video recording in synchronization with measurement signals. Results: The measurement method was found to be suitable for monitoring small children and measurements were successfully performed on children of different ages. The raw signal quality in various measurement situations was acceptable in almost all cases so that respiratory and heart rates as well as SpO₂ could be calculated. However, signal processing requires further development in order to ensure sufficiently accurate data also during movements of children which seemed to cause great deal of artefacts. Also, the oxygen saturation values were not credible. The whole measurement setup was practically feasible for continuous monitoring of small children at home. Conclusion: Based on this study, respiration, heart rate, and SpO₂ can be easily measured, even from small children, but there is still a need for further validation. In the follow-up study, the measurement method will be used also to monitor premature infants in hospital, as well as to explore the potential of home monitoring of small children and newborns in combination with eHealth services