Vital Sign Monitoring in Dynamic Environment via mmWave Radar and Camera Fusion

Contact-free vital sign monitoring, which uses wireless signals for recognizing human vital signs (i.e, breath and heartbeat), is an attractive solution to health and security. However, the subject's body movement and the change in actual environments can result in inaccurate frequency estimation of heartbeat and respiratory. In this paper, we propose a robust mmWave radar and camera fusion system for monitoring vital signs, which can perform consistently well in dynamic scenarios, e.g., when some people move around the subject to be tracked, or a subject waves his/her arms and marches on the spot. Three major processing modules are developed in the system, to enable robust sensing. Firstly, we utilize a camera to assist a mmWave radar to accurately localize the subjects of interest. Secondly, we exploit the calculated subject position to form transmitting and receiving beamformers, which can improve the reflected power from the targets and weaken the impact of dynamic interference. Thirdly, we propose a weighted multi-channel Variational Mode Decomposition (WMC-VMD) algorithm to separate the weak vital sign signals from the dynamic ones due to subject's body movement. Experimental results show that, the 90${^{th}}$ percentile errors in respiration rate (RR) and heartbeat rate (HR) are less than 0.5 RPM (respirations per minute) and 6 BPM (beats per minute), respectively

Matrix pencil method for vital sign detection from signals acquired by microwave sensors

Microwave sensors have recently been introduced as high-temporal resolution sensors, which could be used in the contactless monitoring of artery pulsation and breathing. However, accurate and efficient signal processing methods are still required. In this paper, the matrix pencil method (MPM), as an efficient method with good frequency resolution, is applied to back-reflected microwave signals to extract vital signs. It is shown that decomposing of the signal to its damping exponentials fulfilled by MPM gives the opportunity to separate signals, e.g., breathing and heartbeat, with high precision. A publicly online dataset (GUARDIAN), obtained by a continuous wave microwave sensor, is applied to evaluate the performance of MPM. Two methods of bandpass filtering (BPF) and variational mode decomposition (VMD) are also implemented. In addition to the GUARDIAN dataset, these methods are also applied to signals acquired by an ultra-wideband (UWB) sensor. It is concluded that when the vital sign is sufficiently strong and pure, all methods, e.g., MPM, VMD, and BPF, are appropriate for vital sign monitoring. However, in noisy cases, MPM has better performance. Therefore, for non-contact microwave vital sign monitoring, which is usually subject to noisy situations, MPM is a powerful method

Vital Signs Monitoring Based On UWB Radar

Contactless detection of human vital sign using radar sensors appears to be a promising technology which integrates communication, biomedicine, computer science etc. The radar-based vital sign detection has been actively investigated in the past decade. Due to the advantages such as wide bandwidth, high resolution, small and portable size etc., ultra-wideband (UWB) radar has received a great deal of attention in the health care field. In this thesis, an X4 series UWB radar developed by Xethru Company is adopted to detect human breathing signals through the radar echo reflected by the chest wall movement caused by breath and heartbeat. The emphasis is placed on the estimation of breathing and heart rate based on several signal processing algorithms. Firstly, the research trend of vital sign detection using radar technology is reviewed, based on which the advantages of contactless detection and UWB radar-based technology are highlighted. Then theoretical basis and core algorithms of radar signals detection are presented. Meanwhile, the detection system based on Xethru UWB radar is introduced. Next, several preprocessing methods including SVD-based clutter and noise removal algorithms, the largest variance-based target detection method, and the autocorrelation-based breathing-like signal identification method are investigated, to extract the significant component containing the vital signs from the received raw radar echo signal. Then the thesis investigates four time-frequency analysis algorithms (fast Fourier transform + band-pass filter (FFT+BPF), empirical mode decomposition (EMD), ensemble empirical mode decomposition (EEMD) and variational mode decomposition (VMD) and compare their performances in estimating breathing rate (BR) and heart rate (HR) in different application scenarios. A python-based vital signs detection system is designed to implement the above-mentioned preprocessing and BR and HR estimation algorithms, based on which a large number of single target experiments are undertaken to evaluate the four estimation algorithms. Specifically, the single target experiments are divided into simple setup and challenging setup. In the simple setup, testees face to radar and keep normal breathing in an almost stationary posture, while in the challenging setup, the testee is allowed to do more actions, such as site sitting, changing the breathing frequency, deep hold the breathing. It is shown that the FFT+BPF algorithm gives the highest accuracy and the fastest calculation speed under the simple setup, while in a challenging setup, the VMD algorithm has the highest accuracy and the widest applicability. Finally, double targets breathing signal detection at different distances to the radar are undertaken, aiming to observe whether the breathing signals of two targets will interfere with each other. We found that when two objects are not located at the same distance to the radar, the object closer to the radar will not affect the breath detection of the object far from the radar. When the two targets are located at the same distance, the 'Shading effect' appears in the two breathing signals and only VMD algorithm can separate the breathing signals of the targets

Multiradar Data Fusion for Respiratory Measurement of Multiple People

This study proposes a data fusion method for multiradar systems to enable measurement of the respiration of multiple people located at arbitrary positions. Using the proposed method, the individual respiration rates of multiple people can be measured, even when echoes from some of these people cannot be received by one of the radar systems because of shadowing. In addition, the proposed method does not require information about the positions and orientations of the radar systems used because the method can estimate the layout of these radar systems by identifying multiple human targets that can be measured from different angles using multiple radar systems. When a single target person can be measured using multiple radar systems simultaneously, the proposed method selects an accurate signal from among the multiple signals based on the spectral characteristics. To verify the effectiveness of the proposed method, we performed experiments based on two scenarios with different layouts that involved seven participants and two radar systems. Through these experiments, the proposed method was demonstrated to be capable of measuring the respiration of all seven people by overcoming the shadowing issue. In the two scenarios, the average errors of the proposed method in estimating the respiration rates were 0.33 and 1.24 respirations per minute (rpm), respectively, thus demonstrating accurate and simultaneous respiratory measurements of multiple people using the multiradar system

Multiradar Data Fusion for Respiratory Measurement of Multiple People

This study proposes a data fusion method for multiradar systems to enable measurement of the respiration of multiple people located at arbitrary positions. Using the proposed method, the individual respiration rates of multiple people can be measured, even when echoes from some of these people cannot be received by one of the radar systems because of shadowing. In addition, the proposed method does not require information about the positions and orientations of the radar systems used because the method can estimate the layout of these radar systems by identifying multiple human targets that can be measured from different angles using multiple radar systems. When a single target person can be measured using multiple radar systems simultaneously, the proposed method selects an accurate signal from among the multiple signals based on the spectral characteristics. To verify the effectiveness of the proposed method, we performed experiments based on two scenarios with different layouts that involved seven participants and two radar systems. Through these experiments, the proposed method was demonstrated to be capable of measuring the respiration of all seven people by overcoming the shadowing issue. In the two scenarios, the average errors of the proposed method in estimating the respiration rates were 0.33 and 1.24 respirations per minute (rpm), respectively, thus demonstrating accurate and simultaneous respiratory measurements of multiple people using the multiradar system

Non-Contact Vital Sign Detection Using mm-Wave Radar

Vital Sign detection using radars has been a rising technology in the fields of healthcare, security, and military purposes. Typically, radars used for these tasks operate at lower frequencies due to their low cost and and the ability to detect behind obstacles, such as walls or undre debris. However, this leads to an overall large system as the lower the frequency of operation, the larger the size of the antennas. The system size increases when multiple antennas are used for subject localization. But, with the development of millimeter- wave radars and Antenna-on-Package (AoP) solutions, a more compact and portable radar is possible. In this thesis, a commercial, compact, and portable millimeter wave radar operating at 60 GHz is used to detect the vital signs of subjects. With the use of direction of arrival, beamforming, and frequency tracking, the millimeter wave radar is able to accurately detect the heart rate and respiration rate of subjects with high accuracy. Experiments are performed involving detection with varying distances, detection through drywall, and for a single or even multiple subjects

Noncontact Respiratory Measurement for Multiple People at Arbitrary Locations Using Array Radar and Respiratory-Space Clustering

We developed a noncontact measurement system for monitoring the respiration of multiple people using millimeter-wave array radar. To separate the radar echoes of multiple people, conventional techniques cluster the radar echoes in the time, frequency, or spatial domain. Focusing on the measurement of the respiratory signals of multiple people, we propose a method called respiratory-space clustering, in which individual differences in the respiratory rate are effectively exploited to accurately resolve the echoes from human bodies. The proposed respiratory-space clustering can separate echoes, even when people are located close to each other. In addition, the proposed method can be applied when the number of targets is unknown and can accurately estimate the number and positions of people. We perform multiple experiments involving five or seven participants to verify the performance of the proposed method, and quantitatively evaluate the estimation accuracy for the number of people and the respiratory intervals. The experimental results show that the average root-mean-square error in estimating the respiratory interval is 196 ms using the proposed method. The use of the proposed method, rather the conventional method, improves the accuracy of the estimation of the number of people by 85.0%, which indicates the effectiveness of the proposed method for the measurement of the respiration of multiple people

Accuracy Assessment and Improvement of FMCW Radar-based Vital Signs Monitoring under Practical Scenarios

Acquisition of human vital signs through radar is a very promising technology that can address the shortcomings of the traditional contact-based measurement devices and enable the move toward a contactless vital monitoring system. This research is focused on monitoring breath rate (BR) and heart rate (HR) via a frequency modulated continuous wave (FMCW) radar. Currently, the two approaches used for BR and HR estimation are filter-based and decomposition-based, such as variational mode decomposition (VMD) for high-quality signal separation. We propose an adaptive VMD (AVMD) to address the problem of setting the number of segmentation levels required by the VMD algorithm. Various experiments are conducted under practical scenarios in terms of distance, angle, posture, and activity as well as the existence of a nearby person and fan. We have made a comprehensive assessment of accuracy change and impact in these scenarios. The experimental results show clearly that the proposed AVMD gives higher accuracy compared to the filter-based and VMD-based. A real-time BR-HR monitoring system using the proposed AVMD and the TI’s IWR1843Boost radar board has been implemented to demonstrate its practical uses

Bio-Radar Applications for Remote Vital Signs Monitoring

Nowadays, most vital signs monitoring techniques used in a medical context and/or daily life routines require direct contact with skin, which can become uncomfortable or even impractical to be used regularly. Radar technology has been appointed as one of the most promising contactless tools to overcome these hurdles. However, there is a lack of studies that cover a comprehensive assessment of this technology when applied in real-world environments. This dissertation aims to study radar technology for remote vital signs monitoring, more specifically, in respiratory and heartbeat sensing. Two off-the-shelf radars, based on impulse radio ultra-wideband and frequency modu lated continuous wave technology, were customized to be used in a small proof of concept experiment with 10 healthy participants. Each subject was monitored with both radars at three different distances for two distinct conditions: breathing and voluntary apnea. Signals processing algorithms were developed to detect and estimate respiratory and heartbeat parameters, assessed using qualitative and quantitative methods. Concerning respiration, a minimum error of 1.6% was found when radar respiratory peaks signals were directly compared with their reference, whereas a minimum mean absolute error of 0.3 RPM was obtained for the respiration rate. Concerning heartbeats, their expression in radar signals was not as clear as the respiration ones, however a minimum mean absolute error of 1.8 BPM for heartbeat was achieved after applying a novel selective algorithm developed to validate if heart rate value was estimated with reliability. The results proved the potential for radars to be used in respiratory and heartbeat contactless sensing, showing that the employed methods can be already used in some mo tionless situations. Notwithstanding, further work is required to improve the developed algorithms in order to obtain more robust and accurate systems.Atualmente, a maioria das técnicas usadas para a monitorização de sinais vitais em contexto médicos e/ou diário requer contacto direto com a pele, o que poderá tornar-se incómodo ou até mesmo inviável em certas situações. A tecnologia radar tem vindo a ser apontada como uma das mais promissoras ferramentas para medição de sinais vitais à distância e sem contacto. Todavia, são necessários mais estudos que permitam avaliar esta tecnologia quando aplicada a situações mais reais. Esta dissertação tem como objetivo o estudo da tecnologia radar aplicada no contexto de medição remota de sinais vitais, mais concretamente, na medição de atividade respiratória e cardíaca. Dois aparelhos radar, baseados em tecnologia banda ultra larga por rádio de impulso e em tecnologia de onda continua modulada por frequência, foram configurados e usados numa prova de conceito com 10 participantes. Cada sujeito foi monitorizado com cada um dos radar em duas situações distintas: respirando e em apneia voluntária. Algorit mos de processamento de sinal foram desenvolvidos para detetar e estimar parâmetros respiratórios e cardíacos, avaliados através de métodos qualitativos e quantitativos. Em relação à respiração, o menor erro obtido foi de 1,6% quando os sinais de radar respiratórios foram comparados diretamente com os sinais de referência, enquanto que, um erro médio absoluto mínimo de 0,3 RPM foi obtido para a estimação da frequência respiratória via radar. A expressão cardíaca nos sinais radar não se revelou tão evidente como a respiratória, no entanto, um erro médio absoluto mínimo de 1,8 BPM foi obtido para a estimação da frequência cardíaca após a aplicação de um novo algoritmo seletivo, desenvolvido para validar a confiança dos valores obtidos. Os resultados obtidos provaram o potencial do uso de radares na medição de atividade respiratória e cardíaca sem contacto, sendo esta tecnologia viável de ser implementada em situações onde não existe muito movimento. Não obstante, os algoritmos desenvolvidos devem ser aperfeiçoados no futuro de forma a obter sistemas mais robustos e precisos