Thermal imaging developments for respiratory airflow measurement to diagnose apnoea

Sleep-disordered breathing is a sleep disorder that manifests itself as intermittent pauses (apnoeas) in breathing during sleep. The condition disturbs the sleep and can results in a variety of health problems. Its diagnosis is complex and involves multiple sensors attached to the person to measure electroencephalogram (EEG), electrocardiogram (ECG), blood oxygen saturation (pulse oximetry, S

Charlie: A New Robot Prototype for Improving Communication and social Skills in Children with Autism and a New Single-point Infrared Sensor Technique for Detecting bBeathing and Heart Rate Remotely

This research delivers a new, interactive game-playing robot named CHARLIE and a novel technique for remotely detecting breathing and heart rate using a single-point, thermal infrared sensor (IR). The robot is equipped with a head and two arms, each with two degrees of freedom, and a camera. We trained a human hands classifier and used this classifier along with a standard face classifier to create two autonomous interactive games: single-player ( Imitate Me, Imitate You ) and two-player ( Pass the Pose ). Further, we developed and implemented a suite of new interactive games in which the robot is teleoperated by remote control. Each of these features has been tested and validated through a field study including eight children diagnosed with autism and speech delays. Results from that study show that significant improvements in speech and social skills can be obtained when using CHARLIE with the methodology described herein. Moreover, gains in communication and social interaction are observed to generalize from child-to-robot to co-present others through the scaffolding of communication skills with the systematic approach developed for the study. Additionally, we present a new IR system that continuously targets the sub-nasal region of the face and measures subtle temperature changes corresponding to breathing and cardiac pulse. This research makes four novel contributions: (1) A low-cost, field-tested robot for use in autism therapy, (2) a suite of interactive robot games, (3) a hand classifier created for performing hand detection during the interactive games, and (4) an IR sensor system which remotely collects temperatures and computes breathing and heart rate. Interactive robot CHARLIE is physically designed to be aesthetically appealing to young children between three and six years of age. The hard, wood and metal robot body is covered with a bright green, fuzzy material and additional padding so that it appears toylike and soft. Additionally, several structural features were included to ensure safety during interactive play and to enhance the robustness of the robot. Because children with autism spectrum disorder (ASD) often enjoy exploring new or interesting objects with their hands, the robot must be able to withstand a moderate amount of physical manipulation without causing injury to the child or damaging the robot or its components. CHARLIE plays five distinct interactive games that are designed to be entertaining to young children, appeal to children of varying developmental ability and promote increased speech and social skill through imitation and turn-taking. Remote breathing and heart rate detection Stress is a compounding factor in autism therapy which can inhibit progress toward specific therapeutic goals. The ability to non-invasively detect physical indicators of increasing stress, especially when they can be correlated to specific activities and measured in terms of length and frequency, can relay important metrics about the antecedents that cause stress for a particular child and can be used to help automate the evaluation of a child\u27s progress between sessions. Further, collecting and measuring critical physiological indicators such as breathing and heart rate can enable robots to adjust their behavior based on the perceived emotional, psychological or physical state of their user. The utility and acceptance of robots can be further increased when they are able to learn typical physiological patterns and use these patterns as a baseline for identifying anomalies or possible warning signs of various problems in their human users. We present a new technique for remotely collecting and analyzing breathing and heart rates in real time using an autonomous, low cost infrared (IR) sensor system. This is accomplished by continuously targeting a high precision IR sensor, tracking changes in the sub-nasal skin surface temperature and employing a sinusoidal curve-fitting function, Fast Fourier Transform (FFT), and Discrete Wavelet Transform (DWT) to extract the breathing and heart rate from recorded temperatures

The Role of Edge Robotics As-a-Service in Monitoring COVID-19 Infection

Deep learning technology has been widely used in edge computing. However, pandemics like covid-19 require deep learning capabilities at mobile devices (detect respiratory rate using mobile robotics or conduct CT scan using a mobile scanner), which are severely constrained by the limited storage and computation resources at the device level. To solve this problem, we propose a three-tier architecture, including robot layers, edge layers, and cloud layers. We adopt this architecture to design a non-contact respiratory monitoring system to break down respiratory rate calculation tasks. Experimental results of respiratory rate monitoring show that the proposed approach in this paper significantly outperforms other approaches. It is supported by computation time costs with 2.26 ms per frame, 27.48 ms per frame, 0.78 seconds for convolution operation, similarity calculation, processing one-minute length respiratory signals, respectively. And the computation time costs of our three-tier architecture are less than that of edge+cloud architecture and cloud architecture. Moreover, we use our three-tire architecture for CT image diagnosis task decomposition. The evaluation of a CT image dataset of COVID-19 proves that our three-tire architecture is useful for resolving tasks on deep learning networks by edge equipment. There are broad application scenarios in smart hospitals in the future

Human Recognition from Video Sequences and Off-Angle Face Images Supported by Respiration Signatures

In this work, we study the problem of human identity recognition using human respiratory waveforms extracted from videos combined with component-based off- angle human facial images. Our proposed system is composed of (i) a physiology- based human clustering module and (ii) an identification module based on facial features (nose, mouth, etc.) fetched from face videos. In our proposed methodology we, first, manage to passively extract an important vital sign (breath), cluster human subjects into nostril motion vs. nostril non-motion groups, and, then, localize a set of facial features, before we apply feature extraction and matching.;Our novel human identity recognition system is very robust, since it is working well when dealing with breath signals and a combination of different facial components acquired in uncontrolled luminous conditions. This is achieved by using our proposed Motion Classification approach and Feature Clustering technique based on the breathing waveforms we produce. The contributions of this work are three-fold. First, we collected a set of different datasets where we tested our proposed approach. Specifically, we considered six different types of facial components and their combination, to generate face-based video datasets, which present two diverse data collection conditions, i.e. videos acquired in head fully frontal position (baseline) and head looking up pose. Second, we propose a new way of passively measuring human breath from face videos and show comparatively identical output against baseline breathing waveforms produced by an ADInstruments device. Third, we demonstrate good human recognition performance when using the pro- posed pre-processing procedure of Motion Classification and Feature Clustering, working on partial features of human faces.;Our method achieves increased identification rates across all datasets used, and it manages to obtain a significantly high identification rate (ranging from 96%-100% when using a single or a combination of facial features), yielding an average of 7% raise, when compared to the baseline scenario. To the best of our knowledge, this is the first time that a biometric system is composed of an important human vital sign (breath) that is fused with facial features is such an efficient manner

Mapping of technologies using thermal images to control epidemics

The quest to combat the spread of the new Corona Virus Pandemic is a battle experienced worldwide, more specifically in the year 2020 when it caused a tragedy in the lives of a large part of the world population. The current numbers of contaminated people and deaths are alarming. Transmitted through droplets expelled through the nose or mouth, it leads to fever, which is the most common symptom of COVID-19. A technique that uses thermal images to check dispersed heat is a thermography. These images are captured by thermal cameras or devices with temperature sensors. Thus, the purpose of this work was to map the deposits of patent applications in order to seek technologies related to the use of thermal images to control the pandemic. The search base chosen for this research characterized as exploratory quantitative was Espacenet, which returned a final result of 119 published patent documents. Of these 93 documents were worked on in this article which gave us a more discussed result, since the others were repeated. The research revealed that patent applications in this area were stable until the current year when a Corona Virus pandemic spread, forcing researchers to develop research in order to combat it. The increase in the number of patents in 2020 shows the tendency to increase to 2021 when new research should appear and, consequently, new patented documents may be exposed in the future

VOLUNTARY CONTROL OF BREATHING ACCORDING TO THE BREATHING PATTERN DURING LISTENING TO MUSIC AND NON-CONTACT MEASUREMENT OF HEART RATE AND RESPIRATION

We investigated if listening to songs changes breathing pattern which changes autonomic responses such as heart rate (HR) and heart rate variability (HRV) or change in breathing pattern is a byproduct of listening to songs or change in breathing pattern as well as listening to songs causes changes in autonomic responses. Seven subjects (4 males and 3 females) participated in a pilot study where they listened to two types of songs and used a custom developed biofeedback program to control their breathing pattern to match the one recorded during listening to the songs. Coherencies between EEG, breathing pattern and RR intervals (RRI) were calculated to study the interaction with neural responses. Trends in HRV varied only during listening to songs, suggesting that autonomic response was affected by listening to songs irrespective of control of breathing. Effective coherence during songs while spontaneously breathing was more than during silence and during control of breathing. These results, although preliminary, suggest that listening to songs as well as change in breathing patterns changes the autonomic response but the effect of listening to songs may surpass the effect of changes in breathing. We explored feasibility of using non-contact measurements of HR and breathing rate (BR) by using recently developed Facemesh and other methods for tracking regions of interests from videos of faces of subjects. Performance was better for BR than HR, and over currently used methods. However, refinement of the approach would be needed to get the precision required for detecting subtle changes