Charlie: A New Robot Prototype for Improving Communication and social Skills in Children with Autism and a New Single-point Infrared Sensor Technique for Detecting bBeathing and Heart Rate Remotely

This research delivers a new, interactive game-playing robot named CHARLIE and a novel technique for remotely detecting breathing and heart rate using a single-point, thermal infrared sensor (IR). The robot is equipped with a head and two arms, each with two degrees of freedom, and a camera. We trained a human hands classifier and used this classifier along with a standard face classifier to create two autonomous interactive games: single-player ( Imitate Me, Imitate You ) and two-player ( Pass the Pose ). Further, we developed and implemented a suite of new interactive games in which the robot is teleoperated by remote control. Each of these features has been tested and validated through a field study including eight children diagnosed with autism and speech delays. Results from that study show that significant improvements in speech and social skills can be obtained when using CHARLIE with the methodology described herein. Moreover, gains in communication and social interaction are observed to generalize from child-to-robot to co-present others through the scaffolding of communication skills with the systematic approach developed for the study. Additionally, we present a new IR system that continuously targets the sub-nasal region of the face and measures subtle temperature changes corresponding to breathing and cardiac pulse. This research makes four novel contributions: (1) A low-cost, field-tested robot for use in autism therapy, (2) a suite of interactive robot games, (3) a hand classifier created for performing hand detection during the interactive games, and (4) an IR sensor system which remotely collects temperatures and computes breathing and heart rate. Interactive robot CHARLIE is physically designed to be aesthetically appealing to young children between three and six years of age. The hard, wood and metal robot body is covered with a bright green, fuzzy material and additional padding so that it appears toylike and soft. Additionally, several structural features were included to ensure safety during interactive play and to enhance the robustness of the robot. Because children with autism spectrum disorder (ASD) often enjoy exploring new or interesting objects with their hands, the robot must be able to withstand a moderate amount of physical manipulation without causing injury to the child or damaging the robot or its components. CHARLIE plays five distinct interactive games that are designed to be entertaining to young children, appeal to children of varying developmental ability and promote increased speech and social skill through imitation and turn-taking. Remote breathing and heart rate detection Stress is a compounding factor in autism therapy which can inhibit progress toward specific therapeutic goals. The ability to non-invasively detect physical indicators of increasing stress, especially when they can be correlated to specific activities and measured in terms of length and frequency, can relay important metrics about the antecedents that cause stress for a particular child and can be used to help automate the evaluation of a child\u27s progress between sessions. Further, collecting and measuring critical physiological indicators such as breathing and heart rate can enable robots to adjust their behavior based on the perceived emotional, psychological or physical state of their user. The utility and acceptance of robots can be further increased when they are able to learn typical physiological patterns and use these patterns as a baseline for identifying anomalies or possible warning signs of various problems in their human users. We present a new technique for remotely collecting and analyzing breathing and heart rates in real time using an autonomous, low cost infrared (IR) sensor system. This is accomplished by continuously targeting a high precision IR sensor, tracking changes in the sub-nasal skin surface temperature and employing a sinusoidal curve-fitting function, Fast Fourier Transform (FFT), and Discrete Wavelet Transform (DWT) to extract the breathing and heart rate from recorded temperatures

Advanced sensors technology survey

This project assesses the state-of-the-art in advanced or 'smart' sensors technology for NASA Life Sciences research applications with an emphasis on those sensors with potential applications on the space station freedom (SSF). The objectives are: (1) to conduct literature reviews on relevant advanced sensor technology; (2) to interview various scientists and engineers in industry, academia, and government who are knowledgeable on this topic; (3) to provide viewpoints and opinions regarding the potential applications of this technology on the SSF; and (4) to provide summary charts of relevant technologies and centers where these technologies are being developed

The application of remote sensing techniques: Technical and methodological issues

Capabilities and limitations of modern imaging electromagnetic sensor systems are outlined, and the products of such systems are compared with those of the traditional aerial photographic system. Focus is given to the interface between the rapidly developing remote sensing technology and the information needs of operational agencies, and communication gaps are shown to retard early adoption of the technology by these agencies. An assessment is made of the current status of imaging remote sensors and their potential for the future. Public sources of remote sensor data and several cost comparisons are included

Real-time human ambulation, activity, and physiological monitoring:taxonomy of issues, techniques, applications, challenges and limitations

Automated methods of real-time, unobtrusive, human ambulation, activity, and wellness monitoring and data analysis using various algorithmic techniques have been subjects of intense research. The general aim is to devise effective means of addressing the demands of assisted living, rehabilitation, and clinical observation and assessment through sensor-based monitoring. The research studies have resulted in a large amount of literature. This paper presents a holistic articulation of the research studies and offers comprehensive insights along four main axes: distribution of existing studies; monitoring device framework and sensor types; data collection, processing and analysis; and applications, limitations and challenges. The aim is to present a systematic and most complete study of literature in the area in order to identify research gaps and prioritize future research directions

An inclusive survey of contactless wireless sensing: a technology used for remotely monitoring vital signs has the potential to combating COVID-19

With the Coronavirus pandemic showing no signs of abating, companies and governments around the world are spending millions of dollars to develop contactless sensor technologies that minimize the need for physical interactions between the patient and healthcare providers. As a result, healthcare research studies are rapidly progressing towards discovering innovative contactless technologies, especially for infants and elderly people who are suffering from chronic diseases that require continuous, real-time control, and monitoring. The fusion between sensing technology and wireless communication has emerged as a strong research candidate choice because wearing sensor devices is not desirable by patients as they cause anxiety and discomfort. Furthermore, physical contact exacerbates the spread of contagious diseases which may lead to catastrophic consequences. For this reason, research has gone towards sensor-less or contactless technology, through sending wireless signals, then analyzing and processing the reflected signals using special techniques such as frequency modulated continuous wave (FMCW) or channel state information (CSI). Therefore, it becomes easy to monitor and measure the subject’s vital signs remotely without physical contact or asking them to wear sensor devices. In this paper, we overview and explore state-of-the-art research in the field of contactless sensor technology in medicine, where we explain, summarize, and classify a plethora of contactless sensor technologies and techniques with the highest impact on contactless healthcare. Moreover, we overview the enabling hardware technologies as well as discuss the main challenges faced by these systems.This work is funded by the scientific and technological research council of Turkey (TÜBITAK) under grand 119E39

Estimating heart rate via depth video motion tracking

Depth sensors like Microsoft Kinect can acquire partial geometric information in a 3D scene via captured depth images, with potential application to non-contact health monitoring. However, captured depth videos typically suffer from low bit-depth representation and acquisition noise corruption, and hence using them to deduce health metrics that require tracking subtle 3D structural details is difficult. In this paper, we propose to capture depth video using Kinect 2.0 to estimate the heart rate of a human subject; as blood is pumped to circulate through the head, tiny oscillatory head motion can be detected for periodicity analysis. Specifically, we first perform a joint bit-depth enhancement / denoising procedure to improve the quality of the captured depth images, using a graph-signal smoothness prior for regularization. We then track an automatically detected nose region throughout the depth video to deduce 3D motion vectors. The deduced 3D vectors are then analyzed via principal component analysis to estimate heart rate. Experimental results show improved tracking accuracy using our proposed joint bit-depth enhancement / denoising procedure, and estimated heart rates are close to ground truth

Physiological sensor.

For the purpose of medical diagnosis and research, various physiological signals, such as Photoplethysmograph (PPG), Galvanic Skin Response (GSR) and Skin Temperature (SKT), are measured by different types of medical sensor equipment. However, the sensors are intrusive and the patients must endure some discomfort when encountering these types of medical sensor equipment. These bio sensors also fail to be implemented outside of lab or clinical settings. Recently the development in sensor technology and wireless communication technology have significantly improved the integration of wearable systems, so that we could find new ways to minimize the wearable circuits module, design layers of fabric for wearable system. This thesis documents the successful development of a novel, unobtrusive, low-cost, wrist-worn integrated sensors (PPG, GSR, SKT) system using wireless wearable technology capable of measuring real-time data collection, and monitoring which is important when dealing with treatment and management of many chronic illnesses, neurological disorders, and mental health issues. Examples can include: epileptic seizures, autism spectrum disorder (ASD), depression, drug addiction, and anxiety disorders

Estimating heart rate and rhythm via 3D motion tracking in depth video

Low-cost depth sensors, such as Microsoft Kinect, have potential for non-intrusive, non-contact health monitoring that is robust to ambient lighting conditions. However, captured depth images typically suer from low bit-depth and high acquisition noise, and hence processing them to estimate biometrics is dicult. In this paper, we propose to capture depth video of a human subject using Kinect 2.0 to estimate his/her heart rate and rhythm (regularity); as blood is pumped from the heart to circulate through the head, tiny oscillatory head motion due to Newtonian mechanics can be detected for periodicity analysis. Specifically, we first restore a captured depth video via a joint bit-depth enhancement / denoising procedure, using a graph-signal smoothness prior for regularization. Second, we track an automatically detected head region throughout the depth video to deduce 3D motion vectors. The detected vectors are fed back to the depth restoration module in a loop to ensure that the motion information in two modules are consistent, improving performance of both restoration and motion tracking in the process. Third, the computed 3D motion vectors are projected onto its principal component for 1D signal analysis, composed of trend removal, band-pass filtering, and wavelet-based motion denoising. Finally, the heart rate is estimated via Welch power spectrum analysis, and the heart rhythm is computed via peak detection. Experimental results show accurate estimation of the heart rate and rhythm using our proposed algorithm as compared to rate and rhythm estimated by a portable oximeter

Design study for Thermal Infrared Multispectral Scanner (TIMS)

The feasibility of dividing the 8-12 micrometer thermal infrared wavelength region into six spectral bands by an airborne line scanner system was investigated. By combining an existing scanner design with a 6 band spectrometer, a system for the remote sensing of Earth resources was developed. The elements in the spectrometer include an off axis reflective collimator, a reflective diffraction grating, a triplet germanium imaging lens, a photoconductive mercury cadmium telluride sensor array, and the mechanical assembly to hold these parts and maintain their optical alignment across a broad temperature range. The existing scanner design was modified to accept the new spectrometer and two field filling thermal reference sources