Tracking human face features in thermal images for respiration monitoring

A method has been developed to track a region related to respiration process in thermal images. The respiration region of interest (ROI) consisted of the skin area around the tip of the nose. The method was then used as part of a non-contact respiration rate monitoring that determined the skin temperature changes caused by respiration. The ROI was located by the first determining the relevant salient features of the human face physiology. These features were the warmest and coldest facial points. The tracking method was tested on thermal video images containing no head movements, small random and regular head movements. The method proved valuable for tracking the ROI in all these head movement types. It was also possible to use this tracking method to monitor respiration rate involving a number of head movement types. Currently, more investigations are underway to improve the tracking method so that it can track the ROI in cases larger head movements

Automatic method for detection of characteristic areas in thermal face images

The use of thermal images of a selected area of the head in screening systems, which perform fast and accurate analysis of the temperature distribution of individual areas, requires the use of profiled image analysis methods. There exist methods for automated face analysis which are used at airports or train stations and are designed to detect people with fever. However, they do not enable automatic separation of specific areas of the face. This paper presents an algorithm for image analysis which enables localization of characteristic areas of the face in thermograms. The algorithm is resistant to subjects’ variability and also to changes in the position and orientation of the head. In addition, an attempt was made to eliminate the impact of background and interference caused by hair and hairline. The algorithm automatically adjusts its operation parameters to suit the prevailing room conditions. Compared to previous studies (Marzec et al., J Med Inform Tech 16:151–159, 2010), the set of thermal images was expanded by 34 images. As a result, the research material was a total of 125 patients’ thermograms performed in the Department of Pediatrics and Child and Adolescent Neurology in Katowice, Poland. The images were taken interchangeably with several thermal cameras: AGEMA 590 PAL (sensitivity of 0.1 °C), ThermaCam S65 (sensitivity of 0.08 °C), A310 (sensitivity of 0.05 °C), T335 (sensitivity of 0.05 °C) with a 320×240 pixel optical resolution of detectors, maintaining the principles related to taking thermal images for medical thermography. In comparison to (Marzec et al., J Med Inform Tech 16:151–159, 2010), the approach presented there has been extended and modified. Based on the comparison with other methods presented in the literature, it was demonstrated that this method is more complex as it enables to determine the approximate areas of selected parts of the face including anthropometry. As a result of this comparison, better results were obtained in terms of localization accuracy of the center of the eye sockets and nostrils, giving an accuracy of 87 % for the eyes and 93 % for the nostrils

“Design, Development and Characterization of a Thermal Sensor Brick System for Modular Robotics

This thesis presents the work on thermal imaging sensor brick (TISB) system for modular robotics. The research demonstrates the design, development and characterization of the TISB system. The TISB system is based on the design philosophy of sensor bricks for modular robotics. In under vehicle surveillance for threat detection, which is a target application of this work we have demonstrated the advantages of the TISB system over purely vision-based systems. We have highlighted the advantages of the TISB system as an illumination invariant threat detection system for detecting hidden threat objects in the undercarriage of a car. We have compared the TISB system to the vision sensor brick system and the mirror on a stick. We have also illustrated the operational capability of the system on the SafeBot under vehicle robot to acquire and transmit the data wirelessly. The early designs of the TISB system, the evolution of the designs and the uniformity achieved while maintaining the modularity in building the different sensor bricks; the visual, the thermal and the range sensor brick is presented as part of this work. Each of these sensor brick systems designed and implemented at the Imaging Robotics and Intelligent Systems (IRIS) laboratory consist of four major blocks: Sensing and Image Acquisition Block, Pre-Processing and Fusion Block, Communication Block, and Power Block. The Sensing and Image Acquisition Block is to capture images or acquire data. The Pre-Processing and Fusion Block is to work on the acquired images or data. The Communication Block is for transferring data between the sensor brick and the remote host computer. The Power Block is to maintain power supply to the entire brick. The modular sensor bricks are self-sufficient plug and play systems. The SafeBot under vehicle robot designed and implemented at the IRIS laboratory has two tracked platforms one on each side with a payload bay area in the middle. Each of these tracked platforms is a mobility brick based on the same design philosophy as the modular sensor bricks. The robot can carry one brick at a time or even multiple bricks at the same time. The contributions of this thesis are: (1) designing and developing the hardware implementation of the TISB system, (2) designing and developing the software for the TISB system, and (3) characterizing the TISB system, where this characterization of the system is the major contribution of this thesis. The analysis of the thermal sensor brick system provides the user and future designers with sufficient information on parameters to be considered to make the right choice for future modifications, the kind of applications the TISB could handle and the load that the different blocks of the TISB system could manage. Under vehicle surveillance for threat detection, perimeter / area surveillance, scouting, and improvised explosive device (IED) detection using a car-mounted system are some of the applications that have been identified for this system

Object recognition in infrared imagery using appearance-based methods

Abstract unavailable please refer to PD

Adaptive detection and tracking using multimodal information

This thesis describes work on fusing data from multiple sources of information, and focuses on two main areas: adaptive detection and adaptive object tracking in automated vision scenarios. The work on adaptive object detection explores a new paradigm in dynamic parameter selection, by selecting thresholds for object detection to maximise agreement between pairs of sources. Object tracking, a complementary technique to object detection, is also explored in a multi-source context and an efficient framework for robust tracking, termed the Spatiogram Bank tracker, is proposed as a means to overcome the difficulties of traditional histogram tracking. As well as performing theoretical analysis of the proposed methods, specific example applications are given for both the detection and the tracking aspects, using thermal infrared and visible spectrum video data, as well as other multi-modal information sources

UNOBTRUSIVE Technique Based On Infrared Thermal Imaging For Emotion Recognition In Children- With-asd- Robot Interaction

Emoções são relevantes para as relações sociais, e indivíduos com Transtorno do Espectro Autista (TEA) possuem compreensão e expressão de emoções prejudicadas. Esta tese consiste em estudos sobre a análise de emoções em crianças com desenvolvimento típico e crianças com TEA (idade entre 7 e 12 anos), por meio do imageamento térmico infravermelho (ITIV), uma técnica segura e não obtrusiva (isenta de contato), usada para registrar variações de temperatura em regiões de interesse (RIs) da face, tais como testa, nariz, bochechas, queixo e regiões periorbital e perinasal. Um robô social chamado N-MARIA (Novo-Robô Autônomo Móvel para Interação com Autistas) foi usado como estímulo emocional e mediador de tarefas sociais e pedagógicas. O primeiro estudo avaliou a variação térmica facial para cinco emoções (alegria, tristeza, medo, nojo e surpresa), desencadeadas por estímulos audiovisuais afetivos, em crianças com desenvolvimento típico. O segundo estudo avaliou a variação térmica facial para três emoções (alegria, surpresa e medo), desencadeadas pelo robô social N-MARIA, em crianças com desenvolvimento típico. No terceiro estudo, duas sessões foram realizadas com crianças com TEA, nas quais tarefas sociais e pedagógicas foram avaliadas tendo o robô N-MARIA como ferramenta e mediador da interação com as crianças. Uma análise emocional por variação térmica da face foi possível na segunda sessão, na qual o robô foi o estímulo para desencadear alegria, surpresa ou medo. Além disso, profissionais (professores, terapeuta ocupacional e psicóloga) avaliaram a usabilidade do robô social. Em geral, os resultados mostraram que o ITIV foi uma técnica eficiente para avaliar as emoções por meio de variações térmicas. No primeiro estudo, predominantes decréscimos térmicos foram observados na maioria das RIs, com as maiores variações de emissividade induzidas pelo nojo, felicidade e surpresa, e uma precisão maior que 85% para a classificação das cinco emoções. No segundo estudo, as maiores probabilidades de emoções detectadas pelo sistema de classificação foram para surpresa e alegria, e um aumento significativo de temperatura foi predominante no queixo e nariz. O terceiro estudo realizado com crianças com TEA encontrou aumentos térmicos significativos em todas as RIs e uma classificação com a maior probabilidade para surpresa. N-MARIA foi um estímulo promissor capaz de desencadear emoções positivas em crianças. A interação criança-com-TEA-e-robô foi positiva, com habilidades sociais e tarefas pedagógicas desempenhadas com sucesso pelas crianças. Além disso, a usabilidade do robô avaliada por profissionais alcançou pontuação satisfatória, indicando a N-MARIA como uma potencial ferramenta para terapias

Cluster-analytic classification of facial expressions using infrared measurements of facial thermal features

In previous research, scientists were able to use transient facial thermal features extracted from Thermal Infra-Red Images (TIRIs) for making binary distinction between the affective states. For example, thermal asymmetries localised in facial TIRIs have been used to distinguish anxiety and deceit. Since affective human-computer interaction would require machines to distinguish between the subtle facial expressions of affective states, computers’ able to make such binary distinctions would not suffice a robust human-computer interaction. This work, for the first time, uses affective-state-specific transient facial thermal features extracted from TIRIs to recognise a much wider range of facial expressions under a much wider range of conditions. Using infrared thermal imaging within the 8-14 μm, a database of 324 discrete, time-sequential, visible-spectrum and thermal facial images was acquired, representing different facial expressions from 23 participants in different situations. A facial thermal feature extraction and pattern classification approach was developed, refined and tested on various Gaussian mixture models constructed using the image database. Attempts were made to classify: neutral and pretended happy and sad faces; multiple positive and negative facial expressions; six (pretended) basic facial expressions; partially covered or occluded faces; and faces with evoked happiness, sadness, disgust and anger. The cluster-analytic classification in this work began by segmentation and detection of thermal faces in the acquired TIRIs. The affective-state-specific temperature distributions on the facial skin surface were realised through the pixel grey-level analysis. Examining the affectivestate- specific temperature variations within the selected regions of interest in the TIRIs led to the discovery of some significant Facial Thermal Feature Points (FTFPs) along the major facial muscles. Following a multivariate analysis of the Thermal Intensity values (TIVs) measured at the FTFPs, the TIRIs were represented along the Principal Components (PCs) of a covariance matrix. The resulting PCs were ranked in the order of their effectiveness in the between-cluster separation. Only the most effective PCs were retained to construct an optimised eigenspace. A supervised learning algorithm was invoked for linear subdivision of the optimised eigenspace. The statistical significance levels of the classification results were estimated for validating the discriminant functions. The main contribution of this research has been to show that: the infrared imaging of facial thermal features within the 8-14 μm bandwidth may be used to observe affective-state-specific thermal variations on the face; the pixel-grey level analysis of TIRIs can help localise FTFPs along the major facial muscles of the face; cluster-analytic classification of transient thermal features may help distinguish between the facial expressions of affective states in an optimized eigenspace of input thermal feature vectors. The Gaussian mixture model with one cluster per affect worked better for some facial expressions than others. This made the influence of the Gaussian mixture model structure on the accuracy of the classification results obvious. However, the linear discrimination and confusion patterns observed in this work were consistent with the ones reported in several earlier studies. This investigation also unveiled some important dimensions of the future research on use of facial thermal features in affective human-computer interaction.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Tracking human faces in infrared video

Detecting and tracking face regions in image sequences has applications to important problems such as face recognition, human-computer interaction, and video surveillance. Visible sensors have inherent limitations in solving this task, such as the need for sufficient and specific lighting conditions, as well as sensitivity to variations in skin color. Thermal infrared (IR) imaging sensors image emitted light, not reflected light, and therefore do not have these limitations, providing a 24-hour, 365-day capability while also being more robust to variations in the appearance of individuals. In this paper, we present a system for tracking human heads that has three components. First, a method for modeling thermal emission from human skin that can be used for the purpose of segmenting and detecting faces and other exposed skin regions in IR imagery. Second, the segmentation model is applied to the CONDENSATION algorithm for tracking the head regions over time. This includes a new observation density that is motivated by the segmentation results. Finally, we examine how to use the tracking results to refine the segmentation estimate. 1