Face recognition by means of advanced contributions in machine learning

Face recognition (FR) has been extensively studied, due to both scientific fundamental challenges and current and potential applications where human identification is needed. FR systems have the benefits of their non intrusiveness, low cost of equipments and no useragreement requirements when doing acquisition, among the most important ones. Nevertheless, despite the progress made in last years and the different solutions proposed, FR performance is not yet satisfactory when more demanding conditions are required (different viewpoints, blocked effects, illumination changes, strong lighting states, etc). Particularly, the effect of such non-controlled lighting conditions on face images leads to one of the strongest distortions in facial appearance. This dissertation addresses the problem of FR when dealing with less constrained illumination situations. In order to approach the problem, a new multi-session and multi-spectral face database has been acquired in visible, Near-infrared (NIR) and Thermal infrared (TIR) spectra, under different lighting conditions. A theoretical analysis using information theory to demonstrate the complementarities between different spectral bands have been firstly carried out. The optimal exploitation of the information provided by the set of multispectral images has been subsequently addressed by using multimodal matching score fusion techniques that efficiently synthesize complementary meaningful information among different spectra. Due to peculiarities in thermal images, a specific face segmentation algorithm has been required and developed. In the final proposed system, the Discrete Cosine Transform as dimensionality reduction tool and a fractional distance for matching were used, so that the cost in processing time and memory was significantly reduced. Prior to this classification task, a selection of the relevant frequency bands is proposed in order to optimize the overall system, based on identifying and maximizing independence relations by means of discriminability criteria. The system has been extensively evaluated on the multispectral face database specifically performed for our purpose. On this regard, a new visualization procedure has been suggested in order to combine different bands for establishing valid comparisons and giving statistical information about the significance of the results. This experimental framework has more easily enabled the improvement of robustness against training and testing illumination mismatch. Additionally, focusing problem in thermal spectrum has been also addressed, firstly, for the more general case of the thermal images (or thermograms), and then for the case of facialthermograms from both theoretical and practical point of view. In order to analyze the quality of such facial thermograms degraded by blurring, an appropriate algorithm has been successfully developed. Experimental results strongly support the proposed multispectral facial image fusion, achieving very high performance in several conditions. These results represent a new advance in providing a robust matching across changes in illumination, further inspiring highly accurate FR approaches in practical scenarios.El reconeixement facial (FR) ha estat àmpliament estudiat, degut tant als reptes fonamentals científics que suposa com a les aplicacions actuals i futures on requereix la identificació de les persones. Els sistemes de reconeixement facial tenen els avantatges de ser no intrusius,presentar un baix cost dels equips d’adquisició i no la no necessitat d’autorització per part de l’individu a l’hora de realitzar l'adquisició, entre les més importants. De totes maneres i malgrat els avenços aconseguits en els darrers anys i les diferents solucions proposades, el rendiment del FR encara no resulta satisfactori quan es requereixen condicions més exigents (diferents punts de vista, efectes de bloqueig, canvis en la il·luminació, condicions de llum extremes, etc.). Concretament, l'efecte d'aquestes variacions no controlades en les condicions d'il·luminació sobre les imatges facials condueix a una de les distorsions més accentuades sobre l'aparença facial. Aquesta tesi aborda el problema del FR en condicions d'il·luminació menys restringides. Per tal d'abordar el problema, hem adquirit una nova base de dades de cara multisessió i multiespectral en l'espectre infraroig visible, infraroig proper (NIR) i tèrmic (TIR), sota diferents condicions d'il·luminació. En primer lloc s'ha dut a terme una anàlisi teòrica utilitzant la teoria de la informació per demostrar la complementarietat entre les diferents bandes espectrals objecte d’estudi. L'òptim aprofitament de la informació proporcionada pel conjunt d'imatges multiespectrals s'ha abordat posteriorment mitjançant l'ús de tècniques de fusió de puntuació multimodals, capaces de sintetitzar de manera eficient el conjunt d’informació significativa complementària entre els diferents espectres. A causa de les característiques particulars de les imatges tèrmiques, s’ha requerit del desenvolupament d’un algorisme específic per la segmentació de les mateixes. En el sistema proposat final, s’ha utilitzat com a eina de reducció de la dimensionalitat de les imatges, la Transformada del Cosinus Discreta i una distància fraccional per realitzar les tasques de classificació de manera que el cost en temps de processament i de memòria es va reduir de forma significa. Prèviament a aquesta tasca de classificació, es proposa una selecció de les bandes de freqüències més rellevants, basat en la identificació i la maximització de les relacions d'independència per mitjà de criteris discriminabilitat, per tal d'optimitzar el conjunt del sistema. El sistema ha estat àmpliament avaluat sobre la base de dades de cara multiespectral, desenvolupada pel nostre propòsit. En aquest sentit s'ha suggerit l’ús d’un nou procediment de visualització per combinar diferents bandes per poder establir comparacions vàlides i donar informació estadística sobre el significat dels resultats. Aquest marc experimental ha permès més fàcilment la millora de la robustesa quan les condicions d’il·luminació eren diferents entre els processos d’entrament i test. De forma complementària, s’ha tractat la problemàtica de l’enfocament de les imatges en l'espectre tèrmic, en primer lloc, pel cas general de les imatges tèrmiques (o termogrames) i posteriorment pel cas concret dels termogrames facials, des dels punt de vista tant teòric com pràctic. En aquest sentit i per tal d'analitzar la qualitat d’aquests termogrames facials degradats per efectes de desenfocament, s'ha desenvolupat un últim algorisme. Els resultats experimentals recolzen fermament que la fusió d'imatges facials multiespectrals proposada assoleix un rendiment molt alt en diverses condicions d’il·luminació. Aquests resultats representen un nou avenç en l’aportació de solucions robustes quan es contemplen canvis en la il·luminació, i esperen poder inspirar a futures implementacions de sistemes de reconeixement facial precisos en escenaris no controlats.Postprint (published version

Ubiquitous Technologies for Emotion Recognition

Emotions play a very important role in how we think and behave. As such, the emotions we feel every day can compel us to act and influence the decisions and plans we make about our lives. Being able to measure, analyze, and better comprehend how or why our emotions may change is thus of much relevance to understand human behavior and its consequences. Despite the great efforts made in the past in the study of human emotions, it is only now, with the advent of wearable, mobile, and ubiquitous technologies, that we can aim to sense and recognize emotions, continuously and in real time. This book brings together the latest experiences, findings, and developments regarding ubiquitous sensing, modeling, and the recognition of human emotions

An Experimental and Numerical Investigation of Nitrogen Dioxide Emissions Characteristics of Compression Ignition Dual Fuel Engines

Detailed experimental research was conducted to explore the impact of the addition of gaseous fuels, including H2 and natural gas (NG), and engine load on the emissions of NO2, NO, and NOx from dual fuel engines. The addition of less than 2% of H2 or NG was shown to dramatically increase the emissions of NO2 until a maximum level of NO2 emissions was reached. The increased NO 2 emissions were due to the conversion of NO to NO2. The maximum NO2/NOx ratio obtained with the addition of H2 was 3.2 to 5.0 times that of diesel operation. The maximum NO 2/NOx ratio obtained with the addition of NG was 3.4 to 4.3 times that of diesel operation. Further increasing the amount of gaseous fuel beyond the point of maximum NO2 emissions resulted in a reduction of NO2 emissions. Detailed examination of factors having the potential to affect the formation of NOx and NO2 in compression ignition engines reported a firm correlation between the emissions of NO 2 and emissions of unburned H2 and methane (CH4), and their relative emissions. The presence of unburned gaseous fuels that survived the main combustion process appears to be one of the main factors contributing to the enhanced conversion of NO to NO2. This was supported by the experimental data reported in the literature. The presence of fumigation fuels outside the diesel spray plume might be the main factor contributing to the increased emissions of NO2 from dual fuel engines. The spontaneous combustion of fumigation fuels that are entrained into the diesel spray plume may not contribute to the increased emissions of NO 2. In comparison, the correlations between the increased emissions of NO2 and the variation in bulk mixture temperature and heat release process including maximum heat release rate, and combustion duration were weak.;A single zone, zero-dimensional, constant volume numerical model with detailed chemistry was used to simulate the oxidization process of the gaseous fuel, as well as its effect on the conversion of NO to NO2 after the post-combustion mixing of the gaseous fuel surviving the main combustion process with the NOx-containing combustion products. The gaseous fuel examined included CH4, H2, and carbon monoxide (CO). The simulation results revealed the significant effects of the fuel mixed, its initial concentration in the mixture, and the initial temperature on the oxidization of gaseous fuel, the conversion of NO to NO2, and the destruction of NO2 to NO after the completion of the oxidation process.;The single zone zero-dimensional model was further modified to a variable volume model with the volume of the combustion chamber calculated using the geometry of the 1999 Cummins engine and engine speed. The modified variable volume model with detailed chemistry was used to improve the simulation of the effect on the conversion of NO to NO2 of the post-combustion mixing of surviving gaseous fuel with NOx-containing combustion products. The spatial variation of the local bulk mixture temperature with the progress of the combustion process and the variation of cylinder volume during the expansion process was taken into account by a pseudo temperature at the top dead center (TDC) noted as Tpseudo TDC defined in this research. The simulation identified the importance of the phasing of postcombustion mixing on the oxidation of gaseous fuel and its effect on the conversion of NO to NO2.;A preliminary sensitivity analysis was also conducted to identify the reactions having significant effect on the conversion of NO to NO2 and its destruction to NO. Among the four reactions associated with the formation and destruction of NO2, R186 was identified as the main reaction to the formation of NO2 during the oxidation process of H 2 and CO. This was due to the high concentration of HO2 formed during the oxidation process of H2 and CO in the combustion product. The destruction of NO2 to NO occurred through R187 and R189. (Abstract shortened by UMI.)

Muscle temperature analysis, using thermal imaging, applied to the treatment of muscle recovery

The images help in the different processes where a visual interpretation of a scene is required, in this sense we find many applications where images are used to analyze, interpret and classify certain objects within the image, there are different types of images generated by different sensors, in this paper describes a method to analyze the behavior of the muscle, mainly of the knee, when performing rehabilitation exercises, coupled with an optical image where you can see the state of the muscle and the location, the method proposed as a super position between optical and thermal images, with the intention of being able to know the state of the optical image and to have the same image with information of the behavior of the temperature, the super position that we propose is to have as a base the optical image and on placing the thermal image, the results that are presented are oriented in proposing a new way of analyzing data with thermal information of the behavior of the muscles, by means of a complex image with optical and thermal information, the method is an aid in the treatment of muscular recovery, with the benefits of being scalable and applicable to other muscles and parts of the human body

Face Image Modality Recognition and Photo-Sketch Matching

Face is an important physical characteristic of human body, and is widely used in many crucial applications, such as video surveillance, criminal investigation, and security access system. Based on realistic demand, such as useful face images in dark environment and criminal profile, different modalities of face images appeared, e.g. three-dimensional (3D), near infrared (NIR), and thermal infrared (TIR) face images. Thus, researches with various face image modalities become a hot area. Most of them are set on knowing the modality of face images in advance, which contains a few limitations. In this thesis, we present approaches for face image modality recognition to extend the possibility of cross-modality researches as well as handle new modality-mixed face images. Furthermore, a large facial image database is assembled with five commonly used modalities such as 3D, NIR, TIR, sketch, and visible light spectrum (VIS). Based on the analysis of results, a feature descriptor based on convolutional neural network with linear kernel SVM did an optimal performance.;As we mentioned above, face images are widely used in crucial applications, and one of them is using the sketch of suspect\u27s face, which based on the witness\u27 description, to assist law enforcement. Since it is difficult to capture face photos of the suspect during a criminal activity, automatic retrieving photos based on the suspect\u27s facial sketch is used for locating potential suspects. In this thesis, we perform photo-sketch matching by synthesizing the corresponding pseudo sketch from a given photo. There are three methods applied in this thesis, which are respectively based on style transfer, DualGAN, and cycle-consistent adversarial networks. Among the results of these methods, style transfer based method did a poor performance in photo-sketch matching, since it is an unsupervised one which is not purposeful in photo to sketch synthesis problem while the others need to train pointed models in synthesis stage

Hyperspectral imaging for the remote sensing of blood oxygenation and emotions

This PhD project is a basic research and it concerns with how human’s physiological features, such as tissue oxygen saturation (StO2), can be captured from a stand-off distance and then to understand how this remotely acquired physiological feature can be deployed for biomedical and other applications. This work utilises Hyperspectral Imaging (HSI) within the diffuse optical scattering framework, to assess the StO2 in a contactless remote sensing manner. The assessment involves a detailed investigation about the wavelength dependence of diffuse optical scattering from the skin as well as body tissues, under various forms of optical absorption models. It is concluded that the threechromophore extended Beer Lambert Law model is better suited for assessing the palm and facial tissue oxygenations, especially when spectral data in the wavelengths region of [516-580]nm is used for the analysis. A first attempt of using the facial StO2 to detect and to classify people’s emotional state is initiated in this project. The objective of this work is to understand how strong emotions, such as distress that caused by mental or physical stimulations, can be detected using physiological feature such as StO2. Based on data collected from ~20 participants, it is found that the forehead StO2 is elevated upon the onset of strong emotions that triggered by mental stimulation. The StO2 pattern in the facial region upon strong emotions that are initiated by physical stimulations is quite complicated, and further work is needed for a better understanding of the interplays between bodily physique, individual’s health condition and blood transfusion control mechanism. Most of this work has already been published and future research to follow up when the author returns back to China is highlighted

A systematic review of physiological signals based driver drowsiness detection systems.

Driving a vehicle is a complex, multidimensional, and potentially risky activity demanding full mobilization and utilization of physiological and cognitive abilities. Drowsiness, often caused by stress, fatigue, and illness declines cognitive capabilities that affect drivers' capability and cause many accidents. Drowsiness-related road accidents are associated with trauma, physical injuries, and fatalities, and often accompany economic loss. Drowsy-related crashes are most common in young people and night shift workers. Real-time and accurate driver drowsiness detection is necessary to bring down the drowsy driving accident rate. Many researchers endeavored for systems to detect drowsiness using different features related to vehicles, and drivers' behavior, as well as, physiological measures. Keeping in view the rising trend in the use of physiological measures, this study presents a comprehensive and systematic review of the recent techniques to detect driver drowsiness using physiological signals. Different sensors augmented with machine learning are utilized which subsequently yield better results. These techniques are analyzed with respect to several aspects such as data collection sensor, environment consideration like controlled or dynamic, experimental set up like real traffic or driving simulators, etc. Similarly, by investigating the type of sensors involved in experiments, this study discusses the advantages and disadvantages of existing studies and points out the research gaps. Perceptions and conceptions are made to provide future research directions for drowsiness detection techniques based on physiological signals. [Abstract copyright: © The Author(s), under exclusive licence to Springer Nature B.V. 2022. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Deep Learning Based Face Detection and Recognition in MWIR and Visible Bands

In non-favorable conditions for visible imaging like extreme illumination or nighttime, there is a need to collect images in other spectra, specifically infrared. Mid-Wave infrared (3-5 microm) images can be collected without giving away the location of the sensor in varying illumination conditions. There are many algorithms for face detection, face alignment, face recognition etc. proposed in visible band till date, while the research using MWIR images is highly limited. Face detection is an important pre-processing step for face recognition, which in turn is an important biometric modality. This thesis works towards bridging the gap between MWIR and visible spectrum through three contributions. First, a dual band based deep face detection model that works well in visible and MWIR spectrum is proposed using transfer learning. Different models are trained and tested extensively using visible and MWIR images and the one model that works well for this data is determined. For this model, experiments are conducted to learn the speed/accuracy trade-off. Following this, the available MWIR dataset is extended through augmentation using traditional methods and generative adversarial networks (GANs). Traditional methods used to augment the data are brightness adjustment, contrast enhancement, applying noise to and de-noising the images. A deep learning based GAN architecture is developed and is used to generate new face identities. The generated images are added to the original dataset and the face detection model developed earlier is once again trained and tested. The third contribution is the proposal of another GAN that converts given thermal ace images into their visible counterparts. A pre-trained model is used as discriminator for this purpose and is trained to classify the images as real and fake and an identity network is used to provide further feedback to the generator. The generated visible images are used as probe images and the original visible images are used as gallery images to perform face recognition experiments using a state-of-the-art visible-to-visible face recognition algorithm

Emotion and Stress Recognition Related Sensors and Machine Learning Technologies

This book includes impactful chapters which present scientific concepts, frameworks, architectures and ideas on sensing technologies and machine learning techniques. These are relevant in tackling the following challenges: (i) the field readiness and use of intrusive sensor systems and devices for capturing biosignals, including EEG sensor systems, ECG sensor systems and electrodermal activity sensor systems; (ii) the quality assessment and management of sensor data; (iii) data preprocessing, noise filtering and calibration concepts for biosignals; (iv) the field readiness and use of nonintrusive sensor technologies, including visual sensors, acoustic sensors, vibration sensors and piezoelectric sensors; (v) emotion recognition using mobile phones and smartwatches; (vi) body area sensor networks for emotion and stress studies; (vii) the use of experimental datasets in emotion recognition, including dataset generation principles and concepts, quality insurance and emotion elicitation material and concepts; (viii) machine learning techniques for robust emotion recognition, including graphical models, neural network methods, deep learning methods, statistical learning and multivariate empirical mode decomposition; (ix) subject-independent emotion and stress recognition concepts and systems, including facial expression-based systems, speech-based systems, EEG-based systems, ECG-based systems, electrodermal activity-based systems, multimodal recognition systems and sensor fusion concepts and (x) emotion and stress estimation and forecasting from a nonlinear dynamical system perspective