Artificial vision by thermography : calving prediction and defect detection in carbon fiber reinforced polymer

La vision par ordinateur est un domaine qui consiste à extraire ou identifier une ou plusieurs informations à partir d’une ou plusieurs images dans le but soit d’automatiser une tache, soit de fournir une aide à la décision. Avec l’augmentation de la capacité de calcul des ordinateurs, la vulgarisation et la diversification des moyens d’imagerie tant dans la vie quotidienne, que dans le milieu industriel,ce domaine a subi une évolution rapide lors de dernières décennies. Parmi les différentes modalités d’imagerie pour lesquels il est possible d’utiliser la vision artificielle cette thèse se concentre sur l’imagerie infrarouge. Plus particulièrement sur l’imagerie infrarouge pour les longueurs d’ondes comprises dans les bandes moyennes et longues. Cette thèse se porte sur deux applications industrielles radicalement différentes. Dans la première partie de cette thèse, nous présentons une application de la vision artificielle pour la détection du moment de vêlage en milieux industriel pour des vaches Holstein. Plus précisément l’objectif de cette recherche est de déterminer le moment de vêlage en n’utilisant que des données comportementales de l’animal. À cette fin, nous avons acquis des données en continu sur différents animaux pendant plusieurs mois. Parmi les nombreux défis présentés par cette application l’un d’entre eux concerne l’acquisition des données. En effet, les caméras que nous avons utilisées sont basées sur des capteurs bolométriques, lesquels sont sensibles à un grand nombre de variables. Ces variables peuvent être classées en quatre catégories : intrinsèque, environnemental, radiométrique et géométrique. Un autre défit important de cette recherche concerne le traitement des données. Outre le fait que les données acquises utilisent une dynamique plus élevée que les images naturelles ce qui complique le traitement des données ; l’identification de schéma récurrent dans les images et la reconnaissance automatique de ces derniers grâce à l’apprentissage automatique représente aussi un défi majeur. Nous avons proposé une solution à ce problème. Dans le reste de cette thèse nous nous sommes penchés sur la problématique de la détection de défaut dans les matériaux, en utilisant la technique de la thermographie pulsée. La thermographie pulsée est une méthode très populaire grâce à sa simplicité, la possibilité d’être utilisée avec un grand nombre de matériaux, ainsi que son faible coût. Néanmoins, cette méthode est connue pour produire des données bruitées. La cause principale de cette réputation vient des diverses sources de distorsion auquel les cameras thermiques sont sensibles. Dans cette thèse, nous avons choisi d’explorer deux axes. Le premier concerne l’amélioration des méthodes de traitement de données existantes. Dans le second axe, nous proposons plusieurs méthodes pour améliorer la détection de défauts. Chaque méthode est comparée à plusieurs méthodes constituant l’état de l’art du domaine.Abstract Computer vision is a field which consists in extracting or identifying one or more information from one or more images in order either to automate a task or to provide decision support. With the increase in the computing capacity of computers, the popularization and diversification of imaging means, both in industry, as well as in everyone’s life, this field has undergone a rapid development in recent decades. Among the different imaging modalities for which it is possible to use artificial vision, this thesis focuses on infrared imaging. More particularly on infrared imagery for wavelengths included in the medium and long bands. This thesis focuses on two radically different industrial applications. In the first part of this thesis, we present an application of artificial vision for the detection of the calving moment in industrial environments for Holstein cows. More precisely, the objective of this research is to determine the time of calving using only physiological data from the animal. To this end, we continuously acquired data on different animals over several days. Among the many challenges presented by this application, one of them concerns data acquisition. Indeed, the cameras we used are based on bolometric sensors, which are sensitive to a large number of variables. These variables can be classified into four categories: intrinsic, environmental, radiometric and geometric. Another important challenge in this research concerns the processing of data. Besides the fact that the acquired data uses a higher dynamic range than the natural images which complicates the processing of the data; Identifying recurring patterns in images and automatically recognizing them through machine learning is a major challenge. We have proposed a solution to this problem. In the rest of this thesis we have focused on the problem of defect detection in materials, using the technique of pulsed thermography. Pulse thermography is a very popular method due toits simplicity, the possibility of being used with a large number of materials, as well as its low cost. However, this method is known to produce noisy data. The main cause of this reputation comes from the various sources of distortion to which thermal cameras are sensitive. In this thesis, we have chosen to explore two axes. The first concerns the improvement of existing data processing methods. In the second axis, we propose several methods to improve fault detection. Each method is compared to several methods constituting the state of the art in the field

A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Use of unmanned aircraft systems (UAS) and multispectral imagery for quantifying agricultural areas damaged by wild pigs

Wild pigs (Sus scrofa) cause extensive damage to agricultural crops, resulting in lost production and income. A major challenge associated with assessing damage to crops is locating and quantifying damaged areas within agricultural fields. We evaluated a novel method using multispectral high-resolution aerial imagery, collected from sensors mounted on unmanned aircraft systems (UAS), and feature extraction techniques to detect and map areas of corn fields damaged by wild pigs in southern Missouri, USA. Damaged areas were extracted from orthomosaics using visible and near-infrared band combinations, an object-based classification approach, and hierarchical learning cycles. To validate estimates we also collected ground reference data immediately following flights. Overall accuracy of damage estimates to corn fields were similar among band combinations evaluated, ranging from 74% to 98% when using visible and near-infrared information, compared to 72%–94% with visible information alone. By including near-infrared with visible information, though, we found higher average kappa values (0.76) than with visible information (0.60) alone. We demonstrated that UAS are an appropriate platform for collecting high-resolution multispectral imagery of corn fields and that object-oriented classifiers can be effectively used to delineate areas damaged by wild pigs. The proposed approach outlines a new monitoring technique that can efficiently estimate damage to entire corn fields caused by wild pigs and also has potential to be applied to other crop types

Thermal Imaging for Enhancing Inspection Reliability: Detection and Characterization

Reliable performance of an equipment or structure depends on pre-service quality and in- service degradation of the equipment or structure under operating conditions. The role of non-destructive testing (NDT) is to ensure integrity, and in turn, reliability of equipment or structure. Besides, NDT can also monitor in-service degradation and to avoid premature failure of the equipments/structures and prevent accidents as well as save human life. Up to now, NDT has been used in various fields of applications such as the inspection o

Precision technologies to address dairy cattle welfare: focus on lameness, mastitis and body condition

Specific animal-based indicators that can be used to predict animal welfare have been the core of protocols for assessing the welfare of farm animals, such as those produced by the Welfare Quality project. At the same time, the contribution of technological tools for the accurate and realtime assessment of farm animal welfare is also evident. The solutions based on technological tools fit into the precision livestock farming (PLF) concept, which has improved productivity, economic sustainability, and animal welfare in dairy farms. PLF has been adopted recently; nevertheless, the need for technological support on farms is getting more and more attention and has translated into significant scientific contributions in various fields of the dairy industry, but with an emphasis on the health and welfare of the cows. This review aims to present the recent advances of PLF in dairy cow welfare, particularly in the assessment of lameness, mastitis, and body condition, which are among the most relevant animal-based indications for the welfare of cows. Finally, a discussion is presented on the possibility of integrating the information obtained by PLF into a welfare assessment framework.FE1B-06B2-126F | Jos? Pedro Pinto de Ara?joN/

Face recognition by means of advanced contributions in machine learning

Face recognition (FR) has been extensively studied, due to both scientific fundamental challenges and current and potential applications where human identification is needed. FR systems have the benefits of their non intrusiveness, low cost of equipments and no useragreement requirements when doing acquisition, among the most important ones. Nevertheless, despite the progress made in last years and the different solutions proposed, FR performance is not yet satisfactory when more demanding conditions are required (different viewpoints, blocked effects, illumination changes, strong lighting states, etc). Particularly, the effect of such non-controlled lighting conditions on face images leads to one of the strongest distortions in facial appearance. This dissertation addresses the problem of FR when dealing with less constrained illumination situations. In order to approach the problem, a new multi-session and multi-spectral face database has been acquired in visible, Near-infrared (NIR) and Thermal infrared (TIR) spectra, under different lighting conditions. A theoretical analysis using information theory to demonstrate the complementarities between different spectral bands have been firstly carried out. The optimal exploitation of the information provided by the set of multispectral images has been subsequently addressed by using multimodal matching score fusion techniques that efficiently synthesize complementary meaningful information among different spectra. Due to peculiarities in thermal images, a specific face segmentation algorithm has been required and developed. In the final proposed system, the Discrete Cosine Transform as dimensionality reduction tool and a fractional distance for matching were used, so that the cost in processing time and memory was significantly reduced. Prior to this classification task, a selection of the relevant frequency bands is proposed in order to optimize the overall system, based on identifying and maximizing independence relations by means of discriminability criteria. The system has been extensively evaluated on the multispectral face database specifically performed for our purpose. On this regard, a new visualization procedure has been suggested in order to combine different bands for establishing valid comparisons and giving statistical information about the significance of the results. This experimental framework has more easily enabled the improvement of robustness against training and testing illumination mismatch. Additionally, focusing problem in thermal spectrum has been also addressed, firstly, for the more general case of the thermal images (or thermograms), and then for the case of facialthermograms from both theoretical and practical point of view. In order to analyze the quality of such facial thermograms degraded by blurring, an appropriate algorithm has been successfully developed. Experimental results strongly support the proposed multispectral facial image fusion, achieving very high performance in several conditions. These results represent a new advance in providing a robust matching across changes in illumination, further inspiring highly accurate FR approaches in practical scenarios.El reconeixement facial (FR) ha estat àmpliament estudiat, degut tant als reptes fonamentals científics que suposa com a les aplicacions actuals i futures on requereix la identificació de les persones. Els sistemes de reconeixement facial tenen els avantatges de ser no intrusius,presentar un baix cost dels equips d’adquisició i no la no necessitat d’autorització per part de l’individu a l’hora de realitzar l'adquisició, entre les més importants. De totes maneres i malgrat els avenços aconseguits en els darrers anys i les diferents solucions proposades, el rendiment del FR encara no resulta satisfactori quan es requereixen condicions més exigents (diferents punts de vista, efectes de bloqueig, canvis en la il·luminació, condicions de llum extremes, etc.). Concretament, l'efecte d'aquestes variacions no controlades en les condicions d'il·luminació sobre les imatges facials condueix a una de les distorsions més accentuades sobre l'aparença facial. Aquesta tesi aborda el problema del FR en condicions d'il·luminació menys restringides. Per tal d'abordar el problema, hem adquirit una nova base de dades de cara multisessió i multiespectral en l'espectre infraroig visible, infraroig proper (NIR) i tèrmic (TIR), sota diferents condicions d'il·luminació. En primer lloc s'ha dut a terme una anàlisi teòrica utilitzant la teoria de la informació per demostrar la complementarietat entre les diferents bandes espectrals objecte d’estudi. L'òptim aprofitament de la informació proporcionada pel conjunt d'imatges multiespectrals s'ha abordat posteriorment mitjançant l'ús de tècniques de fusió de puntuació multimodals, capaces de sintetitzar de manera eficient el conjunt d’informació significativa complementària entre els diferents espectres. A causa de les característiques particulars de les imatges tèrmiques, s’ha requerit del desenvolupament d’un algorisme específic per la segmentació de les mateixes. En el sistema proposat final, s’ha utilitzat com a eina de reducció de la dimensionalitat de les imatges, la Transformada del Cosinus Discreta i una distància fraccional per realitzar les tasques de classificació de manera que el cost en temps de processament i de memòria es va reduir de forma significa. Prèviament a aquesta tasca de classificació, es proposa una selecció de les bandes de freqüències més rellevants, basat en la identificació i la maximització de les relacions d'independència per mitjà de criteris discriminabilitat, per tal d'optimitzar el conjunt del sistema. El sistema ha estat àmpliament avaluat sobre la base de dades de cara multiespectral, desenvolupada pel nostre propòsit. En aquest sentit s'ha suggerit l’ús d’un nou procediment de visualització per combinar diferents bandes per poder establir comparacions vàlides i donar informació estadística sobre el significat dels resultats. Aquest marc experimental ha permès més fàcilment la millora de la robustesa quan les condicions d’il·luminació eren diferents entre els processos d’entrament i test. De forma complementària, s’ha tractat la problemàtica de l’enfocament de les imatges en l'espectre tèrmic, en primer lloc, pel cas general de les imatges tèrmiques (o termogrames) i posteriorment pel cas concret dels termogrames facials, des dels punt de vista tant teòric com pràctic. En aquest sentit i per tal d'analitzar la qualitat d’aquests termogrames facials degradats per efectes de desenfocament, s'ha desenvolupat un últim algorisme. Els resultats experimentals recolzen fermament que la fusió d'imatges facials multiespectrals proposada assoleix un rendiment molt alt en diverses condicions d’il·luminació. Aquests resultats representen un nou avenç en l’aportació de solucions robustes quan es contemplen canvis en la il·luminació, i esperen poder inspirar a futures implementacions de sistemes de reconeixement facial precisos en escenaris no controlats.Postprint (published version

Dynamic Thermal Models for Human Body Dissipation

The IR emissivity of human skin is extremly high and measurements of infrared radiation emitted by the skin can be converted directly into accurate temperature value. This process is known as Body IR Thermography. Skin temperature increase associated with increased vascular flow and increased metabolism. In this paper a novel multi-resolution real-time 3D thermal imaging system as potential solution for a human body 3D thermal models standardisation is presented. The system consists of a high-resolution offline 3D scanner and a real-time low-resolution 3D scanner, both of them paired together with a thermal imaging camera. The emphasis of this paper is the presentation of the novel concept of the standardisation of human body 3D thermal models, captured by the multi-resolution real-time 3D thermal imaging system. The standardisation procedure utilises skeleton detection, skeleton transformation, mesh optimisation, and texture mapping. The presented concept enables novel and practical methods for human body 3D thermal models comparison and analysis