Machine Learning for Robust Understanding of Scene Materials in Hyperspectral Images

The major challenges in hyperspectral (HS) imaging and data analysis are expensive sensors, high dimensionality of the signal, limited ground truth, and spectral variability. This dissertation develops and analyzes machine learning based methods to address these problems. In the first part, we examine one of the most important HS data analysis tasks-vegetation parameter estimation. We present two Gaussian processes based approaches for improving the accuracy of vegetation parameter retrieval when ground truth is limited and/or spectral variability is high. The first is the adoption of covariance functions based on well-established metrics, such as, spectral angle and spectral correlation, which are known to be better measures of similarity for spectral data. The second is the joint modeling of related vegetation parameters by multitask Gaussian processes so that the prediction accuracy of the vegetation parameter of interest can be improved with the aid of related vegetation parameters for which a larger set of ground truth is available. The efficacy of the proposed methods is demonstrated by comparing them against state-of-the art approaches on three real-world HS datasets and one synthetic dataset. In the second part, we demonstrate how Bayesian optimization can be applied to jointly tune the different components of hyperspectral data analysis frameworks for better performance. Experimental validation on the spatial-spectral classification framework consisting of a classifier and a Markov random field is provided. In the third part, we investigate whether high dimensional HS spectra can be reconstructed from low dimensional multispectral (MS) signals, that can be obtained from much cheaper, lower spectral resolution sensors. A novel end-to-end convolutional residual neural network architecture is proposed that can simultaneously optimize both the MS bands and the transformation to reconstruct HS spectra from MS signals by analyzing a large quantity of HS data. The learned band can be implemented in sensor hardware and the learned transformation can be incorporated in the data processing pipeline to build a low-cost hyperspectral data collection system. Using a diverse set of real-world datasets, we show how the proposed approach of optimizing MS bands along with the transformation rather than just optimizing the transformation with fixed bands, as proposed by previous studies, can drastically increase the reconstruction accuracy. Additionally, we also investigate the prospects of using reconstructed HS spectra for land cover classification

Visible hyperspectral imaging for predicting intra-muscular fat content from sheep carcasses

Intramuscular fat (IMF) content plays a key role in the quality attributes of meat, such as sensory properties and health considerations. The tenderness, flavour and juiciness of meat are examples of sensory attributes influenced by IMF content. Traditionally, IMF content in meat was determined using destructive, time consuming and at times unsuitable methods in industry applications. However, with recent advancement of technology, there has been an interest in exlporing ways to ascertain meat quality without damage. Hyperspectral imaging analysis is an emerging technology that combines the use of spectroscopy and computer imaging analysis to obtain both the spectral and spatial information of objects of interest. Hyperspectral imaging was initially developed for remote sensing, but has recently emerged as powerful tool for non-destructive analysis of quality in the food industry and has had very accurate results in the prediction of meat qualities such as IMF content. In this thesis, we use a data set of 101 hyperspectral images of sheep carcasses to investigate the ability of multivariate statistical methods to accurately predict IMF content

Spectral Reflectance Reconstruction Using Fuzzy Logic System Training: Color Science Application

[EN] In this work, we address the problem of spectral reflectance recovery from both CIEXYZ and RGB values by means of a machine learning approach within the fuzzy logic framework, which constitutes the first application of fuzzy logic in these tasks. We train a fuzzy logic inference system using the Macbeth ColorChecker DC and we test its performance with a 130 sample target set made out of Artist's paints. As a result, we obtain a fuzzy logic inference system (FIS) that performs quite accurately. We have studied different parameter settings within the training to achieve a meaningful overfitting-free system. We compare the system performance against previous successful methods and we observe that both spectrally and colorimetrically our approach substantially outperforms these classical methods. In addition, from the FIS trained we extract the fuzzy rules that the system has learned, which provide insightful information about how the RGB/XYZ inputs are related to the outputs. That is to say that, once the system is trained, we extract the codified knowledge used to relate inputs and outputs. Thus, we are able to assign a physical and/or conceptual meaning to its performance that allows not only to understand the procedure applied by the system but also to acquire insight that in turn might lead to further improvements. In particular, we find that both trained systems use four reference spectral curves, with some similarities, that are combined in a non-linear way to predict spectral curves for other inputs. Notice that the possibility of being able to understand the method applied in the trained system is an interesting difference with respect to other 'black box' machine learning approaches such as the currently fashionable convolutional neural networks in which the downside is the impossibility to understand their ways of procedure. Another contribution of this work is to serve as an example of how, through the construction of a FIS, some knowledge relating inputs and outputs in ground truth datasets can be extracted so that an analogous strategy could be followed for other problems in color and spectral science.Samuel Morillas acknowledges the support of the Spanish Ministry of Science under grants PRX17/00384, PRX16/00050 and PID2019-107790RB-C22.Amiri, MM.; Garcia-Nieto, S.; Morillas, S.; Fairchild, MD. (2020). Spectral Reflectance Reconstruction Using Fuzzy Logic System Training: Color Science Application. Sensors. 20(17):1-18. https://doi.org/10.3390/s20174726S118201