Deep Learning Meets Hyperspectral Image Analysis: A Multidisciplinary Review

Modern hyperspectral imaging systems produce huge datasets potentially conveying a great abundance of information; such a resource, however, poses many challenges in the analysis and interpretation of these data. Deep learning approaches certainly offer a great variety of opportunities for solving classical imaging tasks and also for approaching new stimulating problems in the spatial–spectral domain. This is fundamental in the driving sector of Remote Sensing where hyperspectral technology was born and has mostly developed, but it is perhaps even more true in the multitude of current and evolving application sectors that involve these imaging technologies. The present review develops on two fronts: on the one hand, it is aimed at domain professionals who want to have an updated overview on how hyperspectral acquisition techniques can combine with deep learning architectures to solve specific tasks in different application fields. On the other hand, we want to target the machine learning and computer vision experts by giving them a picture of how deep learning technologies are applied to hyperspectral data from a multidisciplinary perspective. The presence of these two viewpoints and the inclusion of application fields other than Remote Sensing are the original contributions of this review, which also highlights some potentialities and critical issues related to the observed development trends

A review of neural networks in plant disease detection using hyperspectral data

© 2018 China Agricultural University This paper reviews advanced Neural Network (NN) techniques available to process hyperspectral data, with a special emphasis on plant disease detection. Firstly, we provide a review on NN mechanism, types, models, and classifiers that use different algorithms to process hyperspectral data. Then we highlight the current state of imaging and non-imaging hyperspectral data for early disease detection. The hybridization of NN-hyperspectral approach has emerged as a powerful tool for disease detection and diagnosis. Spectral Disease Index (SDI) is the ratio of different spectral bands of pure disease spectra. Subsequently, we introduce NN techniques for rapid development of SDI. We also highlight current challenges and future trends of hyperspectral data

Challenges and Opportunities in Machine-Augmented Plant Stress Phenotyping

Plant stress phenotyping is essential to select stress-resistant varieties and develop better stress-management strategies. Standardization of visual assessments and deployment of imaging techniques have improved the accuracy and reliability of stress assessment in comparison with unaided visual measurement. The growing capabilities of machine learning (ML) methods in conjunction with image-based phenotyping can extract new insights from curated, annotated, and high-dimensional datasets across varied crops and stresses. We propose an overarching strategy for utilizing ML techniques that methodically enables the application of plant stress phenotyping at multiple scales across different types of stresses, program goals, and environments

Hyperspectral Image Classification -- Traditional to Deep Models: A Survey for Future Prospects

Hyperspectral Imaging (HSI) has been extensively utilized in many real-life applications because it benefits from the detailed spectral information contained in each pixel. Notably, the complex characteristics i.e., the nonlinear relation among the captured spectral information and the corresponding object of HSI data make accurate classification challenging for traditional methods. In the last few years, Deep Learning (DL) has been substantiated as a powerful feature extractor that effectively addresses the nonlinear problems that appeared in a number of computer vision tasks. This prompts the deployment of DL for HSI classification (HSIC) which revealed good performance. This survey enlists a systematic overview of DL for HSIC and compared state-of-the-art strategies of the said topic. Primarily, we will encapsulate the main challenges of traditional machine learning for HSIC and then we will acquaint the superiority of DL to address these problems. This survey breakdown the state-of-the-art DL frameworks into spectral-features, spatial-features, and together spatial-spectral features to systematically analyze the achievements (future research directions as well) of these frameworks for HSIC. Moreover, we will consider the fact that DL requires a large number of labeled training examples whereas acquiring such a number for HSIC is challenging in terms of time and cost. Therefore, this survey discusses some strategies to improve the generalization performance of DL strategies which can provide some future guidelines

Towards automated phenotyping in plant tissue culture

Plant in vitro culture techniques comprise important fundamental methods of modern plant research, propagation and breeding. Innovative scientific approaches to further develop the cultivation process, therefore, have the potential of far-reaching impact on many different areas. In particular, automation can increase efficiency of in vitro propagation, a domain currently con-strained by intensive manual labor. Automated phenotyping of plant in vitro culture bears the potential to extend the evaluation of in vitro plants from manual destructive endpoint measurements to continuous and objective digital quantification of plant traits. Consequently, this can lead to a better understanding of crucial developmental processes and will help to clarify the emergence of physiological disorders of plant in vitro cultures. The aim of this dissertation was to investigate and exemplify the potential of optical sensing methods and machine learning in plant in vitro culture from an interdisciplinary point of view. A novel robotic phenotyping system for automated, non-destructive, multi-dimensional in situ detection of plant traits based on low-cost sensor technology was con-ceptualized, developed and tested. Various sensor technologies, including an RGB camera, a laser distance sensor, a micro spectrometer, and a thermal camera, were applied partly for the first time under these challenging conditions and evaluated with respect to the resulting data quality and feasibility. In addition to the development of new dynamic, semi-automated data processing pipelines, the automatic acquisition of multisensory data across an entire subculture passage of plant in vitro cultures was demonstrated. This allowed novel time series images of different developmental processes of plant in vitro cultures and the emergence of physiological disorders to be captured in situ for the first time. The digital determination of relevant parameters such as projected plant area, average canopy height, and maximum plant height, was demonstrated, which can be used as critical descriptors of plant growth performance in vitro. In addition, a novel method of non-destructive quantification of media volume by depth data was developed which may allow monitoring of water uptake by plants and evaporation from the culture medium. The phenotyping system was used to investigate the etiology of the physiological growth anomaly hyperhydricity. Therefore, digital monitoring of the morphology and along with spectro-scopic studies of reflectance behavior over time were conducted. The new optical characteristics identified by classical spectral analysis, such as reduced reflectance and major absorption peaks of hyperhydricity in the SWIR region could be validated to be the main discriminating features by a trained support vector machine with a balanced accuracy of 84% on test set, demonstrating the feasibility of a spectral detection of hyperhydricity. In addition, an RGB image dataset was used for automated detection of hyperhydricity using deep neural networks. The high-performance metrics with precision of 83.8% and recall of 95.7% on test images underscore the presence of for detection sufficient number of discriminating features within the spatial RGB data, thus a second approach is proposed for automatic detection of hyperhydricity based on RGB images. The resulting multimodal sensor data sets of the robotic phenotyping system were tested as a supporting tool of an e-learning module in higher education to increase the digital skills in the field of sensing, data processing and data analysis, and evaluated by means of a student survey. This proof-of-concept study revealed an overall high level of acceptance and advocacy by students with 70% good to very good rating. However, with increased complexity of the learning task, stu-dents experienced excessive demands and rated the respective session lower. In summary, this study is expected to pave the way for increased use of automated sensor-based phenotyping in conjunction with machine learning in plant research and commercial mi-cropropagation in the future.Die pflanzliche In-vitro-Kultur umfasst wichtige grundlegende Methoden der modernen Pflanzenforschung, -vermehrung und -züchtung. Innovative wissenschaftliche Ansätze zur Wei-terentwicklung des Kultivierungsprozess können daher weitreichenden Einfluss auf viele unter-schiedliche Bereiche haben. Insbesondere die Automatisierung kann die Effizienz der In-vitro-Vermehrung steigern, die derzeit durch die intensive manuelle Arbeit beschränkt wird. Automa-tisierte Phänotypisierung von In-vitro-Kulturen ermöglicht es, die Erfassung von manuellen de-struktiven Endpunktmessungen auf eine kontinuierliche, objektive und digitale Quantifizierung der Pflanzenmerkmale auszuweiten. Dies kann zu einem besseren Verständnis entscheidender Entwicklungsprozesse führen und die Entstehung physiologischer Störungen zu klären. Ziel dieser Dissertation war es, das Potential optischer Erfassungsmethoden und des maschinellen Lernens für die pflanzliche In-vitro-Kultur unter interdisziplinären Gesichtspunk-ten zu untersuchen und exemplarisch aufzuzeigen. Ein neuartiger Phänotypisierungsroboter zur automatisierten, zerstörungsfreien, mehrdimensionalen In-situ-Erfassung von Pflanzenmerkmalen wurde auf Basis kostengünstiger Sensortechnik entwickelt. Unterschiedliche Sensortechnologien, darunter eine RGB-Kamera, ein Laser-Distanzsensor, ein Mikrospektrometer und eine Wärmebildkamera, wurden teils zum ersten Mal unter diesen schwierigen Bedingungen eingesetzt und im Hinblick auf die resultierende Datenqualität und Realisierbarkeit bewertet. Neben der Entwicklung dynamischer, halbautomatischer Datenverarbeitungspipelines, wurde die automatische Erfassung multisensorischer Daten über eine gesamte Subkulturpassage der In-vitro-Kulturen demonstriert. Dadurch konnte erstmals Zeitrafferaufnahmen verschiedener Ent-wicklungsprozesse von pflanzlichen In-vitro-Kulturen und das Auftreten von physiologischen Störungen in situ erfasst werden. Die digitale Bestimmung relevanter Kenngrößen wie der proji-zierten Pflanzenfläche, der durchschnittlichen Bestandshöhe und der maximalen Pflanzenhöhe wurde demonstriert, die als wichtige Deskriptoren für das pflanzliche Wachstum dienen können. Darüber hinaus konnte eine neue Methode für die Pflanzenwissenschaften entwickelt werden, um die Wasseraufnahme von Pflanzen und die Verdunstung von Kulturmedien auf der Grundlage einer zerstörungsfreien Quantifizierung des Medienvolumens zu überwachen. Der Phänotypisierungsroboter wurde zur Untersuchung der Entstehung der Wachs-tumsanomalie Hyperhydrizität eingesetzt. Hierfür wurden ein digitales Monitoring der Morpho-logie der Explantate mit begleitenden spektroskopischen Untersuchungen des Reflexionsverhal-tens im Zeitverlauf durchgeführt. Die durch Spektralanalyse identifizierten optischen Merkmale, wie den reduzierter Reflexionsgrad und die Hauptabsorptionspeaks der Hyperhydrizität in der SWIR-Region, konnten als die wichtigsten Unterscheidungsmerkmale durch ein Support-Vektor-Maschine-Model mit einer Genauigkeit von 84% auf dem Testsatz validiert werden und damit Machbarkeit der spektrale Identifizierung von Hyperhydrizität aufzeigen. Darüber wurde für die automatische Detektion der Hyperhydrizität auf Basis von RGB-Bildern ein neuronales Netz trainiert. Die hohen Kennzahlen im Testdatensatz wie die Präzision von 83,8 % und einem Recall von 95,7 % unterstreichen das Vorhandensein einer für die Erkennung ausreichenden Anzahl von Unterscheidungsmerkmalen innerhalb der räumlichen RGB-Daten. Somit konnte ein zweiter An-satz der automatischen Detektion von Hyperhydrizität durch RGB-Bilder präsentiert werden. Die resultierenden Sensordatensätze des Phänotypisierungsroboters wurden als unter-stützendes Werkzeug eines E-Learning Moduls zur Steigerung digitaler Kompetenzen im Bereich Sensortechnik, Datenverarbeitung und -auswertung in der Hochschulausbildung erprobt und an-hand der Befragung von Studierenden evaluiert. Diese Machbarkeitsstudie ergab eine insgesamt hohe Akzeptanz durch die Studierenden mit 70% guten bis sehr guten Bewertungen. Mit zuneh-mender Komplexität der Lernaufgabe fühlten sich die Studierenden jedoch überfordert und bewerteten die jeweilige Session schlechter. Zusammenfassend zielt diese Arbeit darauf ab den Weg für einen verstärkten Einsatz der automatisierten, sensorbasierten Phänotypisierung in Kombination mit den Techniken des ma-schinellen Lernens der Forschung und der kommerziellen Mikrovermehrung zukünftig zu ebnen.Bundesministerium für Ernährung und Landwirtschaft (BMEL)/Digitale Experimentierfelder/28DE103F18/E

Perspective Chapter: Hyperspectral Imaging for the Analysis of Seafood

Hyperspectral imaging technology is able to provide useful information about the interaction between electromagnetic radiation and matter. This information makes possible chemical characterization of materials in a non-invasive manner. For this reason, the technology has been of great interest for the food industry in recent decades. In this book chapter, we provide a survey of the current status of the use of hyperspectral technology for seafood evaluation. First, we provide a brief description of the optical properties of tissue and an introduction to the instrumentation used to capture these images. Then, we survey the main applications of hyperspectral imaging in the seafood industry, including the quantification of different chemical components, the estimation of freshness, the quality assessment of seafood products, and the detection of nematodes, among others. Finally, we provide a discussion about the current state of the art and the upcoming challenges for the application of this technology in the seafood industry

Clasificación de imágenes hiperespectrales

La alta demanda y uso extensivo de plásticos en la vida moderna está asociada con un impacto económico significativo y una seria huella ecológica. La producción de plásticos conlleva un alto consumo de energía y emisiones de CO2, así como una gran necesidad de recursos fósiles (limitados). Debido a la alta durabilidad de los plásticos, grandes cantidades de basura plástica se acumulan en vertederos desbordantes y los desechos plásticos flotan en los océanos del mundo. Por otra parte, la incineración para generar energía libera aún más CO2 y sustancias tóxicas a la atmósfera. El reciclaje de productos plásticos después de su ciclo de vida puede contribuir significativamente a la reducción de los impactos ambientales y económicos. Para producir productos de reciclaje de alta calidad, se requieren composiciones monofraccionales de polímeros de desecho; es decir, composiciones formadas por un solo tipo de plástico, en lugar de ser una mezcla de diferentes tipos de plásticos reciclados. Existen varios métodos para clasificar y recoger este tipo de polímeros, desde separación manual, clasificación mecánica, flotación por densidad o tecnología por infrarrojos. Sin embargo, las tecnologías de medición existentes, como la espectroscopia de infrarrojo cercano, presentan limitaciones en la clasificación de mezclas complejas y diferentes grados de polímeros. Las tecnologías más recientemente inventadas basadas en espectroscopia de infrarrojo medio, espectroscopia Raman o espectroscopia asistida por láser aún se encuentran en desarrollo y se espera que sean bastante costosas. Según Gruber et al. (2019), la clasificación de residuos plásticos negros utilizando imágenes de fluorescencia y aprendizaje automático ha demostrado ser eficaz para mejorar los procesos de reciclaje [1]. En este trabajo se clasifican residuos en función de su composición química procesando imágenes hiperespectrales (HSI). Las imágenes hiperespectrales consisten en una imagen a la cual se le agregan más información en su tercera dimensión. Así, mientras que una imagen normal consta de 3 dimensiones: ancho, altura y en su tercera dimensión encontramos tres espectros; en una HSI la tercera dimensión la componen muchas bandas. En el caso de este estudio en concreto son 224

Neuro-Fuzzy Based Intelligent Approaches to Nonlinear System Identification and Forecasting

Nearly three decades back nonlinear system identification consisted of several ad-hoc approaches, which were restricted to a very limited class of systems. However, with the advent of the various soft computing methodologies like neural networks and the fuzzy logic combined with optimization techniques, a wider class of systems can be handled at present. Complex systems may be of diverse characteristics and nature. These systems may be linear or nonlinear, continuous or discrete, time varying or time invariant, static or dynamic, short term or long term, central or distributed, predictable or unpredictable, ill or well defined. Neurofuzzy hybrid modelling approaches have been developed as an ideal technique for utilising linguistic values and numerical data. This Thesis is focused on the development of advanced neurofuzzy modelling architectures and their application to real case studies. Three potential requirements have been identified as desirable characteristics for such design: A model needs to have minimum number of rules; a model needs to be generic acting either as Multi-Input-Single-Output (MISO) or Multi-Input-Multi-Output (MIMO) identification model; a model needs to have a versatile nonlinear membership function. Initially, a MIMO Adaptive Fuzzy Logic System (AFLS) model which incorporates a prototype defuzzification scheme, while utilising an efficient, compared to the Takagi–Sugeno–Kang (TSK) based systems, fuzzification layer has been developed for the detection of meat spoilage using Fourier transform infrared (FTIR) spectroscopy. The identification strategy involved not only the classification of beef fillet samples in their respective quality class (i.e. fresh, semi-fresh and spoiled), but also the simultaneous prediction of their associated microbiological population directly from FTIR spectra. In the case of AFLS, the number of memberships for each input variable was directly associated to the number of rules, hence, the “curse of dimensionality” problem was significantly reduced. Results confirmed the advantage of the proposed scheme against Adaptive Neurofuzzy Inference System (ANFIS), Multilayer Perceptron (MLP) and Partial Least Squares (PLS) techniques used in the same case study. In the case of MISO systems, the TSK based structure, has been utilized in many neurofuzzy systems, like ANFIS. At the next stage of research, an Adaptive Fuzzy Inference Neural Network (AFINN) has been developed for the monitoring the spoilage of minced beef utilising multispectral imaging information. This model, which follows the TSK structure, incorporates a clustering pre-processing stage for the definition of fuzzy rules, while its final fuzzy rule base is determined by competitive learning. In this specific case study, AFINN model was also able to predict for the first time in the literature, the beef’s temperature directly from imaging information. Results again proved the superiority of the adopted model. By extending the line of research and adopting specific design concepts from the previous case studies, the Asymmetric Gaussian Fuzzy Inference Neural Network (AGFINN) architecture has been developed. This architecture has been designed based on the above design principles. A clustering preprocessing scheme has been applied to minimise the number of fuzzy rules. AGFINN incorporates features from the AFLS concept, by having the same number of rules as well as fuzzy memberships. In spite of the extensive use of the standard symmetric Gaussian membership functions, AGFINN utilizes an asymmetric function acting as input linguistic node. Since the asymmetric Gaussian membership function’s variability and flexibility are higher than the traditional one, it can partition the input space more effectively. AGFINN can be built either as an MISO or as an MIMO system. In the MISO case, a TSK defuzzification scheme has been implemented, while two different learning algorithms have been implemented. AGFINN has been tested on real datasets related to electricity price forecasting for the ISO New England Power Distribution System. Its performance was compared against a number of alternative models, including ANFIS, AFLS, MLP and Wavelet Neural Network (WNN), and proved to be superior. The concept of asymmetric functions proved to be a valid hypothesis and certainly it can find application to other architectures, such as in Fuzzy Wavelet Neural Network models, by designing a suitable flexible wavelet membership function. AGFINN’s MIMO characteristics also make the proposed architecture suitable for a larger range of applications/problems

Machine learning in marine ecology: an overview of techniques and applications

Machine learning covers a large set of algorithms that can be trained to identify patterns in data. Thanks to the increase in the amount of data and computing power available, it has become pervasive across scientific disciplines. We first highlight why machine learning is needed in marine ecology. Then we provide a quick primer on machine learning techniques and vocabulary. We built a database of ∼1000 publications that implement such techniques to analyse marine ecology data. For various data types (images, optical spectra, acoustics, omics, geolocations, biogeochemical profiles, and satellite imagery), we present a historical perspective on applications that proved influential, can serve as templates for new work, or represent the diversity of approaches. Then, we illustrate how machine learning can be used to better understand ecological systems, by combining various sources of marine data. Through this coverage of the literature, we demonstrate an increase in the proportion of marine ecology studies that use machine learning, the pervasiveness of images as a data source, the dominance of machine learning for classification-type problems, and a shift towards deep learning for all data types. This overview is meant to guide researchers who wish to apply machine learning methods to their marine datasets.Machine learning in marine ecology: an overview of techniques and applicationspublishedVersio