A Data-driven Approach for Detecting Stress in Plants Using Hyperspectral Imagery

A phenotype is an observable characteristic of an individual and is a function of its genotype and its growth environment. Individuals with different genotypes are impacted differently by exposure to the same environment. Therefore, phenotypes are often used to understand morphological and physiological changes in plants as a function of genotype and biotic and abiotic stress conditions. Phenotypes that measure the level of stress can help mitigate the adverse impacts on the growth cycle of the plant. Image-based plant phenotyping has the potential for early stress detection by means of computing responsive phenotypes in a non-intrusive manner. A large number of plants grown and imaged under a controlled environment in a high-throughput plant phenotyping (HTPP) system are increasingly becoming accessible to research communities. They can be useful to compute novel phenotypes for early stress detection. In early stages of stress induction, plants manifest responses in terms of physiological changes rather than morphological, making it difficult to detect using visible spectrum cameras which use only three wide spectral bands in the 380nm - 740 nm range. In contrast, hyperspectral imaging can capture a broad range of wavelengths (350nm - 2500nm) with narrow spectral bands (5nm). Hyperspectral imagery (HSI), therefore, provides rich spectral information which can help identify and track even small changes in plant physiology in response to stress. In this research, a data-driven approach has been developed to identify regions in plants that manifest abnormal reflectance patterns after stress induction. Reflectance patterns of age-matched unstressed plants are first characterized. The normal and stressed reflectance patterns are used to train a classifier that can predict if a point in the plant is stressed or not. Stress maps of a plant can be generated from its hyperspectral image and can be used to track the temporal propagation of stress. These stress maps are used to compute novel phenotypes that represent the level of stress in a plant and the stress trajectory over time. The data-driven approach is validated using a dataset of sorghum plants exposed to drought stress in a LemnaTec Scanalyzer 3D HTPP system. Advisers: Ashok Samal and Sruti Das Choudhur

Machine Learning for High-Throughput Stress Phenotyping in Plants

Advances in automated and high-throughput imaging technologies have resulted in a deluge of high-resolution images and sensor data of plants. However, extracting patterns and features from this large corpus of data requires the use of machine learning (ML) tools to enable data assimilation and feature identification for stress phenotyping. Four stages of the decision cycle in plant stress phenotyping and plant breeding activities where different ML approaches can be deployed are (i) identification, (ii) classification, (iii) quantification, and (iv) prediction (ICQP). We provide here a comprehensive overview and user-friendly taxonomy of ML tools to enable the plant community to correctly and easily apply the appropriate ML tools and best-practice guidelines for various biotic and abiotic stress traits.This article is published as Singh, Arti, Baskar Ganapathysubramanian, Asheesh Kumar Singh, and Soumik Sarkar. "Machine learning for high-throughput stress phenotyping in plants." Trends in plant science 21, no. 2 (2016): 110-124. DOI:10.1016/j.tplants.2015.10.015. Posted with permission.</p

Differentiable Kernels in Generalized Matrix Learning Vector Quantization

In the present paper we investigate the application of differentiable kernel for generalized matrix learning vector quantization as an alternative kernel-based classifier, which additionally provides classification dependent data visualization. We show that the concept of differentiable kernels allows a prototype description in the data space but equipped with the kernel metric. Moreover, using the visualization properties of the original matrix learning vector quantization we are able to optimize the class visualization by inherent visualization mapping learning also in this new kernel-metric data space

Useful Feature Extraction and Machine Learning Techniques for Identifying Unique Pattern Signatures Present in Hyperspectral Image Data

This chapter introduces several feature extraction techniques (FETs) and machine learning algorithms (MLA) that are useful for pattern recognition in hyperspectral data analysis (HDA). This chapter provides a handbook of the most popular FETs that have proven successful. Machine learning algorithms (MLA) for use with HDA are becoming prevalent in pattern recognition literature. Several of these algorithms are explained in detail to provide the user with insights into applying these for pattern recognition. Unsupervised learning applications are useful when the system is provided with the correct set of independent variables. Various forms of linear regression assay adequately solve hyperspectral pattern resolution for identifying phenotypes. K-means is an unsupervised learning algorithm that is used for systematically dividing a dataset into K number of pattern groups. Supervised and unsupervised neural networks (NNs) are used to discern patterns in hyperspectral data with features as inputs and in large datasets where little a priori knowledge is applied. Other supervised machine learning procedures derive valuable feature detectors and descriptors through support vector machine. Several methods using reduced sets for extracting patterns from hyperspectral data are shown by discretized numerical techniques and transformation processes. The accuracy of these methods and their usefulness is generally assessed

Mehitamata õhusõiduki rakendamine põllukultuuride saagikuse ja maa harimisviiside tuvastamisel

A Thesis for applying for the degree of Doctor of Philosophy in Environmental Protection.Väitekiri filosoofiadoktori kraadi taotlemiseks keskkonnakaitse erialal.This thesis aims to examine how machine learning (ML) technologies have aided significant advancements in image analysis in the area of precision agriculture. These multimodal computing technologies extend the use of machine learning to a broader spectrum of data collecting and selection for the advancement of agricultural practices (Nawar et al., 2017) These techniques will assist complicated cropping systems with more informed decisions with less human intervention, and provide a scalable framework for incorporating expert knowledge of the PA system. (Chlingaryan et al., 2018). Complexity, on the other hand, can be seen as a disadvantage in crop trials, as machine learning models require training/testing databases, limited areas with insignificant sampling sizes, time and space-specificity, and environmental factor interventions, all of which complicate parameter selection and make using a single empirical model for an entire region impractical. During the early stages of writing this thesis, we used a relatively traditional machine learning method to address the regression problem of crop yield and biomass prediction [(i.e., random forest regression (RFR), support vector regression (SVR), and artificial neural network (ANN)] to predicted dry matter (DM) yields of red clover. It obtained favourable results, however, the choosing of hyperparameters, the lengthy algorithms selection process, data cleaning, and redundant collinearity issues significantly limited the way of the machine learning application. We will further discuss the recent trend of automated machine learning (AutoML) that has been driving further significant technological innovation in the application of artificial intelligence from its automated algorithm selection and hyperparameter optimization of the deployable pipeline model for unravelling substance problems. However, a present knowledge gap exists in the integration of machine learning (ML) technology with unmanned aerial systems (UAS) and hyperspectral-based imaging data categorization and regression applications. In this thesis, we explored a state-of-the-art (SOTA) and entirely open-source AutoML framework, Auto-sklearn, which was built on one of the most frequently used machine learning systems, Scikit-learn. It was integrated with two unique AutoML visualization tools to examine the recognition and acceptance of multispectral vegetation indices (VI) data collected from UAS and hyperspectral narrow-band VIs across a varied spectrum of agricultural management practices (AMP). These procedures incorporate soil tillage method (STM), cultivation method (CM), and manure application (MA), and are classified as four-crop combination fields (i.e., red clover-grass mixture, spring wheat, pea-oat mixture, and spring barley). Additionally, they have not been thoroughly evaluated and lack characteristics that are accessible in agriculture remote sensing applications. This thesis further explores the existing gaps in the knowledge base for several critical crop categories and cultivation management methods referring to biomass and yield analysis, as well as to gain a better understanding of the potential for remotely sensed solutions to field-based and multifunctional platforms to meet precision agriculture demands. To overcome these knowledge gaps, this research introduces a rapid, non-destructive, and low-cost framework for field-based biomass and grain yield modelling, as well as the identification of agricultural management practices. The results may aid agronomists and farmers in establishing more accurate agricultural methods and in monitoring environmental conditions more effectively.Doktoritöö eesmärk oli uurida, kuidas masinõppe (MÕ) tehnoloogiad võimaldavad edusamme täppispõllumajanduse valdkonna pildianalüüsis. Multimodaalsed arvutustehnoloogiad laiendavad masinõppe kasutamist põllumajanduses andmete kogumisel ja valimisel (Nawar et al., 2017). Selline täpsemal informatsioonil põhinev tehnoloogia võimaldab keerukate viljelussüsteemide puhul teha otsuseid inimese vähema sekkumisega, ja loob skaleeritava raamistiku täppispõllumajanduse jaoks (Chlingaryan et al., 2018). Põllukultuuride katsete korral on komplekssete masinõppemudelite kasutamine keerukas, sest alad on piiratud ning valimi suurus ei ole piisav; vaja on testandmebaase, kindlaid aja- ja ruumitingimusi ning keskkonnategureid. See komplitseerib parameetrite valikut ning muudab ebapraktiliseks ühe empiirilise mudeli kasutamise terves piirkonnas. Siinse uurimuse algetapis rakendati suhteliselt traditsioonilist masinõppemeetodit, et lahendada saagikuse ja biomassi prognoosimise regressiooniprobleem (otsustusmetsa regression, tugivektori regressioon ja tehisnärvivõrk) punase ristiku prognoositava kuivaine saagikuse suhtes. Saadi sobivaid tulemusi, kuid hüperparameetrite valimine, pikk algoritmide valimisprotsess, andmete puhastamine ja kollineaarsusprobleemid takistasid masinõpet oluliselt. Automatiseeritud masinõppe (AMÕ) uusimate suundumustena rakendatakse tehisintellekti, et lahendada põhiprobleemid automatiseeritud algoritmi valiku ja rakendatava pipeline-mudeli hüperparameetrite optimeerimise abil. Seni napib teadmisi MÕ tehnoloogia integreerimiseks mehitamata õhusõidukite ning hüperspektripõhiste pildiandmete kategoriseerimise ja regressioonirakendustega. Väitekirjas uuriti nüüdisaegset ja avatud lähtekoodiga AMÕ tehnoloogiat Auto-sklearn, mis on ühe enimkasutatava masinõppesüsteemi Scikit-learn edasiarendus. Süsteemiga liideti kaks unikaalset AMÕ visualiseerimisrakendust, et uurida mehitamata õhusõidukiga kogutud andmete multispektraalsete taimkatteindeksite ja hüperspektraalsete kitsaribaandmete taimkatteindeksite tuvastamist ja rakendamist põllumajanduses. Neid võtteid kasutatakse mullaharimisel, kultiveerimisel ja sõnnikuga väetamisel nelja kultuuriga põldudel (punase ristiku rohusegu, suvinisu, herne-kaera segu, suvioder). Neid ei ole põhjalikult hinnatud, samuti ei hõlma need omadusi, mida kasutatatakse põllumajanduses kaugseire rakendustes. Uurimus käsitleb biomassi ja saagikuse seni uurimata analüüsivõimalusi oluliste põllukultuuride ja viljelusmeetodite näitel. Hinnatakse ka kaugseirelahenduste potentsiaali põllupõhiste ja multifunktsionaalsete platvormide kasutamisel täppispõllumajanduses. Uurimus tutvustab kiiret, keskkonna suhtes kahjutut ja mõõduka hinnaga tehnoloogiat põllupõhise biomassi ja teraviljasaagi modelleerimiseks, et leida sobiv viljelusviis. Töö tulemused võimaldavad põllumajandustootjatel ja agronoomidel tõhusamalt valida põllundustehnoloogiaid ning arvestada täpsemalt keskkonnatingimustega.Publication of this thesis is supported by the Estonian University of Life Scieces and by the Doctoral School of Earth Sciences and Ecology created under the auspices of the European Social Fund

Evaluation of hyperspectral band selection techniques for real-time applications

Processing hyperspectral image data can be computationally expensive and difficult to employ for real-time applications due to its extensive spatial and spectral information. Further, applications in which computational resources may be limited can be hindered by the volume of data that is common with airborne hyperspectral image data. This paper proposes utilizing band selection to down-select the number of spectral bands to consider for a given classification task such that classification can be done at the edge. Specifically, we consider the following state of the art band selection techniques: Fast Volume-Gradient-based Band Selection (VGBS), Improved Sparse Subspace Clustering (ISSC), Maximum-Variance Principal Component Analysis (MVPCA), and Normalized Cut Optimal Clustering MVPCA (NC-OC-MVPCA), to investigate their feasibility at identifying discriminative bands such that classification performance is not drastically hindered. This would greatly benefit applications where time-sensitive solutions are needed to ensure optimal outcomes. In this research, an NVIDIA AGX Xavier module is used as the edge device to run trained models on as a simulated deployed unmanned aerial system. Performance of the proposed approach is measured in terms of classification accuracy and run time

Roadmap on signal processing for next generation measurement systems

Signal processing is a fundamental component of almost any sensor-enabled system, with a wide range of applications across different scientific disciplines. Time series data, images, and video sequences comprise representative forms of signals that can be enhanced and analysed for information extraction and quantification. The recent advances in artificial intelligence and machine learning are shifting the research attention towards intelligent, data-driven, signal processing. This roadmap presents a critical overview of the state-of-the-art methods and applications aiming to highlight future challenges and research opportunities towards next generation measurement systems. It covers a broad spectrum of topics ranging from basic to industrial research, organized in concise thematic sections that reflect the trends and the impacts of current and future developments per research field. Furthermore, it offers guidance to researchers and funding agencies in identifying new prospects.AerodynamicsMicrowave Sensing, Signals & System

Detection of Fusarium Head Blight in Wheat Grains Using Hyperspectral and RGB Imaging

In modern agriculture, it is imperative to ensure that crops are healthy and safe for consumption. Fusarium Head Blight (FHB) can cause significant damage to wheat grains by reducing essential components such as moisture, protein, and starch, while also introducing dangerous toxins. Therefore, accurately distinguishing between healthy and FHB-infected wheat grains is essential to guarantee stable and reliable wheat production while limiting financial losses and ensuring food safety. This thesis proposes effective methods to classify healthy and FHB infected wheat grains using Hyperspectral Imaging (HSI) and Red Green Blue (RGB) images. The approach includes a combination of Principal Component Analysis (PCA) with morphology, in addition to dark and white reference correction, to create masks for grains in each image. The classification for the hyperspectral images was achieved using a Partial Least Squares Discriminant Analysis (PLS-DA) model for hyperspectral images and a Convolutional Neural Network (CNN) model for RGB images. Both object-based and pixel-based approaches were compared for the PLS-DA model. The results indicated that the object-based approach outperformed the pixel-based approach and other well-known machine learning algorithms, including Random Forest (RF), linear Support Vector Machine (SVM), Stochastic Gradient Descent (SGD) calibrated one-vs-all and DecisionTree. The PLS-DA model using the object-based method yielded better results when tested on all wheat varieties, achieving an F1-score of 99.4%. Specific wavelengths were investigated based on a loading plot, and four effective wavelengths were identified, 953 nm, 1373 nm, 1923 nm and 2493 nm, with classification accuracy found to be similar to the full spectral range. Moreover, the moisture and water content in the grains were analyzed using hyperspectral images through an aquagram, which demonstrated that healthy grains exhibited higher absorbance values than infected grains for all Water Matrix Coordinates (WAMACS). Furthermore, the CNN model was trained on cropped individual grains, and the classification accuracy was similar to the PLS-DA model, with an F1- score of 98.1%. These findings suggest that HSI is suitable for identifying FHB-infected wheat grains, while RGB images may provide a cost-effective alternative to hyperspectral images for this specific classification task. Further research should consider to explore the potential benefits of HSI for deeper investigations into how water absorption affects spectral measurements and moisture content in grains, in addition to user-friendly interfaces for deep learning based image classification

Leveraging Computer Vision for Applications in Biomedicine and Geoscience

Skin cancer is one of the most common types of cancer and is usually classified as either non-melanoma and melanoma skin cancer. Melanoma skin cancer accounts for about half of all skin cancer-related deaths. The 5-year survival rate is 99% when the cancer is detected early but drops to 25% once it becomes metastatic. In other words, the key to preventing death is early detection. Foraminifera are microscopic single-celled organisms that exist in marine environments and are classified as living a benthic or planktic lifestyle. In total, roughly 50,000 species are known to have existed, of which about 9,000 are still living today. Foraminifera are important proxies for reconstructing past ocean and climate conditions and as bio-indicators of anthropogenic pollution. Since the 1800s, the identification and counting of foraminifera have been performed manually. The process is resource-intensive. In this dissertation, we leverage recent advances in computer vision, driven by breakthroughs in deep learning methodologies and scale-space theory, to make progress towards both early detection of melanoma skin cancer and automation of the identification and counting of microscopic foraminifera. First, we investigate the use of hyperspectral images in skin cancer detection by performing a critical review of relevant, peer-reviewed research. Second, we present a novel scale-space methodology for detecting changes in hyperspectral images. Third, we develop a deep learning model for classifying microscopic foraminifera. Finally, we present a deep learning model for instance segmentation of microscopic foraminifera. The works presented in this dissertation are valuable contributions in the fields of biomedicine and geoscience, more specifically, towards the challenges of early detection of melanoma skin cancer and automation of the identification, counting, and picking of microscopic foraminifera