Hyperspectral classification of Cyperus esculentus clones and morphologically similar weeds

Cyperus esculentus (yellow nutsedge) is one of the world's worst weeds as it can cause great damage to crops and crop production. To eradicate C. esculentus, early detection is key-a challenging task as it is often confused with other Cyperaceae and displays wide genetic variability. In this study, the objective was to classify C. esculentus clones and morphologically similar weeds. Hyperspectral reflectance between 500 and 800 nm was tested as a measure to discriminate between (I) C. esculentus and morphologically similar Cyperaceae weeds, and between (II) different clonal populations of C. esculentus using three classification models: random forest (RF), regularized logistic regression (RLR) and partial least squares-discriminant analysis (PLS-DA). RLR performed better than RF and PLS-DA, and was able to adequately classify the samples. The possibility of creating an affordable multispectral sensing tool, for precise in-field recognition of C. esculentus plants based on fewer spectral bands, was tested. Results of this study were compared against simulated results from a commercially available multispectral camera with four spectral bands. The model created with customized bands performed almost equally well as the original PLS-DA or RLR model, and much better than the model describing multispectral image data from a commercially available camera. These results open up the opportunity to develop a dedicated robust tool for C. esculentus recognition based on four spectral bands and an appropriate classification model

Convolutional neural network ensemble learning for hyperspectral imaging-based blackberry fruit ripeness detection in uncontrolled farm environment

Fruit ripeness estimation models have for decades depended on spectral index features or colour-based features, such as mean, standard deviation, skewness, colour moments, and/or histograms for learning traits of fruit ripeness. Recently, few studies have explored the use of deep learning techniques to extract features from images of fruits with visible ripeness cues. However, the blackberry (Rubus fruticosus) fruit does not show obvious and reliable visible traits of ripeness when mature and therefore poses great difficulty to fruit pickers. The mature blackberry, to the human eye, is black before, during, and post-ripening. To address this engineering application challenge, this paper proposes a novel multi-input convolutional neural network (CNN) ensemble classifier for detecting subtle traits of ripeness in blackberry fruits. The multi-input CNN was created from a pre-trained visual geometry group 16-layer deep convolutional network (VGG16) model trained on the ImageNet dataset. The fully connected layers were optimized for learning traits of ripeness of mature blackberry fruits. The resulting model served as the base for building homogeneous ensemble learners that were ensemble using the stack generalization ensemble (SGE) framework. The input to the network is images acquired with a stereo sensor using visible and near-infrared (VIS-NIR) spectral filters at wavelengths of 700 nm and 770 nm. Through experiments, the proposed model achieved 95.1% accuracy on unseen sets and 90.2% accuracy with in-field conditions. Further experiments reveal that machine sensory is highly and positively correlated to human sensory over blackberry fruit skin texture

Semantic Segmentation based deep learning approaches for weed detection

Global increase in herbicide use to control weeds has led to issues such as evolution of herbicide-resistant weeds, off-target herbicide movement, etc. Precision agriculture advocates Site Specific Weed Management (SSWM) application to achieve precise and right amount of herbicide spray and reduce off-target herbicide movement. Recent advancements in Deep Learning (DL) have opened possibilities for adaptive and accurate weed recognitions for field based SSWM applications with traditional and emerging spraying equipment; however, challenges exist in identifying the DL model structure and train the model appropriately for accurate and rapid model applications over varying crop/weed growth stages and environment. In our study, an encoder-decoder based DL architecture was proposed that performs pixel-wise Semantic Segmentation (SS) classifications of crop, soil, and weed patches in the fields. The objective of this study was to develop a robust weed detection algorithm using DL techniques that can accurately and reliably locate weed infestations in low altitude Unmanned Aerial Vehicle (UAV) imagery with acceptable application speed. Two different encoder-decoder based SS models of LinkNet and UNet were developed using transfer learning techniques. We performed various measures such as backpropagation optimization and refining of the dataset used for training to address the class-imbalance problem which is a common issue in developing weed detection models. It was found that LinkNet model with ResNet18 as the encoder section and use of ‘Focal loss’ loss function was able to achieve the highest mean and class-wise Intersection over Union scores for different class categories while performing predictions on unseen dataset. The developed state-of-art model did not require a large amount of data during training and the techniques used to develop the model in our study provides a propitious opportunity that performs better than the existing SS based weed detections models. The proposed model integrates a futuristic approach to develop a model that could be used for weed detection on aerial imagery from UAV and perform real-time SSWM applications Advisor: Yeyin Sh

Spectral filter design based on in-field hyperspectral imaging and machine learning for mango ripeness estimation

Hyperspectral imaging (HSI) is a powerful technology already used for many objectives in agriculture. Applications include disease monitoring, plant phenotyping, yield estimation or fruit composition and ripeness. However, the cost of hyperspectral sensors is typically an order of magnitude higher than simpler RGB cameras, which can be prohibitive. Given that in HSI processing the spectral data often contains redundancies, the full spectra are not always required for a specific application and there is an opportunity to design a lower cost multi-spectral sensing system by dimensionality reduction. In past work, HSI dimensionality reduction has been applied in the form of band selection to achieve faster computation times. If, however, the objective is to design a lower cost multi-spectral camera system, band selection is poorly suited because real-world sensor and optical filter responses do not typically replicate the individual bands of a hyperspectral sensor. The objective of this paper is to develop a new methodology for filter selection by simulating several imaging devices with different real-world optical filters, to use a high cost HSI device to design a lower cost multi-spectral solution for a specific application. In this paper, we apply the technique to the specific task of mango fruit maturity estimation (dry matter), which was recently shown to be possible using HSI. Mango HSI acquired under field conditions from an UGV was used as input for the experiments. These involved the simulation of imaging devices, using support vector machines for modelling, and testing several filter combinations by brute force or optimisation with genetic algorithms. The mango prediction performance of the simulations was compared to the best performance obtained with full HSI data, which had an R of 0.74. The best values came from the simulation of a four-sensor device with four distinct filters, achieving R up to 0.69 for mango dry matter estimation. The results showed that genetic algorithms, when compared to brute force approaches, were able to obtain the best solution in an efficient way, and that a good performance for mango ripeness estimation can be achieved from the combination of four spectral filters that would allow to implement them into a low-cost, custom-made multi-spectral sensor. The methods exposed in this paper are more broadly applicable to applications beyond mango maturity estimation.Salvador Gutiérrez would like to acknowledge the research funding FPI grant 299/2016 by Universidad de La Rioja, Gobierno de La Rioja. This work is supported by the Australian Centre for Field Robotics (ACFR) at The University of Sydney

Novel spectral imaging instrumentation for environmental sensing in extreme environments

Spectral imaging techniques provide a valuable means of improving our understanding of the world around us. Environmental monitoring approaches that utilise these techniques are, therefore, essential to our understanding of the effects of climate change. Hyperspectral imaging applications are of particular benefit to a broad range of environmental monitoring scenarios, providing rich datasets that combine both spectral and spatial information, enabling intricate features and variations to be visualised. However, to date, most commercially available hyperspectral instrumentation remains bulky and expensive, significantly limiting their user-base and accessibility. These factors substantially limit the use of these instruments resulting in much of our information coming from a few well-resourced research teams across a limited number of more easily accessed field locations. These limitations, have a compounded effect on the quality and robustness of hyperspectral data outputs, particularly within more extreme settings, as the comparatively small sample of more accessible locations is not necessarily representative of the much larger whole. This thesis presents on the development and testing of three novel low-cost hyperspectral imaging instruments designed specifically for environmental monitoring applications, providing valuable, low-cost alternatives to currently available commercial systems. Specifically, the three instruments presented within this thesis are: a low-cost laboratory-based hyperspectral imager, a semi-portable instrument capable of accurate data capture within a laboratory setting; the Hyperspectral Smartphone, an ultra-low-cost smartphone-based fully portable hyperspectral imager; and a low-cost high-resolution hyperspectral imager capable of resolving mm-scale spatial targets. All instruments were rigorously tested to analyse and evaluate their performances. Each instrument was shown to perform well across a range of environmental monitoring applications demonstrating that expensive commercial instrumentation is not required to achieve accurate and robust hyperspectral imaging. These low-cost instruments could promote the widespread dissemination of accessible hyperspectral imaging equipment, facilitating the democratisation of hyperspectral measurement modalities across environmental monitoring applications and beyond