BRUISE DETECTION IN APPLES USING 3D INFRARED IMAGING AND MACHINE LEARNING TECHNOLOGIES

Bruise detection plays an important role in fruit grading. A bruise detection system capable of finding and removing damaged products on the production lines will distinctly improve the quality of fruits for sale, and consequently improve the fruit economy. This dissertation presents a novel automatic detection system based on surface information obtained from 3D near-infrared imaging technique for bruised apple identification. The proposed 3D bruise detection system is expected to provide better performance in bruise detection than the existing 2D systems. We first propose a mesh denoising filter to reduce noise effect while preserving the geometric features of the meshes. Compared with several existing mesh denoising filters, the proposed filter achieves better performance in reducing noise effect as well as preserving bruised regions in 3D meshes of bruised apples. Next, we investigate two different machine learning techniques for the identification of bruised apples. The first technique is to extract hand-crafted feature from 3D meshes, and train a predictive classifier based on hand-crafted features. It is shown that the predictive model trained on the proposed hand-crafted features outperforms the same models trained on several other local shape descriptors. The second technique is to apply deep learning to learn the feature representation automatically from the mesh data, and then use the deep learning model or a new predictive model for the classification. The optimized deep learning model achieves very high classification accuracy, and it outperforms the performance of the detection system based on the proposed hand-crafted features. At last, we investigate GPU techniques for accelerating the proposed apple bruise detection system. Specifically, the dissertation proposes a GPU framework, implemented in CUDA, for the acceleration of the algorithm that extracts vertex-based local binary patterns. Experimental results show that the proposed GPU program speeds up the process of extracting local binary patterns by 5 times compared to a single-core CPU program

Recent Advances and Applications of Hyperspectral Imaging for Fruit and Vegetable Quality Assessment

Hyperspectral imaging systems are starting to be used as a scientific tool for food quality assessment. A typical hyperspectral image is composed of a set of a relatively wide range of monochromatic images corresponding to continuous wavelengths that normally contain redundant information or may exhibit a high degree of correlation. In addition, computation of the classifiers used to deal with the data obtained from the images can become excessively complex and time-consuming for such high-dimensional datasets, and this makes it difficult to incorporate such systems into an industry that demands standard protocols or high-speed processes. Therefore, recent works have focused on the development of new systems based on this technology that are capable of analysing quality features that cannot be inspected using visible imaging. Many of those studies have also centred on finding new statistical techniques to reduce the hyperspectral images to multispectral ones, which are easier to implement in automatic, non-destructive systems. This article reviews recent works that use hyperspectral imaging for the inspection of fruit and vegetables. It explains the different technologies available to acquire the images and their use for the non-destructive inspection of the internal and external features of these products. Particular attention is paid to the works aimed at reducing the dimensionality of the images, with details of the statistical techniques most commonly used for this task

Automated early plant disease detection and grading system: Development and implementation

As the agriculture industry grows, many attempts have been made to ensure high quality of produce. Diseases and defects found in plants and crops, affect the agriculture industry greatly. Hence, many techniques and technologies have been developed to help solving or reducing the impact of plant diseases. Imagining analysis tools, and gas sensors are becoming more frequently integrated into smart systems for plant disease detection. Many disease detection systems incorporate imaging analysis tools and Volatile Organic Compound (VOC) profiling techniques to detect early symptoms of diseases and defects of plants, fruits and vegetative produce. These disease detection techniques can be further categorized into two main groups; preharvest disease detection and postharvest disease detection techniques. This thesis aims to introduce the available disease detection techniques and to compare it with the latest innovative smart systems that feature visible imaging, hyperspectral imaging, and VOC profiling. In addition, this thesis incorporates the use of image analysis tools and k-means segmentation to implement a preharvest Offline and Online disease detection system. The Offline system to be used by pathologists and agriculturists to measure plant leaf disease severity levels. K-means segmentation and triangle thresholding techniques are used together to achieve good background segmentation of leaf images. Moreover, a Mamdani-Type Fuzzy Logic classification technique is used to accurately categorize leaf disease severity level. Leaf images taken from a real field with varying resolutions were tested using the implemented system to observe its effect on disease grade classification. Background segmentation using k-means clustering and triangle thresholding proved to be effective, even in non-uniform lighting conditions. Integration of a Fuzzy Logic system for leaf disease severity level classification yielded in classification accuracies of 98%. Furthermore, a robot is designed and implemented as a robotized Online system to provide field based analysis of plant health using visible and near infrared spectroscopy. Fusion of visible and near infrared images are used to calculate the Normalized Deference Vegetative Index (NDVI) to measure and monitor plant health. The robot is designed to have the functionality of moving across a specified path within an agriculture field and provide health information of leaves as well as position data. The system was tested in a tomato greenhouse under real field conditions. The developed system proved effective in accurately classifying plant health into one of 3 classes; underdeveloped, unhealthy, and healthy with an accuracy of 83%. A map with plant health and locations is produced for farmers and agriculturists to monitor the plant health across different areas. This system has the capability of providing early vital health analysis of plants for immediate action and possible selective pesticide spraying

GuavaNet: A deep neural network architecture for automatic sensory evaluation to predict degree of acceptability for Guava by a consumer

This thesis is divided into two parts:Part I: Analysis of Fruits, Vegetables, Cheese and Fish based on Image Processing using Computer Vision and Deep Learning: A Review. It consists of a comprehensive review of image processing, computer vision and deep learning techniques applied to carry out analysis of fruits, vegetables, cheese and fish.This part also serves as a literature review for Part II.Part II: GuavaNet: A deep neural network architecture for automatic sensory evaluation to predict degree of acceptability for Guava by a consumer. This part introduces to an end-to-end deep neural network architecture that can predict the degree of acceptability by the consumer for a guava based on sensory evaluation

Quasi Real-Time Apple Defect Segmentation Using Deep Learning

Defect segmentation of apples is an important task in the agriculture industry for quality control and food safety. In this paper, we propose a deep learning approach for the automated segmentation of apple defects using convolutional neural networks (CNNs) based on a U-shaped architecture with skip-connections only within the noise reduction block. An ad-hoc data synthesis technique has been designed to increase the number of samples and at the same time to reduce neural network overfitting. We evaluate our model on a dataset of multi-spectral apple images with pixel-wise annotations for several types of defects. In this paper, we show that our proposal outperforms in terms of segmentation accuracy general-purpose deep learning architectures commonly used for segmentation tasks. From the application point of view, we improve the previous methods for apple defect segmentation. A measure of the computational cost shows that our proposal can be employed in real-time (about 100 frame-per-second on GPU) and in quasi-real-time (about 7/8 frame-per-second on CPU) visual-based apple inspection. To further improve the applicability of the method, we investigate the potential of using only RGB images instead of multi-spectral images as input images. The results prove that the accuracy in this case is almost comparable with the multi-spectral case

Nondestructive Multivariate Classification of Codling Moth Infested Apples Using Machine Learning and Sensor Fusion

Apple is the number one on the list of the most consumed fruits in the United States. The increasing market demand for high quality apples and the need for fast, and effective quality evaluation techniques have prompted research into the development of nondestructive evaluation methods. Codling moth (CM), Cydia pomonella L. (Lepidoptera: Tortricidae), is the most devastating pest of apples. Therefore, this dissertation is focused on the development of nondestructive methods for the detection and classification of CM-infested apples. The objective one in this study was aimed to identify and characterize the source of detectable vibro-acoustic signals coming from CM-infested apples. A novel approach was developed to correlate the larval activities to low-frequency vibro-acoustic signals, by capturing the larval activities using a digital camera while simultaneously registering the signal patterns observed in the contact piezoelectric sensors on apple surface. While the larva crawling was characterized by the low amplitude and higher frequency (around 4 Hz) signals, the chewing signals had greater amplitude and lower frequency (around 1 Hz). In objective two and three, vibro-acoustic and acoustic impulse methods were developed to classify CM-infested and healthy apples. In the first approach, the identified vibro-acoustic patterns from the infested apples were used for the classification of the CM-infested and healthy signal data. The classification accuracy was as high as 95.94% for 5 s signaling time. For the acoustic impulse method, a knocking test was performed to measure the vibration/acoustic response of the infested apple fruit to a pre-defined impulse in comparison to that of a healthy sample. The classification rate obtained was 99% for a short signaling time of 60-80 ms. In objective four, shortwave near infrared hyperspectral imaging (SWNIR HSI) in the wavelength range of 900-1700 nm was applied to detect CM infestation at the pixel level for the three apple cultivars reaching an accuracy of up to 97.4%. In objective five, the physicochemical characteristics of apples were predicted using HSI method. The results showed the correlation coefficients of prediction (Rp) up to 0.90, 0.93, 0.97, and 0.91 for SSC, firmness, pH and moisture content, respectively. Furthermore, the effect of long-term storage (20 weeks) at three different storage conditions (0 °C, 4 °C, and 10 °C) on CM infestation and the detectability of the infested apples was studied. At a constant storage temperature the detectability of infested samples remained the same for the first three months then improved in the fourth month followed by a decrease until the end of the storage. Finally, a sensor data fusion method was developed which showed an improvement in the classification performance compared to the individual methods. These findings indicated there is a high potential of acoustic and NIR HSI methods for detecting and classifying CM infestation in different apple cultivars

Spoilage Detection in Raspberry Fruit Based on Spectral Imaging Using Convolutional Neural Networks

Effective spoilage detection of perishable food items like fruits and vegetables is essential for retailers who stock and sell large quantities of these items. This research is aimed at developing a non-destructive, rapid and accurate method which is based on Spectral Imaging (SI) used in tandem with Convolutional Neural Network (CNN) to predict whether the fruit is fresh or rotten. The study also aims to determine the number of days before which the fruit rots. This research employs a primary, quantitative and inductive methods to investigate the Deep Learning based approach to detect fruit spoilage. Raspberry fruit in particular has been chosen for the experiment. Baskets of raspberries from three different stores were bought and stored in the refrigerator at four-degree Celsius. Images of these baskets was captured on a daily basis using an RGB digital camera until all the baskets of fruits were rotten. The study employs a Supervised learning-based classification approach where-by the data is labelled based on the physical appearance of fruits in the basket. The results show that a Spectral imaging technique used along with a CNN yields a good accuracy of 86% with the F1 score of 0.82 to classify the fruits as Good or Bad but does not fare well in estimating the number of days before the fruit actually rots. The ability of CNN to process and identify patterns in a SI to detect spoilage in fruits would help fruit retail operators to optimize their business chain

A fruit recognition method for automatic harvesting

Automation of harvesting is always one of the hottest topics in greenhouse operation. But before this, a reliable method of identifying mature fruit clusters on plants is required. This thesis presents a method to detect and recognize mature tomato fruit clusters on a complex-structured tomato plant containing clutter and occlusion in a tomato greenhouse. A color stereo vision camera is applied as the vision sensor. The proposed method performs a 3D reconstruction with the data collected by the stereo camera to create a 3D environment for further processing. The Color Layer Growing (CLG) method is introduced to segment the mature fruits from the leaves, stalks, background and noise. Target fruit clusters can then be located by depth segmentation. The experimental data was collected from a tomato greenhouse and the method is justified by the experimental results

Visible and Hyperspectral Imaging Systems for the Detection and Discrimination of Mechanical and Microbiological Damage of Mushrooms

Horticultural products such as mushrooms are exposed to environmental conditions during their postharvest life, which may affect product quality. Loss of whiteness during storage is particularly important in the mushroom industry. Rough handling and distribution, fruiting body senescence and bacterial infections are among the main causes of mushroom discolouration. The aim of this work was to study the use of visible and hyperspectral imaging (HSI) systems for the detection and discrimination of mechanical and microbiological damage of mushrooms. This piece of research involved a) monitoring the browning of mushroom with visible computer imaging systems, b) investigating the effect of mechanical damage on the kinetics of enzymes responsible for mushroom browning, c) exploring the potential use of Vis-NIR HSI to predict PPO activity in mushroom caps and d) studying the potential application of Vis-NIR HSI for microbial and viral detection on mushroom caps and for their discrimination from mechanical damage. Results presented in this thesis show that the efficacy of commercial webcams was limited in the detection of mechanical damage on mushroom caps. Damage increased the activity of PPOs on mushroom pileipellis, but the effect of the extent of damage was not significant at the levels of study. Vis-NIR HSI showed some potential as a tool to estimate the activity of PPO enzymes on mushroom caps. The combination of HSI with chemometric tools allowed for the differentiation of mechanically and microbiologically damaged mushroom classes. Results from this study could be used for developing non-destructive monitoring systems for mechanical and microbiological damage detection and discrimination. The potential application of such systems as on-line process analytical tools would facilitate rapid assessment of mushroom quality.

Proceedings of the 2021 Joint Workshop of Fraunhofer IOSB and Institute for Anthropomatics, Vision and Fusion Laboratory

2021, the annual joint workshop of the Fraunhofer IOSB and KIT IES was hosted at the IOSB in Karlsruhe. For a week from the 2nd to the 6th July the doctoral students extensive reports on the status of their research. The results and ideas presented at the workshop are collected in this book in the form of detailed technical reports