Designing a fruit identification algorithm in orchard conditions to develop robots using video processing and majority voting based on hybrid artificial neural network

The first step in identifying fruits on trees is to develop garden robots for different purposes such as fruit harvesting and spatial specific spraying. Due to the natural conditions of the fruit orchards and the unevenness of the various objects throughout it, usage of the controlled conditions is very difficult. As a result, these operations should be performed in natural conditions, both in light and in the background. Due to the dependency of other garden robot operations on the fruit identification stage, this step must be performed precisely. Therefore, the purpose of this paper was to design an identification algorithm in orchard conditions using a combination of video processing and majority voting based on different hybrid artificial neural networks. The different steps of designing this algorithm were: (1) Recording video of different plum orchards at different light intensities; (2) converting the videos produced into its frames; (3) extracting different color properties from pixels; (4) selecting effective properties from color extraction properties using hybrid artificial neural network-harmony search (ANN-HS); and (5) classification using majority voting based on three classifiers of artificial neural network-bees algorithm (ANN-BA), artificial neural network-biogeography-based optimization (ANN-BBO), and artificial neural network-firefly algorithm (ANN-FA). Most effective features selected by the hybrid ANN-HS consisted of the third channel in hue saturation lightness (HSL) color space, the second channel in lightness chroma hue (LCH) color space, the first channel in L*a*b* color space, and the first channel in hue saturation intensity (HSI). The results showed that the accuracy of the majority voting method in the best execution and in 500 executions was 98.01% and 97.20%, respectively. Based on different performance evaluation criteria of the classifiers, it was found that the majority voting method had a higher performance.European Union (EU) under Erasmus+ project entitled “Fostering Internationalization in Agricultural Engineering in Iran and Russia” [FARmER] with grant number 585596-EPP-1-2017-1-DE-EPPKA2-CBHE-JPinfo:eu-repo/semantics/publishedVersio

Nondestructive Detection of Codling Moth Infestation in Apples Using Pixel-Based NIR Hyperspectral Imaging with Machine Learning and Feature Selection

Codling moth (CM) (Cydia pomonella L.), a devastating pest, creates a serious issue for apple production and marketing in apple-producing countries. Therefore, effective nondestructive early detection of external and internal defects in CM-infested apples could remarkably prevent postharvest losses and improve the quality of the final product. In this study, near-infrared (NIR) hyperspectral reflectance imaging in the wavelength range of 900–1700 nm was applied to detect CM infestation at the pixel level for three organic apple cultivars, namely Gala, Fuji and Granny Smith. An effective region of interest (ROI) acquisition procedure along with different machine learning and data processing methods were used to build robust and high accuracy classification models. Optimal wavelength selection was implemented using sequential stepwise selection methods to build multispectral imaging models for fast and effective classification purposes. The results showed that the infested and healthy samples were classified at pixel level with up to 97.4% total accuracy for validation dataset using a gradient tree boosting (GTB) ensemble classifier, among others. The feature selection algorithm obtained a maximum accuracy of 91.6% with only 22 selected wavelengths. These findings indicate the high potential of NIR hyperspectral imaging (HSI) in detecting and classifying latent CM infestation in apples of different cultivars

GuavaNet: A deep neural network architecture for automatic sensory evaluation to predict degree of acceptability for Guava by a consumer

This thesis is divided into two parts:Part I: Analysis of Fruits, Vegetables, Cheese and Fish based on Image Processing using Computer Vision and Deep Learning: A Review. It consists of a comprehensive review of image processing, computer vision and deep learning techniques applied to carry out analysis of fruits, vegetables, cheese and fish.This part also serves as a literature review for Part II.Part II: GuavaNet: A deep neural network architecture for automatic sensory evaluation to predict degree of acceptability for Guava by a consumer. This part introduces to an end-to-end deep neural network architecture that can predict the degree of acceptability by the consumer for a guava based on sensory evaluation

Pixelated Semantic Colorization

While many image colorization algorithms have recently shown the capability of producing plausible color versions from gray-scale photographs, they still suffer from limited semantic understanding. To address this shortcoming, we propose to exploit pixelated object semantics to guide image colorization. The rationale is that human beings perceive and distinguish colors based on the semantic categories of objects. Starting from an autoregressive model, we generate image color distributions, from which diverse colored results are sampled. We propose two ways to incorporate object semantics into the colorization model: through a pixelated semantic embedding and a pixelated semantic generator. Specifically, the proposed convolutional neural network includes two branches. One branch learns what the object is, while the other branch learns the object colors. The network jointly optimizes a color embedding loss, a semantic segmentation loss and a color generation loss, in an end-to-end fashion. Experiments on PASCAL VOC2012 and COCO-stuff reveal that our network, when trained with semantic segmentation labels, produces more realistic and finer results compared to the colorization state-of-the-art

Hyperspectral Imaging and Their Applications in the Nondestructive Quality Assessment of Fruits and Vegetables

Over the past decade, hyperspectral imaging has been rapidly developing and widely used as an emerging scientific tool in nondestructive fruit and vegetable quality assessment. Hyperspectral imaging technique integrates both the imaging and spectroscopic techniques into one system, and it can acquire a set of monochromatic images at almost continuous hundreds of thousands of wavelengths. Many researches based on spatial image and/or spectral image processing and analysis have been published proposing the use of hyperspectral imaging technique in the field of quality assessment of fruits and vegetables. This chapter presents a detailed overview of the introduction, latest developments and applications of hyperspectral imaging in the nondestructive assessment of fruits and vegetables. Additionally, the principal components, basic theories, and corresponding processing and analytical methods are also reported in this chapter

Multisensory System for Fruit Harvesting Robots. Experimental Testing in Natural Scenarios and with Different Kinds of Crops

The motivation of this research was to explore the feasibility of detecting and locating fruits from different kinds of crops in natural scenarios. To this end, a unique, modular and easily adaptable multisensory system and a set of associated pre-processing algorithms are proposed. The offered multisensory rig combines a high resolution colour camera and a multispectral system for the detection of fruits, as well as for the discrimination of the different elements of the plants, and a Time-Of-Flight (TOF) camera that provides fast acquisition of distances enabling the localisation of the targets in the coordinate space. A controlled lighting system completes the set-up, increasing its flexibility for being used in different working conditions. The pre-processing algorithms designed for the proposed multisensory system include a pixel-based classification algorithm that labels areas of interest that belong to fruits and a registration algorithm that combines the results of the aforementioned classification algorithm with the data provided by the TOF camera for the 3D reconstruction of the desired regions. Several experimental tests have been carried out in outdoors conditions in order to validate the capabilities of the proposed system.The motivation of this research was to explore the feasibility of detecting and locating fruits from different kinds of crops in natural scenarios. To this end, a unique, modular and easily adaptable multisensory system and a set of associated pre-processing algorithms are proposed. The offered multisensory rig combines a high resolution colour camera and a multispectral system for the detection of fruits, as well as for the discrimination of the different elements of the plants, and a Time-Of-Flight (TOF) camera that provides fast acquisition of distances enabling the localisation of the targets in the coordinate space. A controlled lighting system completes the set-up, increasing its flexibility for being used in different working conditions. The pre-processing algorithms designed for the proposed multisensory system include a pixel-based classification algorithm that labels areas of interest that belong to fruits and a registration algorithm that combines the results of the aforementioned classification algorithm with the data provided by the TOF camera for the 3D reconstruction of the desired regions. Several experimental tests have been carried out in outdoors conditions in order to validate the capabilities of the proposed system

Multispectral Method for Apple Defect Detection using Hyperspectral Imaging System

Hyperspectral imaging is a non-destructive detection technology and a powerful analytical tool that integrates conventional imaging and spectroscopy to get both spatial and spectral information from the objects for food safety and quality analysis. A recently developed hyperspectral imaging system was used to investigate the wavelength between 530nm and 835nm to detect defects on Red Delicious apples. The combination of band ratio method and relative intensity method were developed in this paper, which using the multispectral wavebands selected from hyperspectral images. The results showed that the hyperspectral imaging system with the properly developed multispectral method could generally identify 95% of the defects on apple surface accurately. The developed algorithms could help enhance food safety and protect public health while reducing human error and labor cost for food industr