Automated systems based on machine vision for inspecting citrus fruits from the field to postharvest - A review

[EN] Computer vision systems are becoming a scientific but also a commercial tool for food quality assessment. In the field, these systems can be used to predict yield, as well as for robotic harvesting or the early detection of potentially dangerous diseases. In postharvest handling, it is mostly used for the automated inspection of the external quality of the fruits and for sorting them into commercial categories at very high speed. More recently, the use of hyperspectral imaging is allowing not only the detection of defects in the skin of the fruits but also their association to certain diseases of particular importance. In the research works that use this technology, wavelengths that play a significant role in detecting some of these dangerous diseases are found, leading to the development of multispectral imaging systems that can be used in industry. This article reviews recent works that use colour and non-standard computer vision systems for the automated inspection of citrus. It explains the different technologies available to acquire the images and their use for the non-destructive inspection of internal and external features of these fruits. Particular attention is paid to inspection for the early detection of some dangerous diseases like citrus canker, black spot, decay or citrus Huanglongbing.This work was supported by the Instituto Nacional de Investigacion y Tecnologia Agraria y Alimentaria (INIA) through projects RTA2012-00062-C04-01 and RTA2012-00062-C04-03 with the support of European FEDER funds. The authors would like to thank and acknowledge the contributions that were made by all the students, postdocs, technicians and visiting scholars in the Precision Agriculture Laboratory at the University of Florida and the Computer Vision Laboratory at the Agricultural Engineering Centre of IVIA.Cubero García, S.; Lee, WS.; Aleixos Borrás, MN.; Albert Gil, FE.; Blasco Ivars, J. (2016). Automated systems based on machine vision for inspecting citrus fruits from the field to postharvest - A review. Food and Bioprocess Technology. 9(10):1623-1639. https://doi.org/10.1007/s11947-016-1767-1S16231639910Adebayo, S. E., Hashim, N., Abdan, K., & Hanafi, M. (2016). Application and potential of backscattering imaging techniques in agricultural and food processing—a review. Journal of Food Engineering, 169, 155–164.Aleixos, N., Blasco, J., Navarrón, F., & Moltó, E. (2002). Multispectral inspection of citrus in real time using machine vision and digital signal processors. Computers and Electronics in Agriculture, 33(2), 121–137.Annamalai, P., & Lee, W. S. (2003). Citrus yield mapping system using machine vision. ASAE Paper No. 031002. St. Joseph: ASAE.Annamalai, P., & Lee, W. S. (2004). Identification of green citrus fruits using spectral characteristics. ASAE Paper No. FL04–1001. St. Joseph: ASAE.Balasundaram, D., Burks, T. F., Bulanon, D. M., Schubert, T., & Lee, W. S. (2009). Spectral reflectance characteristics of citrus canker and other peel conditions of grapefruit. Postharvest Biology and Technology, 51, 220–226.Bansal, R., Lee, W. S., & Satish, S. (2013). Green citrus detection using fast Fourier transform (FFT) leakage. Precision Agriculture, 14(1), 59–70.Barreiro, P., Zheng, C., Sun, D.-W., Hernández-Sánchez, N., Pérez-Sánchez, J. M., & Ruiz-Cabello, J. (2008). Non-destructive seed detection in mandarins: comparison of automatic threshold methods in FLASH and COMSPIRA MRIs. Postharvest Biology and Technology, 47, 189–198.Basavaprasad, B., & Ravi, M. (2014). A comparative study on classification of image segmentation methods with a focus on graph based techniques. International Journal of Research in Engineering and Technology, 3, 310–315.Birth, G. S. (1976). How light interacts with foods. In: Gafney J.Jr.(Ed.), Quality detection in foods (pp. 6–11). St. Joseph: ASAE.Blanc, P.G.R., Blasco, J., Moltó, E., Gómez-Sanchis, J., & Cubero, S. (2010) System for the automatic selective separation of rotten citrus fruits. Patent number EP2133157 A1 CN101678405A, EP2133157A4, EP2133157B1, US20100121484Blasco, J., Aleixos, N., & Moltó, E. (2007a). Computer vision detection of peel defects in citrus by means of a region oriented segmentation algorithm. Journal of Food Engineering, 81(3), 535–543.Blasco, J., Aleixos, N., Gómez, J., & Moltó, E. (2007b). Citrus sorting by identification of the most common defects using multispectral computer vision. Journal of Food Engineering, 83(3), 384–393.Blasco, J., Aleixos, N., Gómez-Sanchis, J., & Moltó, E. (2009). Recognition and classification of external skin damages in citrus fruits using multispectral data and morphological features. Biosystems Engineering, 103(2), 137–145.Blasco, J., Cubero, S., & Moltó, E. (2016). Quality evaluation of citrus fruits. In D.-W. Sun (Ed.), Computer vision technology for food quality evaluation (2nd ed.). San Diego: Academic Press.Bulanon, D. M., Burks, T. F., & Alchanatis, V. (2009). Image fusion of visible and thermal images for fruit detection. Biosystems Engineering, 103, 12–22.Bulanon, D.M., Burks, T.F., Kim, D.G., & Ritenour, M.A. (2013). Citrus black spot detection using hyperspectral image analysis. Agricultural Engineering International: CIGR Journal, 15,(3)171.Burks, T. F., Villegas, F., Hannan, M. W., & Flood, S. (2003). Engineering and horticultural aspects of robotic fruit harvesting: opportunities and constraints. HortTechnology, 15(1), 79–87.Campbell, B. L., Nelson, R. G., Ebel, R. C., Dozier, W. A., Adrian, J. L., & Hockema, B. R. (2004). Fruit quality characteristics that affect consumer preferences for Satsuma mandarins. Hortscience, 39(7), 1664–1669.Chinchuluun, R., Lee, W. S., & Ehsani, R. (2009). Machine vision system for determining citrus count and size on a canopy shake and catch harvester. Applied Engineering in Agriculture, 25(4), 451–458.Choi, D., Lee, W. S., Ehsani, R., & Roka, F. M. (2015). A machine vision system for quantification of citrus fruit dropped on the ground under the canopy. Transactions of the ASABE, 58(4), 933–946.Codex Alimentarius, (2011). Codex standard for oranges. Available at: http://www.codexalimentarius.org/download/standards/10372/CXS_245e.pdf . Accessed March 2016Cubero, S., Aleixos, N., Albert, A., Torregrosa, A., Ortiz, C., García-Navarrete, O., & Blasco, J. (2014a). Optimised computer vision system for automatic pre-grading of citrus fruit in the field using a mobile platform. Precision Agriculture, 15(1), 80–94.Cubero, S., Aleixos, N., Moltó, E., Gómez-Sanchis, J., & Blasco, J. (2011). Advances in machine vision applications for automatic inspection and quality evaluation of fruits and vegetables. Food and Bioprocess Technology, 4(4), 487–504.Cubero, S., Diago, M. P., Blasco, J., Tardáguila, J., Millán, B., & Aleixos, N. (2014b). A new method for pedicel/peduncle detection and size assessment of grapevine berries and other fruits by image analysis. Biosystems Engineering, 117, 62–72.Dong, C.-W., Ye, Y., Zhang, J.-Q., Zhu, H.-K., & Liu, F. (2014). Detection of thrips defect on green-peel citrus using hyperspectral imaging technology combining PCA and B-Spline lighting correction method. Journal of Integrative Agriculture, 13(10), 2229–2235.FAOSTAT (2012). URL: http://faostat.fao.org http://www.fao.org/fileadmin/templates/est/COMM_MARKETS_MONITORING/Citrus/Documents/CITRUS_BULLETIN_2012.pdf . Accessed March 2016.Farrell, T. J., Patterson, M. S., & Wilson, B. (1992). A diffusion-theory model of spatially resolved steady-state diffuse reflectance for the noninvasive determination of tissue optical-properties in vivo. Medical Physics, 19, 879–888.Flood, S. J., Burks, T. F., & Teixeira, A. A. (2006). Physical properties of oranges in response to applied gripping forces for robotic harvesting. Transactions of ASAE, 49(2), 341–346.Gaffney, J. J. (1973). Reflectance properties of citrus fruit. Transactions of ASAE, 16(2), 310–314.Garcia-Ruiz, F., Sankaran, S., Maja, J. M., Lee, W. S., Rasmussen, J., & Ehsani, R. (2013). Comparison of two aerial imaging platforms for identification of Huanglongbing infected citrus trees. Computers and Electronics in Agriculture, 91, 106–115.Gómez, J., Blasco, J., Moltó, E., & Camps-Valls, G. (2007). Hyperspectral detection of citrus damage with a Mahalanobis kernel classifier. Electronics Letters, 43(20), 1082–1084.Gómez-Sanchis, J., Blasco, J., Soria-Olivas, E., Lorente, D., Escandell-Montero, P., Martínez-Martínez, J. M., Martínez-Sober, M., & Aleixos, N. (2013). Hyperspectral LCTF-based system for classification of decay in mandarins caused by Penicillium digitatum and Penicillium italicum using the most relevant bands and non-linear classifiers. Postharvest Biology and Technology, 82, 76–86.Gómez-Sanchis, J., Gómez-Chova, L., Aleixos, N., Camps-Valls, G., Montesinos-Herrero, C., Moltó, E., & Blasco, J. (2008). Hyperspectral system for early detection of rottenness caused by Penicillium digitatum in mandarins. Journal of Food Engineering, 89(1), 80–86.Gómez-Sanchis, J., Lorente, D., Soria-Olivas, E., Aleixos, N., Cubero, S., & Blasco, J. (2014). Development of a hyperspectral computer vision system based on two liquid crystal tuneable filters for fruit inspection. Application to detect citrus fruits decay. Food and Bioprocess Technology, 7, 1047–1056.Gómez-Sanchis, J., Martín-Guerrero, J. D., Soria-Olivas, E., Martínez-Sober, M., Magdalena-Benedito, R., & Blasco, J. (2012). Detecting rottenness caused by Penicillium in citrus fruits using machine learning techniques. Expert Systems with Applications, 39(1), 780–785.Gong, A., Yu, J., He, Y., & Qiu, Z. (2013). Citrus yield estimation based on images processed by an android mobile phone. Biosystems Engineering, 115, 162–170.Gottwald, T. R., Graham, J. H., & Schubert, T. S. (2002). Citrus canker: the pathogen and its impact. Plant Health Progress. doi: 10.1094/PHP-2002-0812-01-RV.Hannan, M., Burks, T. F., & Bulanon, D.M. (2009). A machine vision algorithm for orange fruit detection. The CIGR Ejournal. Manuscript 1281. Vol XI. December 2009.Harrell, R. C., Adsit, P. D., & Slaughter, D. C. (1985). Real-time vision-servoing of a robotic tree-fruit harvester. ASAE Paper No (pp. 85–3550). St. Joseph: ASAE.Hernández-Sánchez, N., Barreiro, P., & Ruiz-Cabello, J. (2006). On-line identification of seeds in mandarins with magnetic resonance imaging. Biosystems Engineering, 95, 529–536.Holmes, G. J., & Eckert, J. W. (1999). Sensitivity of Penicillium digitatum and P. italicum to postharvest citrus fungicides in California. Phytopathology, 89(9), 716–721.Iqbal, S. M., Gopal, A., Sankaranarayanan, P. E., & Nair, A. B. (2016). Classification of selected citrus fruits based on color using machine vision system. International Journal of Food Properties, 19, 272–288.Jackson, J. E. (1991). A user’s guide to principal components. New York: Wiley.Jafari, A., Fazayeli, A., & Zarezadeh, M. R. (2014). Estimation of orange skin thickness based on visual texture coarseness. Biosystems Engineering, 117, 73–82.Jiménez-Cuesta, M. J., Cuquerella, J., & Martínez-Jávega, J. M. (1981). Determination of a color index for citrus fruit degreening. In Proceedings of the International Society of Citriculture, 2, 750–753.Kim, D. G., Burks, T. F., Qin, J., & Bulanon, D. M. (2009). Classification of grapefruit peel diseases using color texture feature analysis. International Journal of Agricultural and Biological Engineering, 2, 41–50.Kim, D. G., Burks, T. F., Ritenour, M. A., & Qin, J. (2014). Citrus black spot detection using hyperspectral imaging. International Journal of Agricultural and Biological Engineering, 7, 20–27.Kohno, Y., Kondo, N., Iida, M., Kurita, M., Shiigi, T., Ogawa, Y., Kaichi, T., & Okamoto, S. (2011). Development of a mobile grading machine for citrus fruit. Engineering in Agriculture, Environment and Food, 4, 7–11.Kondo, N., Kuramoto, M., Shimizu, H., Ogawa, Y., Kurita, M., Nishizu, T., Chong, V. K., & Yamamoto, K. (2009). Identification of fluorescent substance in mandarin orange skin for machine vision system to detect rotten citrus fruits. Engineering in Agriculture, Environment and Food, 2, 54–59.Kurita, M., Kondo, N., Shimizu, H., Ling, P. P., Falzea, P. D., Shiigi, T., Ninomiya, K., Nishizu, T., & Yamamoto, K. (2009). A double image acquisition system with visible and UV LEDs for citrus fruit. Journal of Robotics and Mechatronics, 21, 533–540.Kurtulmus, F., Lee, W. S., & Vardar, A. (2011). Green citrus detection using eigenfruit, color and circular Gabor texture features under natural outdoor conditions. Computers and Electronics in Agriculture, 78(2), 140–149.Ladaniya, M. S. (2010). Citrus fruit: biology, technology and evaluation. San Diego: Academic Press.Li, H., Lee, W. S., & Wang, K. (2016). Immature green citrus fruit detection and counting based on fast normalized cross correlation (FNCC) using natural outdoor colour images. Precision Agriculture. doi: 10.1007/s11119-016-9443-z.Li, H., Lee, W. S., Wang, K., Ehsani, R., & Yang, C. (2014). Extended spectral angle mapping (ESAM) for citrus greening disease detection using airborne hyperspectral imaging. Precision Agriculture, 15, 162–183.Li, J., Rao, X., & Ying, Y. (2011). Detection of common defects on oranges using hyperspectral reflectance imaging. Computers and Electronics in Agriculture, 78, 38–48.Li, J., Rao, X., & Ying, Y. (2012a). Development of algorithms for detecting citrus canker based on hyperspectral reflectance imaging. Journal of the Science of Food and Agriculture, 92, 125–134.Li, J., Rao, X., Wang, F., Wu, W., & Ying, Y. (2013). Automatic detection of common surface defects on oranges using combined lighting transform and image ratio methods. Postharvest Biology and Technology, 82, 59–69.Li, J., Rao, X., Ying, Y., & Wang, D. (2010). Detection of navel oranges canker based on hyperspectral imaging technology. Nongye Gongcheng Xuebao/Transactions of the Chinese Society of Agricultural Engineering, 26, 222–228.Li, X., Lee, W. S., Li, M., Ehsani, R., Mishra, A., Yang, C., & Mangan, R. (2012b). Spectral difference analysis and airborne imaging classification for citrus greening infected trees. Computers and Electronics in Agriculture, 83, 32–46.Li, X., Lee, W. S., Li, M., Ehsani, R., Mishra, A. R., Yang, C., & Mangan, R. L. (2015). Feasibility study on Huanglongbing (citrus greening) detection based on WorldView-2 satellite imagery. Biosystems Engineering, 132, 28–38.Lopes, L. B., VanDeWall, H., Li, H. T., Venugopal, V., Li, H. K., Naydin, S., Hosmer, J., Levendusky, M., Zheng, H., Bentley, M. V., Levin, R., & Hass, M. A. (2010). Topical delivery of lycopene using microemulsions: enhanced skin penetration and tissue antioxidant activity. Journal of Pharmaceutical Sciences, 99, 1346–1357.López, J. J., Cobos, M., & Aguilera, E. (2011). Computer-based detection and classification of flaws in citrus fruits. Neural Computing and Applications, 20, 975–981.López-García, F., Andreu, G., Blasco, J., Aleixos, N., & Valiente, J. M. (2010). Automatic detection of skin defects in citrus fruits using a multivariate image analysis approach. Computers and Electronics in Agriculture, 71, 189–197.Lorente, D., Aleixos, N., Gómez-Sanchis, J., Cubero, S., & Blasco, J. (2013a). Selection of optimal wavelength features for decay detection in citrus fruit using the ROC curve and neural networks. Food and Bioprocess Technology, 6(2), 530–541.Lorente, D., Aleixos, N., Gómez-Sanchis, J., Cubero, S., García-Navarrete, O. L., & Blasco, J. (2012). Recent advances and applications of hyperspectral imaging for fruit and vegetable quality assessment. Food and Bioprocess Technology, 5(4), 1121–1142.Lorente, D., Blasco, J., Serrano, A. J., Soria-Olivas, E., Aleixos, N., & Gómez-Sanchis, J. (2013b). Comparison of ROC feature selection method for the detection of decay in citrus fruit using hyperspectral images. Food and Bioprocess Technology, 6(12), 3613–3619.Lorente, D., Zude, M., Regen, C., Palou, L., Gómez-Sanchis, J., & Blasco, J. (2013c). Early decay detection in citrus fruit using laser-light backscattering imaging. Postharvest Biology and Technology, 86, 424–430.Lorente, D., Zude, M., Idler, C., Gómez-Sanchis, J., & Blasco, J. (2015). Laser-light backscattering imaging for early decay detection in citrus fruit using both a statistical and a physical model. Journal of Food Engineering, 154, 76–85.Maf Industries. (2016). VIOTEC brochure. http://mafindustries.com/wp-content/uploads/2015/02/viotec3.pdf . Accessed March 2016.Magwaza, L. S., Opara, U. L., Nieuwoudt, H., Cronje, P. J. R., Saeys, W., & Nicolaï, B. (2012). NIR spectroscopy applications for internal and external quality analysis of citrus fruit—a review. Food and Bioprocess Technology, 5(2), 425–444.Mehta, S. S., & Burks, T. F. (2014). Vision-based control of robotic manipulator for citrus harvesting. Computers and Electronics in Agriculture, 102, 146–158.Moltó, E., Blasco, J., & Gómez-Sanchis, J. (2010). Analysis of hyperspectral images of citrus fruits. In D.-W. Sun (Ed.), Hyperspectral imaging for food quality analysis and control (pp. 321–348). California: Academic Press.Moltó, E., Plá, F., & Juste, F. (1992). Vision systems for the location of citrus fruit in a tree canopy. Journal of Agricultural Engineering Research, 52, 101–110.Momin, A., Kondo, N., Kuramoto, M., Ogawa, Y., Yamamoto, K., & Shiigi, T. (2012). Investigation of excitation wavelength for fluorescence emission of citrus peels based on UV-VIS spectra. Engineering in Agriculture, Environment and Food, 5, 126–132.Momin, A., Kondo, N., Ogawa, Y., Ido, K., & Ninomiya, K. (2013b). Patterns of fluorescence associated with citrus peel defects. Engineering in Agriculture, Environment and Food, 6, 54–60.Momin, A., Kuramoto, M., Kondo, N., Ido, K., Ogawa, Y., Shiigi, T., & Ahmad, U. (2013a). Identification of UV-fluorescence components for detecting peel defects of lemon and yuzu using machine vision. Engineering in Agriculture, Environment and Food, 6, 165–171.Morgan, S. P., & Stockford, I. M. (2003). Surface-reflection elimination in polarization imaging of superficial tissue. Optics Letters, 28, 114–116.Niphadkar, N. P., Burks, T. F., Qin, J., & Ritenour, M. (2013b). Edge effect compensation for citrus canker lesion detection due to light source variation—a hyperspectral imaging application. Agricultural Engineering International: CIGR Journal, 15, 314–327.Niphadkar, N. P., Burks, T. F., Qin, J. W., & Ritenour, M. A. (2013a). Estimation of citrus canker lesion size using hyperspectral reflectance imaging. International Journal of Agricultural and Biological Engineering, 6, 41–51.Obenland, D., Margosan, D., Smilanick, J. L., & Mackey, B. (2010). Ultraviolet fluorescence to identify navel oranges with poor peel quality and decay. HortTechnology, 20, 991–995.Ogawa, Y., Abdul, M. M., Kuramoto, M., Kohno, Y., Shiigi, T., Yamamoto, K., & Kondo, K. (2011). Rotten part detection on citrus fruit surfaces by use of fluorescent images. The Review of Laser Engineering, 394, 255–261.Okamoto, H., & Lee, W. S. (2009). Green citrus detection using hyperspectral imaging. Computers and Electronics in Agriculture, 66(2), 201–208.Omid, M., Khojastehnazhand, M., & Tabatabaeefar, A. (2010). Estimating volume and mass of citrus fruits by image processing technique. Journal of Food Engineering, 100, 315–321.Ottavian, M., Barolo, M., & García-Muñoz, S. (2013). Maintenance of machine vision systems for product quality assessment. Part I. Addressing changes in lighting conditions. Industrial & Engineering Chemistry Research, 52, 12309–12318.Ottavian, M., Barolo, M., & García-Muñoz, S. (2014). Maintenance of machine vision systems for product quality assessment. Part II. Addressing camera replacement. Industrial & Engineering Chemistry Research, 53, 1529–1536.Palou, L. (2014). Penicillium digitatum, Penicillium italicum (green mold, blue mold). In S. Bautista-Baños (Ed.), Postharvest decay. Control strategies. London: Elsevier.Palou, L., Smilanick, J. L., Montesinos-Herrero, C., Valencia-Chamorro, S., & Pérez-Gago, M. B. (2011). Novel approaches for postharvest preservation of fresh citrus fruits. In Slaker (Ed.), Citrus fruits: properties, consumption and nutrition. New York: Nova Science Publishers, Inc..Pongnumkul, S., Chaovalit, P., & Surasvadi, N. (2015). Applications of smartphone-based sensors in agriculture: a systematic review of research. Journal of Sensors, Open Access Article ID 195308.Pourreza, A., Lee, W. S., Ehsani, R., Schueller, J. K., & Raveh, E. (2015a). An optimum method for real-time in-field detection of Huanglongbing disease using a vision sensor. Computers and Electronics in Agriculture, 110, 221–232.Pourreza, A., Lee, W. S., Etxeberria, E., & Banerjee, A. (2015b). An evaluation of a vision based sensor performance in Huanglongbing disease identification. Biosystems Engineering, 130, 13–22.Qin, J., Burks, T. F., Kim, M. S., Chao, K., & Ritenour, M. A. (2008). Citrus canker detection using hyperspectral reflectance imaging and PCA-based image classification method. Sensing and Instrumentation for Food Quality and Safety, 2(3), 168–177.Qin, J., Burks, T. F., Ritenour, M. A., & Gordon Bonn, W. (2009). Detection of citrus canker using hyperspectral reflectance imaging with spectral information divergence. Journal of Food Engineering, 93, 183–191.Qin, J., Burks, T. F., Zhao, X., Niphadkar, N., & Ritenour, M. A. (2011). Multispectral detection of citrus canker using hyperspectral band selection. Transactions of the ASABE, 54, 2331–2341.Qin, J., Burks, T. F., Zhao, X., Niphadkar, N., & Ritenour, M. A. (2012). Development of a two-band spectral imaging system for real-time citrus canker detection. Journal of Food Engineering, 108, 87–93.Sengupta, S., & Lee, W. S. (2014). Identification and determination of the number of immature green citrus fruit under different ambient light conditions.

Optimised computer vision system for automatic pre-grading of citrus fruit in the field using a mobile platform

The mechanisation and automation of citrus harvesting is considered to be one of the best options to reduce production costs. Computer vision technology has been shown to be a useful tool for fresh fruit and vegetable inspection, and is currently used in post-harvest fruit and vegetable automated grading systems in packing houses. Although computer vision technology has been used in some harvesting robots, it is not commonly utilised in fruit grading during harvesting due to the difficulties involved in adapting it to field conditions. Carrying out fruit inspection before arrival at the packing lines could offer many advantages, such as having an accurate fruit assessment in order to decide among different fruit treatments or savings in the cost of transport and marketing non-commercial fruit. This work presents a computer vision system, mounted on a mobile platform where workers place the harvested fruits, that was specially designed for sorting fruit in the field. Due to the specific field conditions, an efficient and robust lighting system, very low-power image acquisition and processing hardware, and a reduced inspection chamber had to be developed. The equipment is capable of analysing fruit colour and size at a speed of eight fruits per second. The algorithms developed achieved prediction accuracy with an R-2 coefficient of 0.993 for size estimation and an R-2 coefficient of 0.918 for the colour index.This research work has been funded by the Instituto Nacional de Investigacion y Tecnologia Agraria y Alimentaria de Espana (INIA) and the European FEDER funds (projects RTA2009-00118-C02-01 and RTA2009-00118-C02-02). The authors wish to thank the collaboration of the company Argiles Diseny i Fabricacio, S.L.Cubero García, S.; Aleixos Borrás, MN.; Albert Gil, FE.; Torregrosa, A.; Ortiz Sánchez, MC.; García Navarrete, OL.; Blasco Ivars, J. (2014). Optimised computer vision system for automatic pre-grading of citrus fruit in the field using a mobile platform. Precision Agriculture. 15(1):80-94. doi:10.1007/s11119-013-9324-7S8094151Baeten, J., Donné, K., Boedrij, S., Beckers, W., & Claesen, E. (2008). Autonomous fruit picking machine: A robotic apple harvester. Springer Tracts in Advanced Robotics, 42, 531–539.Blasco, J., Aleixos, N., Gómez-Sanchis, J., & Moltó, E. (2009a). Recognition and classification of external skin damage in citrus fruits using multispectral data and morphological features. Biosystems Engineering, 103, 137–145.Blasco, J., Aleixos, N., Roger, J. M., Rabatel, G., & Moltó, E. (2002). Robotic weed control using machine vision. Biosystems Engineering, 83(2), 149–157.Blasco, J., Cubero, S., Gómez-Sanchis, J., Mira, P., & Moltó, E. (2009b). Development of a machine for the automatic sorting of pomegranate (Punica granatum) arils based on computer vision. Journal of Food Engineering, 90, 27–34.Chong, V. K., Monta, M., Ninomiya, K., Kondo, N., Namba, K., Terasaki, E., et al. (2008). Development of mobile eggplant grading robot for dynamic in-field variability sensing––manufacture of robot and performance test. Engineering in Agriculture, Environment and Food, 1(2), 68–76.Coppock, G. E., & Jutras, P. J. (1960). Mechanizing citrus fruit harvesting. Transactions of the ASAE, 3(2), 130–132.Cubero, S., Aleixos, N., Moltó, E., Gómez-Sanchis, J., & Blasco, J. (2011). Advances in machine vision applications for automatic inspection and quality evaluation of fruits and vegetables. Food and Bioprocess Technology, 4(4), 487–504.Cubero, S., Moltó, E., Gutiérrez, A., Aleixos, N., García-Navarrete, O. L., Juste, F., et al. (2010). Real-time inspection of fruit on a mobile harvesting platform in field conditions using computer vision. Progress in Agricultural Engineering Science, 6, 1–16.DOGV. (2006). Diari Oficial de la Comunitat Valenciana, 5346, 30321–30328.Edan, Y., Rogozin, D., Flash, T., & Miles, G. E. (2000). Robotic melon harvesting. IEEE Transactions on Robotics and Automation, 16(6), 831–834.Ehsani, M. R., Grift, T. E., Maja, J. M., & Zhong, D. (2009). Two fruit counting techniques for citrus mechanical harvesting machinery. Computers and Electronics in Agriculture, 65(2), 186–191.Feng, G., Qixin, C., & Masateru, N. (2008). Fruit detachment and classification method for strawberry harvesting robot. International Journal of Advanced Robotic Systems, 5(1), 41–48.Gómez-Sanchis, J., Gómez-Chova, L., Aleixos, N., Camps-Valls, G., Montesinos-Herrero, C., Moltó, E., et al. (2008). Hyperspectral system for early detection of rottenness caused by Penicillium digitatum in mandarins. Journal of Food Engineering, 89(1), 80–86.HunterLab. (2008). Applications note, 8(9) http://www.hunterlab.com/appnotes/an08_96a.pdf . Accessed Nov 2012.Jiménez-Cuesta, M.J., Cuquerella, J., & Martínez-Jávega, J.M. (1981). Determination of a color index for citrus fruit degreening. In: Proceedings of the International Society of Citriculture, Tokyo (Japan), vol. 2 (pp. 750–753).Jutras, P.J., & Coppock, G.E. (1958). Mechanization of citrus fruit picking. Florida State Horticultural Society, 71, 201,204.Kohno, Y., Kondo, N., Iida, M., Kurita, M., Shiigi, T., Ogawa, Y., et al. (2011). Development of a mobile grading machine for citrus fruit. Engineering in Agriculture, Environment and Food, 4(1), 7–11.Kondo, N. (2009). Robotization in fruit grading system. Sensors and Instrumentation for Food Quality, 3, 81–87.Lee, W. S., & Slaughter, D. C. (2004). Recognition of partially occluded plant leaves using a modified Watershed algorithm. Transactions of the ASAE, 47, 1269–1280.Lee, W. S., Slaughter, D. C., & Giles, D. K. (1999). Robotic weed control system for tomatoes. Precision Agriculture, 1(1), 95–113.Li, Z., Li, P., & Liu, J. (2011). Physical and mechanical properties of tomato fruits as related to robot harvesting. Journal of Food Engineering, 103(2), 170–178.Lorente, D., Aleixos, N., Gómez-Sanchis, J., Cubero, S., García-Navarrete, O. L., & Blasco, J. (2012). Recent advances and applications of hyperspectral imaging for fruit and vegetable quality assessment. Food and Bioprocess Technology, 5(4), 1121–1142.Mazzetto, F., Calcante, A., Mena, A., & Vercesi, A. (2010). Integration of optical and analogue sensors for monitoring canopy health and vigour in precision viticulture. Precision Agriculture, 11(6), 636–649.McBratney, A., Whelan, B., Ancev, T., & Bouma, J. (2005). Future directions of precision agriculture. Precision Agriculture, 6(1), 7–23.Mizushima, A., & Lu, R. (2011). Cost benefits analysis of in-field presorting for the apple industry. Applied Engineering in Agriculture, 27(1), 33–40.Muscato, G., Prestifilippo, M., Abbate, N., & Rizzuto, I. (2005). A prototype of an orange picking robot: Past history and experimental results. Industrial Robot, 32(2), 128–138.Nieuwenhuizen, A. T., Hofstee, J. W., & van Henten, E. J. (2010). Adaptive detection of volunteer potato plants in sugar beet fields. Precision Agriculture, 11, 433–447.Official Journal of European Communities. (2001). 14.09.2001. pp. L244/12–L244/18. http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2001:244:0012:0018:EN:PDF . Accessed May 2013.Ortiz, C., Blasco, J., Balasch, S., & Torregrosa, A. (2011). Shock absorbing surfaces for collecting fruit during the mechanical harvesting of citrus. Biosystems Engineering, 110, 2–9.Qiao, J., Sasao, A., Shibusawa, S., Kondo, N., & Morimoto, E. (2004). Mobile fruit grading robot (part1)––Development of a robotic system for grading sweet peppers. Journal of the Japanese Society of Agricultural Machinery (JSAM), 66(2), 113–122.Qiao, J., Sasao, A., Shibusawa, S., Kondo, N., & Morimoto, E. (2005). Mapping yield and quality using the mobile fruit grading robot. Biosystems Engineering, 90(2), 135–142.Ruiz-Altisent, M., Ortiz-Cañavate, J., & Valero, C. (2004). Fruit and vegetables harvesting systems. In: R. Dris and S. M. Jain (Eds.), Production practices and quality assessment of food crops, vol. 1: Preharvest practice (pp. 261–285). Dordrecht: Kluwer.Torregrosa, A., Gil, J., Ortiz, C., Ortí, E., & Martín, B. (2009). Mechanical harvesting of oranges and mandarins in Spain. Biosystems Engineering, 104(1), 18–24.Vidal, A., Talens, P., Prats-Montalbán, J. M., Cubero, S., Albert, F., & Blasco, J. (2012). In-line estimation of the standard colour index of citrus fruits using a computer vision system developed for a mobile platform. Food and Bioprocess Technology,. doi: 10.1007/s11947-012-1015-2

Machine vision applications in agriculture

Keynote paper. [Abstract]: With the trend of computers towards convergence with multimedia entertainment, tools for vision processing are becoming commonplace. This has led to the pursuit of a host of unusual applications in the National Centre for Engineering in Agriculture, in addition to work on vision guidance. These range from the identification of animal species, through the location of macadamia nuts as they are harvested and visual tracking for behaviour analysis of small marsupials to the measurement of the volume of dingo teeth

Preliminary technology utilization assessment of the robotic fruit harvester

The results of an analysis whose purpose was to examine the history and progress of mechanical fruit harvesting, to determine the significance of a robotic fruit tree harvester and to assess the available market for such a product are summarized. Background information that can be used in determining the benefit of a proof of principle demonstration is provided. Such a demonstration could be a major step toward the transfer of this NASA technology

Multispectral images of peach related to firmness and maturity at harvest

wo multispectral maturity classifications for red soft-flesh peaches (‘Kingcrest’, ‘Rubyrich’ and ‘Richlady’ n = 260) are proposed and compared based on R (red) and R/IR (red divided by infrared) images obtained with a three CCD camera (800 nm, 675 nm and 450 nm). R/IR histograms were able to correct the effect of 3D shape on light reflectance and thus more Gaussian histograms were produced than R images. As fruits ripened, the R/IR histograms showed increasing levels of intensity. Reference measurements such as firmness and visible spectra also varied significantly as the fruit ripens, firmness decreased while reflectance at 680 nm increased (chlorophyll absorption peak)

In-Line estimation of the standard colour index of citrus fruits using a computer vision system developed for a mobile platform

The final publication is available at Springer via http://dx.doi.org/10.1007/s11947-012-1015-2A key aspect for the consumer when it comes to deciding on a particular product is the colour. In order to make fruit available to consumers as early as possible, the collection of oranges and mandarins begins before they ripen fully and reach their typical orange colour. As a result, they are therefore subjected to certain degreening treatments, depending on their standard colour citrus index at harvest. Recently, a mobile platform that incorporates a computer vision system capable of pre-sorting the fruit while it is being harvested has been developed as an aid in the harvesting task. However, due to the restrictions of working in the field, the computer vision system developed for this machine is limited in its technology and processing capacity compared to conventional systems. This work shows the optimised algorithms for estimating the colour of citrus in-line that were developed for this mobile platform and its performance is evaluated against that of a spectrophotometer used as a reference in the analysis of colour in food. The results obtained prove that our analysis system predicts the colour index of citrus with a good reliability (R2 = 0.925) working in real time. Findings also show that it is effective for classifying harvested fruits in the field according to their colour. © 2012 Springer Science+Business Media New York.This work was partially funded by the INIA through research project RTA2009-00118-C02-01 with the support of European FEDER funds, and by the project PAID-05-11-2745, Vicerectorat d'Investigacio, Universitat Politecnica de Valencia.Vidal, A.; Talens Oliag, P.; Prats-Montalbán, JM.; Cubero García, S.; Albert Gil, FE.; Blasco Ivars, J. (2013). In-Line estimation of the standard colour index of citrus fruits using a computer vision system developed for a mobile platform. Food and Bioprocess Technology. 6(12):3412-3419. https://doi.org/10.1007/s11947-012-1015-2S34123419612Arzate-Vázquez, I., Chanona-Pérez, J. J., Perea-Flores, M. J., Calderón-Domínguez, G., Moreno-Armendáriz, M. A., Calvo, H., Godoy-Calderón, S., Quevedo, R., & Gutiérrez-López, G. (2011). Image processing applied to classification of avocado variety Hass (Persea americana Mill.) during the ripening process. Food and Bioprocess Technology, 4(7), 1307–1313.Blasco, J., Aleixos, N., Cubero, S., Gómez-Sanchis, J., & Moltó, E. (2009). Automatic sorting of satsuma (Citrus unshiu) segments using computer vision and morphological features. Computers and Electronics in Agriculture, 66, 1–8.Campbell, B. L., Nelson, R. G., Ebel, C. E., Dozier, W. A., Adrian, J. L., & Hockema, B. R. (2004). Fruit quality characteristics that affect consumer preferences for satsuma mandarins. HortScience, 39(7), 1664–1669.Cavazza, A., Corradini, C., Rinaldi, M., Salvadeo, P., Borromei, C., & Massini, R. (2012). Evaluation of pasta thermal treatment by determination of carbohydrates, furosine, and color indices. Food and Bioprocess Technology. doi: 10.1007/s11947-012-0906-6 . In-press.Cubero, S., Aleixos, N., Moltó, E., Gómez-Sanchis, J., & Blasco, J. (2011). Advances in machine vision applications for automatic inspection and quality evaluation of fruits and vegetables. Food and Bioprocess Technology, 4(4), 487–504.Cubero, S., Moltó, E., Gutiérrez, A., Aleixos, N., García-Navarrete, O. L., Juste, F., & Blasco, J. (2010). Real-time inspection of fruit on a mobile harvesting platform in field conditions using computer vision. Progress in Agricultural Engineering Science, 6, 1–16.Díaz, R., Faus, G., Blasco, M., Blasco, J., & Moltó, E. (2000). The application of a fast algorithm for the classification of olives by machine vision. Food Research International, 33, 305–309.DOGV (2006) Diari Oficial de la Comunitat Valenciana, 5346, 30321-30328.Gardner, J. L. (2007). Comparison of calibration methods for tristimulus colorimeters. Journal of Research of the National Institute of Standards and Technology, 112, 129–138.Hashim, N., Janius, R. B., Baranyai, L., Rahman, R. A., Osman, A., & Zude, M. (2011). Kinetic model for colour changes in bananas during the appearance of chilling injury symptoms. Food and Bioprocess Technology, 5(8), 2952–2963.HunterLab (2008): Applications note, 8(9), http://www.hunterlab.com/appnotes/an08_96a.pdf . Accessed September 2012.Hutchings, J. B., Luo, R., & Ji, W. (2002). Calibrated colour imaging analysis of food. In D. MacDougall (Ed.), Colour in Food (pp. 352–366). Cambridge: Woodhead Publishing.Jiménez-Cuesta MJ, Cuquerella J & Martínez-Jávega JM (1981) Determination of a color index for citrus fruit degreening. In Proc. of the International Society of Citriculture, Vol. 2, 750-753Kang, S. P., East, A. R., & Trujillo, F. J. (2008). Colour vision system evaluation of bicolour fruit: A case study with ‘B74’ mango. Postharvest Biology and Technology, 49, 77–85.Lang, C., & Hübert, T. (2011). A colour ripeness indicator for apples. Food and Bioprocess Technology, 5(8), 3244–3249.López-Camelo, A. F., & Gómez, P. A. (2004). Comparison of color indexes for tomato ripening. Horticultura Brasileira, 22(3), 534–537. http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0102-05362004000300006 .López-García, F., Andreu-García, A., Blasco, J., Aleixos, N., & Valiente, J. M. (2010). Automatic detection of skin defects in citrus fruits using a multivariate image analysis approach. Computers and Electronics in Agriculture, 71, 189–197.Lorente, D., Aleixos, N., Gómez-Sanchis, J., Cubero, S., García-Navarrete, O. L., & Blasco, J. (2012). Recent advances and applications of hyperspectral imaging for fruit and vegetable quality assessment. Food and Bioprocess Technology, 5(4), 1121–1142.Mendoza, F., Dejmek, P., & Aguilera, J. M. (2006). Calibrated color measurements of agricultural foods using image analysis. Postharvest Biology and Technology, 41, 285–295.Montgomery, D. C. (2005). Design and analysis of experiments, 6th ed. Tempe: Wiley.Noboru, O., & Robertson, A. R. (2005). Colorimetry. West Sussex: Wiley.Pathare, P. B., Opara, U. L., & Al-Said, F. A. (2012). Colour measurement and analysis in fresh and processed foods: a review. Food and Bioprocess Technology. doi: 10.1007/s11947-012-0867-9 . In-press.Quevedo, R. A., Aguilera, J. M., & Pedreschi, F. (2010). Colour of salmon fillets by computer vision and sensory panel. Food and Bioprocess Technology, 3, 637–643.Sahin, S., & Sumnu, S. G. (2006). Physical properties of foods. New York: Springer.Quevedo, R., Valencia, E., Alvarado, F., Ronceros, B., & Bastias, J. M. (2011). Comparison of whiteness index vs. fractal Fourier in the determination of bloom chocolate using image analysis. Food and Bioprocess Technology. doi: 10.1007/s11947-011-0729-x . In-press.Smith, T., & Guild, J. (1931). The C.I.E. colorimetric standards and their use. Transactions of the Optical Society, 33(3), 73–134.Yam, K. L., & Papadakis, S. E. (2004). A simple digital imaging method for measuring and analyzing color of food surfaces. Journal of Food Engineering, 61, 137–142

A comprehensive review of fruit and vegetable classification techniques

Recent advancements in computer vision have enabled wide-ranging applications in every field of life. One such application area is fresh produce classification, but the classification of fruit and vegetable has proven to be a complex problem and needs to be further developed. Fruit and vegetable classification presents significant challenges due to interclass similarities and irregular intraclass characteristics. Selection of appropriate data acquisition sensors and feature representation approach is also crucial due to the huge diversity of the field. Fruit and vegetable classification methods have been developed for quality assessment and robotic harvesting but the current state-of-the-art has been developed for limited classes and small datasets. The problem is of a multi-dimensional nature and offers significantly hyperdimensional features, which is one of the major challenges with current machine learning approaches. Substantial research has been conducted for the design and analysis of classifiers for hyperdimensional features which require significant computational power to optimise with such features. In recent years numerous machine learning techniques for example, Support Vector Machine (SVM), K-Nearest Neighbour (KNN), Decision Trees, Artificial Neural Networks (ANN) and Convolutional Neural Networks (CNN) have been exploited with many different feature description methods for fruit and vegetable classification in many real-life applications. This paper presents a critical comparison of different state-of-the-art computer vision methods proposed by researchers for classifying fruit and vegetable