Método para el registro automático de imágenes basado en transformaciones proyectivas planas dependientes de las distancias y orientado a imágenes sin características comunes

Tesis inédita de la Universidad Complutense de Madrid, Facultad de Ciencias Físicas, Departamento de Arquitectura de Computadores y Automática, leída el 18-12-2015Multisensory data fusion oriented to image-based application improves the accuracy, quality and availability of the data, and consequently, the performance of robotic systems, by means of combining the information of a scene acquired from multiple and different sources into a unified representation of the 3D world scene, which is more enlightening and enriching for the subsequent image processing, improving either the reliability by using the redundant information, or the capability by taking advantage of complementary information. Image registration is one of the most relevant steps in image fusion techniques. This procedure aims the geometrical alignment of two or more images. Normally, this process relies on feature-matching techniques, which is a drawback for combining sensors that are not able to deliver common features. For instance, in the combination of ToF and RGB cameras, the robust feature-matching is not reliable. Typically, the fusion of these two sensors has been addressed from the computation of the cameras calibration parameters for coordinate transformation between them. As a result, a low resolution colour depth map is provided. For improving the resolution of these maps and reducing the loss of colour information, extrapolation techniques are adopted. A crucial issue for computing high quality and accurate dense maps is the presence of noise in the depth measurement from the ToF camera, which is normally reduced by means of sensor calibration and filtering techniques. However, the filtering methods, implemented for the data extrapolation and denoising, usually over-smooth the data, reducing consequently the accuracy of the registration procedure...La fusión multisensorial orientada a aplicaciones de procesamiento de imágenes, conocida como fusión de imágenes, es una técnica que permite mejorar la exactitud, la calidad y la disponibilidad de datos de un entorno tridimensional, que a su vez permite mejorar el rendimiento y la operatividad de sistemas robóticos. Dicha fusión, se consigue mediante la combinación de la información adquirida por múltiples y diversas fuentes de captura de datos, la cual se agrupa del tal forma que se obtiene una mejor representación del entorno 3D, que es mucho más ilustrativa y enriquecedora para la implementación de métodos de procesamiento de imágenes. Con ello se consigue una mejora en la fiabilidad y capacidad del sistema, empleando la información redundante que ha sido adquirida por múltiples sensores. El registro de imágenes es uno de los procedimientos más importantes que componen la fusión de imágenes. El objetivo principal del registro de imágenes es la consecución de la alineación geométrica entre dos o más imágenes. Normalmente, este proceso depende de técnicas de búsqueda de patrones comunes entre imágenes, lo cual puede ser un inconveniente cuando se combinan sensores que no proporcionan datos con características similares. Un ejemplo de ello, es la fusión de cámaras de color de alta resolución (RGB) con cámaras de Tiempo de Vuelo de baja resolución (Time-of-Flight (ToF)), con las cuales no es posible conseguir una detección robusta de patrones comunes entre las imágenes capturadas por ambos sensores. Por lo general, la fusión entre estas cámaras se realiza mediante el cálculo de los parámetros de calibración de las mismas, que permiten realizar la trasformación homogénea entre ellas. Y como resultado de este xii Abstract procedimiento, se obtienen mapas de profundad y de color de baja resolución. Con el objetivo de mejorar la resolución de estos mapas y de evitar la pérdida de información de color, se utilizan diversas técnicas de extrapolación de datos. Un factor crucial a tomar en cuenta para la obtención de mapas de alta calidad y alta exactitud, es la presencia de ruido en las medidas de profundidad obtenidas por las cámaras ToF. Este problema, normalmente se reduce mediante la calibración de estos sensores y con técnicas de filtrado de datos. Sin embargo, las técnicas de filtrado utilizadas, tanto para la interpolación de datos, como para la reducción del ruido, suelen producir el sobre-alisamiento de los datos originales, lo cual reduce la exactitud del registro de imágenes...Sección Deptal. de Arquitectura de Computadores y Automática (Físicas)Fac. de Ciencias FísicasTRUEunpu

Application of Machine Vision Techniques in Low-Cost Devices to Improve Efficiency in Precision Farming

[EN] In the context of recent technological advancements driven by distributed work and open-source resources, computer vision stands out as an innovative force, transforming how machines interact with and comprehend the visual world around us. This work conceives, designs, implements, and operates a computer vision and artificial intelligence method for object detection with integrated depth estimation. With applications ranging from autonomous fruit-harvesting systems to phenotyping tasks, the proposed Depth Object Detector (DOD) is trained and evaluated using the Microsoft Common Objects in Context dataset and the MinneApple dataset for object and fruit detection, respectively. The DOD is benchmarked against current state-of-the-art models. The results demonstrate the proposed method's efficiency for operation on embedded systems, with a favorable balance between accuracy and speed, making it well suited for real-time applications on edge devices in the context of the Internet of things.This work was partially supported with grant PID2021-123673OB-C31, TED2021-131295BC32 funded by MCIN/AEI/10.13039/501100011033 and by ERDF A way of making Europe , PROMETEO grant CIPROM/2021/077 from the Conselleria de Innovación, Universidades, Ciencia y Sociedad Digital Generalitat Valenciana and Early Research Project grant PAID-06-23 by the Vice Rectorate Office for Research from Universitat Politècnica de València (UPV).Jaramillo-Hernández, JF.; Julian, V.; Marco-Detchart, C.; Rincón, JA. (2024). Application of Machine Vision Techniques in Low-Cost Devices to Improve Efficiency in Precision Farming. Sensors. 24(3). https://doi.org/10.3390/s2403093724

Fruit sizing using AI: A review of methods and challenges

Fruit size at harvest is an economically important variable for high-quality table fruit production in orchards and vineyards. In addition, knowing the number and size of the fruit on the tree is essential in the framework of precise production, harvest, and postharvest management. A prerequisite for analysis of fruit in a real-world environment is the detection and segmentation from background signal. In the last five years, deep learning convolutional neural network have become the standard method for automatic fruit detection, achieving F1-scores higher than 90 %, as well as real-time processing speeds. At the same time, different methods have been developed for, mainly, fruit size and, more rarely, fruit maturity estimation from 2D images and 3D point clouds. These sizing methods are focused on a few species like grape, apple, citrus, and mango, resulting in mean absolute error values of less than 4 mm in apple fruit. This review provides an overview of the most recent methodologies developed for in-field fruit detection/counting and sizing as well as few upcoming examples of maturity estimation. Challenges, such as sensor fusion, highly varying lighting conditions, occlusions in the canopy, shortage of public fruit datasets, and opportunities for research transfer, are discussed.This work was partly funded by the Department of Research and Universities of the Generalitat de Catalunya (grants 2017 SGR 646 and 2021 LLAV 00088) and by the Spanish Ministry of Science and Innovation / AEI/10.13039/501100011033 / FEDER (grants RTI2018-094222-B-I00 [PAgFRUIT project] and PID2021-126648OB-I00 [PAgPROTECT project]). The Secretariat of Universities and Research of the Department of Business and Knowledge of the Generalitat de Catalunya and European Social Fund (ESF) are also thanked for financing Juan Carlos Miranda’s pre-doctoral fellowship (2020 FI_B 00586). The work of Jordi Gené-Mola was supported by the Spanish Ministry of Universities through a Margarita Salas postdoctoral grant funded by the European Union - NextGenerationEU.info:eu-repo/semantics/publishedVersio

Automatic plant features recognition using stereo vision for crop monitoring

Machine vision and robotic technologies have potential to accurately monitor plant parameters which reflect plant stress and water requirements, for use in farm management decisions. However, autonomous identification of individual plant leaves on a growing plant under natural conditions is a challenging task for vision-guided agricultural robots, due to the complexity of data relating to various stage of growth and ambient environmental conditions. There are numerous machine vision studies that are concerned with describing the shape of leaves that are individually-presented to a camera. The purpose of these studies is to identify plant species, or for the autonomous detection of multiple leaves from small seedlings under greenhouse conditions. Machine vision-based detection of individual leaves and challenges presented by overlapping leaves on a developed plant canopy using depth perception properties under natural outdoor conditions is yet to be reported. Stereo vision has recently emerged for use in a variety of agricultural applications and is expected to provide an accurate method for plant segmentation and identification which can benefit from depth properties and robustness. This thesis presents a plant leaf extraction algorithm using a stereo vision sensor. This algorithm is used on multiple leaf segmentation and overlapping leaves separation using a combination of image features, specifically colour, shape and depth. The separation between the connected and the overlapping leaves relies on the measurement of the discontinuity in depth gradient for the disparity maps. Two techniques have been developed to implement this task based on global and local measurement. A geometrical plane from each segmented leaf can be extracted and used to parameterise a 3D model of the plant image and to measure the inclination angle of each individual leaf. The stem and branch segmentation and counting method was developed based on the vesselness measure and Hough transform technique. Furthermore, a method for reconstructing the segmented parts of hibiscus plants is presented and a 2.5D model is generated for the plant. Experimental tests were conducted with two different selected plants: cotton of different sizes, and hibiscus, in an outdoor environment under varying light conditions. The proposed algorithm was evaluated using 272 cotton and hibiscus plant images. The results show an observed enhancement in leaf detection when utilising depth features, where many leaves in various positions and shapes (single, touching and overlapping) were detected successfully. Depth properties were more effective in separating between occluded and overlapping leaves with a high separation rate of 84% and these can be detected automatically without adding any artificial tags on the leaf boundaries. The results exhibit an acceptable segmentation rate of 78% for individual plant leaves thereby differentiating the leaves from their complex backgrounds and from each other. The results present almost identical performance for both species under various lighting and environmental conditions. For the stem and branch detection algorithm, experimental tests were conducted on 64 colour images of both species under different environmental conditions. The results show higher stem and branch segmentation rates for hibiscus indoor images (82%) compared to hibiscus outdoor images (49.5%) and cotton images (21%). The segmentation and counting of plant features could provide accurate estimation about plant growth parameters which can be beneficial for many agricultural tasks and applications

Lidar-based Obstacle Detection and Recognition for Autonomous Agricultural Vehicles

Today, agricultural vehicles are available that can drive autonomously and follow exact route plans more precisely than human operators. Combined with advancements in precision agriculture, autonomous agricultural robots can reduce manual labor, improve workflow, and optimize yield. However, as of today, human operators are still required for monitoring the environment and acting upon potential obstacles in front of the vehicle. To eliminate this need, safety must be ensured by accurate and reliable obstacle detection and avoidance systems.In this thesis, lidar-based obstacle detection and recognition in agricultural environments has been investigated. A rotating multi-beam lidar generating 3D point clouds was used for point-wise classification of agricultural scenes, while multi-modal fusion with cameras and radar was used to increase performance and robustness. Two research perception platforms were presented and used for data acquisition. The proposed methods were all evaluated on recorded datasets that represented a wide range of realistic agricultural environments and included both static and dynamic obstacles.For 3D point cloud classification, two methods were proposed for handling density variations during feature extraction. One method outperformed a frequently used generic 3D feature descriptor, whereas the other method showed promising preliminary results using deep learning on 2D range images. For multi-modal fusion, four methods were proposed for combining lidar with color camera, thermal camera, and radar. Gradual improvements in classification accuracy were seen, as spatial, temporal, and multi-modal relationships were introduced in the models. Finally, occupancy grid mapping was used to fuse and map detections globally, and runtime obstacle detection was applied on mapped detections along the vehicle path, thus simulating an actual traversal.The proposed methods serve as a first step towards full autonomy for agricultural vehicles. The study has thus shown that recent advancements in autonomous driving can be transferred to the agricultural domain, when accurate distinctions are made between obstacles and processable vegetation. Future research in the domain has further been facilitated with the release of the multi-modal obstacle dataset, FieldSAFE

Neuromorphic perception for greenhouse technology using event-based sensors

Event-Based Cameras (EBCs), unlike conventional cameras, feature independent pixels that asynchronously generate outputs upon detecting changes in their field of view. Short calculations are performed on each event to mimic the brain. The output is a sparse sequence of events with high temporal precision. Conventional computer vision algorithms do not leverage these properties. Thus a new paradigm has been devised. While event cameras are very efficient in representing sparse sequences of events with high temporal precision, many approaches are challenged in applications where a large amount of spatially-temporally rich information must be processed in real-time. In reality, most tasks in everyday life take place in complex and uncontrollable environments, which require sophisticated models and intelligent reasoning. Typical hard problems in real-world scenes are detecting various non-uniform objects or navigation in an unknown and complex environment. In addition, colour perception is an essential fundamental property in distinguishing objects in natural scenes. Colour is a new aspect of event-based sensors, which work fundamentally differently from standard cameras, measuring per-pixel brightness changes per colour filter asynchronously rather than measuring “absolute” brightness at a constant rate. This thesis explores neuromorphic event-based processing methods for high-noise and cluttered environments with imbalanced classes. A fully event-driven processing pipeline was developed for agricultural applications to perform fruits detection and classification to unlock the outstanding properties of event cameras. The nature of features in such data was explored, and methods to represent and detect features were demonstrated. A framework for detecting and classifying features was developed and evaluated on the N-MNIST and Dynamic Vision Sensor (DVS) gesture datasets. The same network was evaluated on laboratory recorded and real-world data with various internal variations for fruits detection such as overlap, variation in size and appearance. In addition, a method to handle highly imbalanced data was developed. We examined the characteristics of spatio-temporal patterns for each colour filter to help expand our understanding of this novel data and explored their applications in classification tasks where colours were more relevant features than shapes and appearances. The results presented in this thesis demonstrate the potential and efficacy of event- based systems by demonstrating the applicability of colour event data and the viability of event-driven classification

Machine Learning Algorithms for Robotic Navigation and Perception and Embedded Implementation Techniques

L'abstract è presente nell'allegato / the abstract is in the attachmen

Computer vision based classification of fruits and vegetables for self-checkout at supermarkets

The field of machine learning, and, in particular, methods to improve the capability of machines to perform a wider variety of generalised tasks are among the most rapidly growing research areas in today’s world. The current applications of machine learning and artificial intelligence can be divided into many significant fields namely computer vision, data sciences, real time analytics and Natural Language Processing (NLP). All these applications are being used to help computer based systems to operate more usefully in everyday contexts. Computer vision research is currently active in a wide range of areas such as the development of autonomous vehicles, object recognition, Content Based Image Retrieval (CBIR), image segmentation and terrestrial analysis from space (i.e. crop estimation). Despite significant prior research, the area of object recognition still has many topics to be explored. This PhD thesis focuses on using advanced machine learning approaches to enable the automated recognition of fresh produce (i.e. fruits and vegetables) at supermarket self-checkouts. This type of complex classification task is one of the most recently emerging applications of advanced computer vision approaches and is a productive research topic in this field due to the limited means of representing the features and machine learning techniques for classification. Fruits and vegetables offer significant inter and intra class variance in weight, shape, size, colour and texture which makes the classification challenging. The applications of effective fruit and vegetable classification have significant importance in daily life e.g. crop estimation, fruit classification, robotic harvesting, fruit quality assessment, etc. One potential application for this fruit and vegetable classification capability is for supermarket self-checkouts. Increasingly, supermarkets are introducing self-checkouts in stores to make the checkout process easier and faster. However, there are a number of challenges with this as all goods cannot readily be sold with packaging and barcodes, for instance loose fresh items (e.g. fruits and vegetables). Adding barcodes to these types of items individually is impractical and pre-packaging limits the freedom of choice when selecting fruits and vegetables and creates additional waste, hence reducing customer satisfaction. The current situation, which relies on customers correctly identifying produce themselves leaves open the potential for incorrect billing either due to inadvertent error, or due to intentional fraudulent misclassification resulting in financial losses for the store. To address this identified problem, the main goals of this PhD work are: (a) exploring the types of visual and non-visual sensors that could be incorporated into a self-checkout system for classification of fruits and vegetables, (b) determining a suitable feature representation method for fresh produce items available at supermarkets, (c) identifying optimal machine learning techniques for classification within this context and (d) evaluating our work relative to the state-of-the-art object classification results presented in the literature. An in-depth analysis of related computer vision literature and techniques is performed to identify and implement the possible solutions. A progressive process distribution approach is used for this project where the task of computer vision based fruit and vegetables classification is divided into pre-processing and classification techniques. Different classification techniques have been implemented and evaluated as possible solution for this problem. Both visual and non-visual features of fruit and vegetables are exploited to perform the classification. Novel classification techniques have been carefully developed to deal with the complex and highly variant physical features of fruit and vegetables while taking advantages of both visual and non-visual features. The capability of classification techniques is tested in individual and ensemble manner to achieved the higher effectiveness. Significant results have been obtained where it can be concluded that the fruit and vegetables classification is complex task with many challenges involved. It is also observed that a larger dataset can better comprehend the complex variant features of fruit and vegetables. Complex multidimensional features can be extracted from the larger datasets to generalise on higher number of classes. However, development of a larger multiclass dataset is an expensive and time consuming process. The effectiveness of classification techniques can be significantly improved by subtracting the background occlusions and complexities. It is also worth mentioning that ensemble of simple and less complicated classification techniques can achieve effective results even if applied to less number of features for smaller number of classes. The combination of visual and nonvisual features can reduce the struggle of a classification technique to deal with higher number of classes with similar physical features. Classification of fruit and vegetables with similar physical features (i.e. colour and texture) needs careful estimation and hyper-dimensional embedding of visual features. Implementing rigorous classification penalties as loss function can achieve this goal at the cost of time and computational requirements. There is a significant need to develop larger datasets for different fruit and vegetables related computer vision applications. Considering more sophisticated loss function penalties and discriminative hyper-dimensional features embedding techniques can significantly improve the effectiveness of the classification techniques for the fruit and vegetables applications

Innovations in non-destructive techniques for fruit quality control applied to manipulation and inspection lines

Tesis por compendioLa industria alimentaria, concretamente el sector poscosecha, necesita innovar en sus procesos productivos, optimizando los mismos para rentabilizar sus actividades, garantizando productos de calidad capaces de satisfacer las necesidades de los consumidores. La presente tesis doctoral se centra en evaluar el potencial de la espectroscopia VIS-NIR para la caracterización e inspección de la calidad de la fruta tanto fuera de línea como a tiempo real en procesos automatizados. En un primer lugar, la viabilidad de la técnica se estudió a nivel de laboratorio en estado estático (off-line), con el fin de conocer y optimizar las condiciones de medición. Posteriormente, se evaluó la calidad interna y externa de diferentes tipos de frutas como son caqui, nectarina y mango. En una segunda etapa, se llevó a cabo una automatización de los procesos de inspección mediante el desarrollo de nuevos prototipos in-line. Para este propósito, y con el objetivo de completar y corroborar los resultados obtenidos de manera estática, se estudió la integración de dos sondas VIS-NIR en una garra robótica capaz de manipular mangos. Finalmente, se estudió la integración de una sonda VIS-NIR a una cinta transportadora. Los resultados obtenidos a nivel estático han demostrado que la espectroscopia VIS-NIR es un método no destructivo muy prometedor para predecir la astringencia en caqui. Así mismo, ha demostrado ser una adecuada herramienta para clasificar al 100% entre variedades de nectarinas como "Big Top" y "Diamond Ray" con una apariencia externa e interna muy similar, pero con diferentes propiedades organolépticas. De manera similar, fue posible clasificar al 100% variedades como "Big Top" y "Magique" de apariencia externa y composición similar pero distinto color de pulpa., y además se desarrolló un índice de calidad interna (IQI) para evaluar la calidad de las nectarinas. Por lo que respecta a los trabajos off-line realizados con mangos de la variedad "Osteen", fue posible predecir su calidad interna mediante los índices de madurez (RPI) y de calidad (IQI) con un gran rendimiento. A su vez, los ensayos experimentales efectuados con estos mismos mangos bajo la manipulación no destructiva de una garra robótica, demostraron que los mejores modelos eran capaces de predecir tanto la firmeza mecánica, el contenido en sólidos solubles, la luminosidad de la pulpa, así como el índice RPI de las muestras en base a la información obtenida por los acelerómetros instalados en los dedos de la garra robótica. En cuanto a los ensayos realizados de manera in-line, el primer prototipo desarrollado se basó en la integración de dos sondas VIS-NIR en una garra robótica dispuesta con dos acelerómetros. El sistema desarrollado permitió alcanzar una buena estimación de la calidad del mango a través del índice RPI fusionando la información tanto de los espectros VIS-NIR como del impacto no destructivo de los acelerómetros. De este modo quedó demostrado que era posible obtener una predicción similar trabajando de forma in-line como trabajando de manera off-line para la predicción del mismo índice de calidad en mangos. El segundo prototipo in-line desarrollado se basa en la integración de una sonda VIS-NIR en una cinta transportadora para la identificación de distintas variedades y orígenes de manzanas. El prototipo desarrollado permitió registrar resultados de clasificación tan buenos como los efectuados de manera off-line con, por ejemplo, nectarina. De este modo, se puede concluir que la espectroscopia VIS-NIR permite monitorear la calidad y clasificar fruta poscosecha tanto en modo off-line como in-line. Los nuevos prototipos desarrollados aportan claras ventajas respecto a los procesos tradicionales realizados a mano, como son la reducción del tiempo de inspección, la disminución de la cantidad de residuos generados y la posibilidad de inspeccionar toda la producción, obteniendo así un análisis más estandarizThe food industry, concretely the post-harvest sector, needs to innovate in their production processes, optimizing them to make their activities profitable, guaranteeing quality products capable of satisfying the needs of consumers. The present doctoral thesis focuses on evaluating the potential of visible and near infrared spectroscopy (VIS-NIR) for the characterization and inspection of fruit quality both off-line and in real time in automated processes. Firstly, the viability of the technique was studied at the laboratory level in a static mode (off-line), in order to know and optimise the measurement conditions. Subsequently, the internal and external quality of different types of fruits such as persimmon, nectarine and mango were evaluated. Secondly, an automation of the inspection processes was carried out through the development of new in-line prototypes. For this purpose, and with the aim of completing and corroborating the results obtained in a static mode, the integration of two VIS-NIR probes in a robotic gripper capable of manipulating mangoes was studied. Finally, the integration of a VIS-NIR probe to a conveyor belt was studied as an in-line monitoring tool on the inspection process of different apple varieties. The results obtained in static mode have shown that VIS-NIR spectroscopy is a very promising non-destructive method to predict the astringency in persimmon. Likewise, it has demonstrated to be an adequate tool to classify 100% between nectarine varieties such as 'Big Top' and 'Diamond Ray' with very similar external and internal appearance, but with different organoleptic properties. Similarly, it was possible to classify 100% varieties such as 'Big Top' and 'Magique' with external appearance and similar composition but different pulp colour. An internal quality index (IQI) was developed to evaluate the quality of nectarines, which can be predicted through VIS-NIR spectroscopy. Regarding the off-line work carried out with mangoes of 'Osteen' variety, it was possible to predict its internal quality through the indexes of maturity (RPI) and quality (IQI) with a high performance. Moreover, the experimental tests carried out with these same mangoes under the non-destructive manipulation of a robotic gripper, showed that the best models were able to predict both the mechanical firmness, the soluble solids content, the brightness of the pulp, as well as the RPI index of the samples based on the information obtained by the accelerometers installed on the fingers of the robotic gripper. Regarding the tests carried out in an in-line mode, the first developed prototype was based on the integration of two VIS-NIR probes in a robotic gripper fitted with two accelerometers. The developed system allowed reaching a good estimation of mango quality through the RPI index. In this way, it was demonstrated that it was possible to obtain a similar prediction working in-line as off-line mode for the prediction of the same quality index in mangoes. The second developed in-line prototype is based on the integration of a VIS-NIR probe in a conveyor belt for the identification of different varieties and origins of apples, achieving a success rate of 98% with the system. The developed prototype allowed to register classification results as good as those carried out off-line with, for example, nectarine. In this way, it can be concluded that VIS-NIR spectroscopy allows monitoring the quality and classifying post-harvest fruit in both off-line and in-line mode, being a tool that allows improving and guaranteeing the correct quality and food safety. The new developed prototypes provide clear advantages over the traditional processes performed by hand, such as the reduction of inspection time, the reduction of the amount of waste generated by destructive quality analysis and the possibility of inspecting full production, obtaining a more standardised analysis of the quality of the products.La indústria alimentària, concretament el sector postcollita, necessita innovar en els seus processos productius, optimitzant els mateixos per a rendibilitzar les seues activitats, garantint productes de qualitat capaços de satisfer les necessitats dels consumidors. La present tesi doctoral es centra en avaluar el potencial de l'espectroscòpia visible i infraroig pròxim (VIS-NIR) per a la caracterització i la inspecció de la qualitat de la fruita tant fora de línia com a temps real en processos automatitzats. En un primer lloc, la viabilitat de la tècnica es va estudiar a nivell de laboratori en estat estàtic (off-line), a fi de conéixer i optimitzar les condicions de mesurament. Posteriorment, es va avaluar la qualitat interna i externa de diferents tipus de fruites com són caqui, nectarina i mango. En una segona etapa, es va dur a terme una automatització dels processos d'inspecció per mitjà del desenvolupament de nous prototips in-line. Per aquest propòsit, i amb l'objectiu de completar i corroborar els resultats obtinguts de manera estàtica, es va estudiar la integració de dos sondes VIS-NIR en una garra robòtica capaç de manipular. Finalment, es va estudiar la integració d'una sonda VIS-NIR a una cinta transportadora. Els resultats obtinguts a nivell estàtic han demostrat que l'espectroscòpia VIS-NIR és un mètode no destructiu molt prometedor per a predir l'astringència en caqui. Així mateix, ha demostrat ser una adequada ferramenta per a classificar al 100% entre varietats de nectarines com "Big Top" i "Diamond Ray" amb una aparença externa i interna molt semblant, però amb diferents propietats organolèptiques. De manera semblant, va ser possible classificar al 100% varietats com "Big Top" i "Magique" d'aparença externa i composició semblant però distint color de polpa. Es va desenvolupar un índex de qualitat interna (IQI) per avaluar la qualitat de les nectarines. Pel que fa als treballs off-line realitzats amb mangos de la varietat "Osteen" va ser possible predir la seua qualitat interna mitjançant els índexs de maduresa (RPI) i de qualitat (IQI) amb un gran rendiment. Al mateix temps, els assajos experimentals efectuats amb estos mateixos mangos baix la manipulació no destructiva d'una garra robòtica, van demostrar que els millors models eren capaços de predir tant la fermesa mecánica, el contingut en sòlids solubles, la lluminositat de la polpa, així com l'índex RPI de les mostres basant-se en l'informació obtinguda pels acceleròmetres instal¿lats en els dits de la garra robòtica. En quant als assajos realitzats de manera in-line, el primer prototip desenvolupat es va basar en la integració de dos sondes VIS-NIR en una garra robòtica disposada amb dos acceleròmetres. El sistema desenvolupat va permetre aconseguir una bona estimació de la qualitat del mango a través de l'índex RPI fusionant l'informació tant dels espectres VIS-NIR com de l'impacte no destructiu dels acceleròmetres. D'esta manera va quedar demostrat que era possible obtindre una predicció semblant treballant de forma in-line com off-line per a la predicció del mateix índex de qualitat en mangos. El segon prototip in-line desenvolupat es va basar en la integració d'una sonda VIS-NIR en una cinta transportadora per a l'identificació de distintes varietats i orígens de pomes. El prototip desenvolupat va permetre registrar resultats de classificació tan bons com els efectuats de manera off-line. D'aquesta manera, es pot concloure que l'espectroscòpia VIS-NIR permet monitorar la qualitat i classificar fruita postcollita tant en mode off-line com in-line. Els nous prototips desenvolupats aporten clars avantatges respecte als processos tradicionals realitzats a mà, com són la reducció del temps d'inspecció, la disminució de la quantitat de residus generats pels anàlisis destructives de qualitat i la possibilitat d'inspeccionar tota la producció, obtenint així un anàlisi més estandarditzCortés López, V. (2018). Innovations in non-destructive techniques for fruit quality control applied to manipulation and inspection lines [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/110969TESISCompendi

Application of hyperspectral imaging combined with chemometrics for the non-destructive evaluation of the quality of fruit in postharvest

Tesis por compendio[ES] El objetivo de esta tesis doctoral es evaluar la técnica de imagen hiperespectral en el rango visible e infrarrojo cercano, en combinación con técnicas quimiométricas para la evaluación de la calidad de la fruta en poscosecha de manera eficaz y sostenible. Con este fin, se presentan diferentes estudios en los que se evalúa la calidad de algunas frutas que por su valor económico, estratégico o social, son de especial importancia en la Comunidad Valenciana como son el caqui 'Rojo Brillante', la granada 'Mollar de Elche', el níspero 'Algerie' o diferentes cultivares de nectarina. En primer lugar se llevó a cabo la monitorización de la calidad poscosecha de nectarinas 'Big Top' y 'Magique' usando imagen hiperespectral en reflectancia y transmitancia. Al mismo tiempo se evaluó la transmitancia para la detección de huesos abiertos. Se llevó a cabo también un estudio para distinguir los cultivares 'Big Top' y "Diamond Ray", los cuales poseen un aspecto muy similar pero sabor diferente. En cuanto al caqui 'Rojo Brillante', la imagen hiperespectral fue estudiada por una parte para monitorear su madurez, y por otra parte para evaluar la astringencia de esta fruta, que debe ser completamente eliminada antes de su comercialización. Las propiedades físico-químicas de la granada 'Mollar de Elche' fueron evaluadas usando imagen de color e hiperespectral durante su madurez usando la información de la fruta intacta y de los arilos. Finalmente, esta técnica se usó para caracterizar e identificar los defectos internos y externos del níspero 'Algerie'. En la predicción de los índices de calidad IQI y RPI usando imagen en reflectancia y transmitancia se obtuvieron valores de R2 alrededor de 0,90 y en la discriminación por firmeza, una precisión entorno al 95 % usando longitudes de onda seleccionadas. En cuanto a la detección de huesos abiertos, el uso de la imagen hiperespectral en transmitancia obtuvo un 93,5 % de clasificación correcta de frutas con hueso normal y 100 % con hueso abierto usando modelos PLS-DA y 7 longitudes de onda. Los resultados obtenidos en la clasificación de los cultivares 'Big Top' y 'Diamond Ray' mostraron una fiabilidad superior al 96,0 % mediante el uso de modelos PLS-DA y 14 longitudes de onda seleccionadas, superando a la imagen de color (56,9 %) y a un panel entrenado (54,5 %). Con respecto al caqui, los resultados obtenidos indicaron que es posible distinguir entre tres estados de madurez con una precisión del 96,0 % usando modelos QDA y se predijo su firmeza obteniendo un valor de R2 de 0,80 usando PLS-R. En cuanto a la astringencia, se llevaron a cabo dos estudios similares en los que en el primero se discriminó la fruta de acuerdo al tiempo de tratamiento con altas concentraciones de CO2 con una precisión entorno al 95,0 % usando QDA. En el segundo se discriminó la fruta de acuerdo a un valor de contenido en taninos (0,04 %) y se determinó qué área de la fruta era mejor para realizar esta discriminación. Así se obtuvo una precisión del 86,9 % usando la zona media y 23 longitudes de onda. Los resultados obtenidos para la granada indicaron que la imagen de color e hiperespectral poseen una precisión similar en la predicción de las propiedades fisicoquímicas usando PLS-R y la información de la fruta intacta. Sin embargo, cuando se usó la información de los arilos, la imagen hiperespectral fue más precisa. En cuanto a la discriminación del estado de madurez usando PLS-DA, la imagen hiperespectral ofreció mayor precisión, 95,0 %, usando la información de la fruta intacta y del 100 % usando la de los arilos. Finalmente, los resultados obtenidos para el níspero indicaron que la imagen hiperespectral junto con el método de clasificación XGBOOST pudo discriminar entre muestras con y sin defectos con una precisión del 97,5 % y entre muestras sin defectos o con defectos internos o externos con una precisión del 96,7 %. Además fue posible distinguir entre los dife[CA] L'objectiu de la present tesi doctoral se centra en avaluar la capacitat de la imatge hiperespectral en el rang visible i infraroig pròxim, en combinació amb mètodes quimiomètrics, per a l'avaluació de la qualitat de la fruita en post collita de manera eficaç i sostenible. A aquest efecte, es presenten diferents estudis en els quals s'avalua la qualitat d'algunes fruites que pel seu valor econòmic, estratègic o social, són d'especial importància a la Comunitat Valenciana com són el caqui 'Rojo Brillante', la magrana 'Mollar de Elche', el nispro 'Algerie' o diferents cultivares de nectarina. En primer lloc es va dur a terme la monitorització de la qualitat post collita de nectarines 'Big Top' i 'Magique' per mitjà d'imatge hiperespectral en reflectància i trasnmitancia. Així mateix es va avaluar la transmitància per a la detecció d'ossos oberts. Es va dur a terme també un estudi per distingir els cultivares 'Big Top' i 'Diamond Ray', els quals posseeixen un aspecte molt semblant però sabor diferent. Pel que fa al caqui 'Rojo Brillante', la imatge hiperespectral va ser estudiada d'una banda per a monitoritzar la seua maduresa, i per un altre costat per avaluar l'astringència, que ha de ser completament eliminada abans de la seua comercialització. Les propietats fisicoquímiques de la magrana 'Mollar de Elche' van ser avaluades per la imatge de color i hiperespectral durant la seua maduresa usant la informació de la fruita intacta i els arils. Finalment, aquesta tècnica es va fer servir per caracteritzar i identificar els defectes interns i externs del nispro 'Algerie'. En la predicció dels índexs de qualitat IQI i RPI usant imatge en reflectància com en trasnmitancia es van obtindre valors de R2 al voltant de 0,90 i en la discriminació per fermesa una precisió entorn del 95,0 % utilitzant longituds d'ona seleccionades. Pel que fa a la detecció d'ossos oberts, l'ús de la imatge hiperespectral en transmitància va obtindre un 93,5 % classificació correcta de fruites amb os normal i 100 % amb os obert usant models PLS-DA i 7 longituds d'ona. Els resultats obtinguts en la classificació dels cultivares 'Big Top' i 'Diamond Ray' van mostrar una fiabilitat superior al 96,0 % per mitjà de l'ús de models PLS-DA i 14 longituds d'ona, superant a la imatge de color (56,9 %) i a un panell sensorial entrenat (54,5 %). Quant al caqui, els resultats obtinguts van indicar que és possible distingir entre tres estats de maduresa amb una precisió del 96,0 % usant models QDA i es va predir la seua fermesa obtenint un valor de R2 de 0,80 usant PLS-R. Pel que fa a l'astringència, es van dur a terme dos estudis similars en què el primer es va discriminar la fruita d'acord al temps de tractament amb altes concentracions de CO2 amb una precisió al voltant del 95,0 % usant QDA. En el segon, es va discriminar la fruita d'acord a un valor de contingut en tanins (0,04 %) i es va determinar quina part de la fruita era millor per a realitzar aquesta discriminació. Així es va obtindre una precisió del 86,9 % usant la zona mitjana i 23 longituds d'ona. Els resultats obtinguts per la magrana van indicar que la imatge de color i hiperespectral posseïxen una precisió semblant a la predicció de les propietats fisicoquímiques usant PLS-R i la informació de la fruita intacta. No obstant això, quan es va usar la informació dels arils, la imatge hiperespectral va ser més precisa. Quant a la discriminació de l'estat de maduresa usant PLS-DA, la imatge hiperespectral va oferir major precisió (95,0 %) usant la informació de la fruita intacta i del 100 % usant la dels arils. Finalment, els resultats obtinguts pel nispro indiquen que la imatge hiperespectral juntament amb el mètode de classificació XGBOOST va poder discriminar entre mostres amb i sense defectes amb una precisió del 97,5 % i entre mostres sense defectes o amb defectes interns o externs amb una precisió del 96,7 %. A més, va ser possible distingir entre[EN] The objective of this doctoral thesis is to evaluate the potential of the hyperspectral imaging in the visible and near infrared range in combination with chemometrics for the assessment of the postharvest quality of fruit in a non-destructive, efficient and sustainable manner. To this end, different studies are presented in which the quality of some fruits is evaluated. Due to their economic, strategic or social value, the selected fruits are of special importance in the Valencian Community, such as Persimmon 'Rojo Brillante', the pomegranate 'Mollar de Elche', the loquat 'Algerie' or different nectarine cultivars. First, the quality monitoring of 'Big Top' and 'Magique' nectarines was carried out using reflectance and transmittance images. At the same time, transmittance was evaluated for the detection of split pit. In addition, a classification was performed to distinguish the 'Big Top' and 'Diamond Ray' cultivars, which look very similar but have different flavour. Whereas that for the 'Rojo Brillante' persimmon, the hyperspectral imaging was studied on the one hand to monitor its maturity, and on the other hand to evaluate the astringency of this fruit, which must be completely eliminated before its commercialization. The physicochemical properties of the 'Mollar de Elche' pomegranate were evaluated by means of hyperspectral and colour imaging during its maturity using the information from the intact fruit and arils. Finally, this technique was used to characterise and identify the internal and external defects of the 'Algerie' loquat. In the prediction of the IQI and RPI quality indexes using reflectance and transmittance images, R2 values around 0.90 were obtained and in the discrimination according to firmness, accuracy around 95.0 % using selected wavelengths was obtained. Regarding the split pit detection, the use of the hyperspectral image in transmittance mode obtained a 93.5 % of fruits with normal bone correctly classified and 100% with split pit using PLS-DA models and 7 wavelengths. The results obtained in the classification of 'Big Top' and 'Diamond Ray' fruits show accuracy higher than 96.0 % by using PLS-DA models and 14 selected wavelengths, higher than the obtained with colour image (56.9 %) and a trained panel (54.5 %). According to persimmon, the results obtained indicated that it is possible to distinguish between three states of maturity with an accuracy of 96.0 % using QDA models and its firmness was predicted obtaining a R2 value of 0.80 using PLS-R. Regarding astringency, two similar studies were carried out. In the first study, the fruit was classified according to the time of treatment with high concentrations of CO2 with a precision of around 95.0 % using QDA. In the second, the fruit was discriminated according to a threshold value of soluble tannins (0.04 %) and was determined what fruit area was better to perform this discrimination. Thus, an accuracy of 86.9 % was obtained using the middle area and 23 wavelengths. The results obtained for the pomegranate indicated that the use of colour and hyperspectral images have a similar precision in the prediction of physicochemical properties using PLS-R and the intact fruit information. However, when the information from the arils was used, the hyperspectral image was more accurate. Regarding the discrimination by the state of maturity using PLS-DA, the hyperspectral image offered greater precision, of 95.0 % using the information from the intact fruit and 100 % using that from the arils. Finally, the results obtained for the 'Algerie' loquat indicated that the hyperspectral image with the XGBOOST classification method could discriminate between sound samples and samples with defects with accuracy of 97.5 % and between sound samples or samples with internal or external defects with an accuracy of 96.7 %. It was also possible to distinguish between the different defects with an accuracy of 95.9 %.Munera Picazo, SM. (2019). Application of hyperspectral imaging combined with chemometrics for the non-destructive evaluation of the quality of fruit in postharvest [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/125954TESISCompendi