Automatic detection of skin defects in citrus fruits using a multivariate image analysis approach

One of the main problems in the post-harvest processing of citrus is the detection of visual defects in order to classify the fruit depending on their appearance. Species and cultivars of citrus present a high rate of unpredictability in texture and colour that makes it difficult to develop a general, unsupervised method able of perform this task. In this paper we study the use of a general approach that was originally developed for the detection of defects in random colour textures. It is based on a Multivariate Image Analysis strategy and uses Principal Component Analysis to extract a reference eigenspace from a matrix built by unfolding colour and spatial data from samples of defect-free peel. Test images are also unfolded and projected onto the reference eigenspace and the result is a score matrix which is used to compute defective maps based on the T2 statistic. In addition, a multiresolution scheme is introduced in the original method to speed up the process. Unlike the techniques commonly used for the detection of defects in fruits, this is an unsupervised method that only needs a few samples to be trained. It is also a simple approach that is suitable for real-time compliance. Experimental work was performed on 120 samples of oranges and mandarins from four different cultivars: Clemenules, Marisol, Fortune, and Valencia. The success ratio for the detection of individual defects was 91.5%, while the classification ratio of damaged/sound samples was 94.2%. These results show that the studied method can be suitable for the task of citrus inspection. © 2010 Elsevier B.V. All rights reserved.This work has been supported by the Spanish Ministry of Education (MEC) and by European FEDER funds, through the research projects DPI2007-66596-C02-01 (VISTAC) and DPI-2007-66596-C02-02.López García, F.; Andreu García, G.; Blasco Ivars, J.; Aleixos Borrás, MN.; Valiente González, JM. (2010). Automatic detection of skin defects in citrus fruits using a multivariate image analysis approach. Computers and Electronics in Agriculture. 71(2):189-197. doi:10.1016/j.compag.2010.02.001S18919771

Detection of visual defects in citrus fruits: multivariate image analysis vs graph image segmentation

¿The final publication is available at Springer via http://dx.doi.org/10.1007/978-3-642-40261-6_28This paper presents an application of visual quality control in orange post-harvesting comparing two different approaches. These approaches correspond to two very different methodologies released in the area of Computer Vision. The first approach is based on Multivariate Image Analysis (MIA) and was originally developed for the detection of defects in random color textures. It uses Principal Component Analysis and the T2 statistic to map the defective areas. The second approach is based on Graph Image Segmentation (GIS). It is an efficient segmentation algorithm that uses a graph-based representation of the image and a predicate to measure the evidence of boundaries between adjacent regions. While the MIA approach performs novelty detection on defects using a trained model of sound color textures, the GIS approach is strictly an unsupervised method with no training required on sound or defective areas. Both methods are compared through experimental work performed on a ground truth of 120 samples of citrus coming from four different cultivars. Although the GIS approach is faster and achieves better results in defect detection, the MIA method provides less false detections and does not need to use the hypothesis that the bigger area in samples always correspond to the non-damaged areaLópez García, F.; Andreu García, G.; Valiente González, JM.; Atienza Vanacloig, VL. (2013). Detection of visual defects in citrus fruits: multivariate image analysis vs graph image segmentation. En Computer Analysis of Images and Patterns. Springer Verlag (Germany). 8047:237-244. doi:10.1007/978-3-642-40261-6S237244804

Pixel classification methods for identifying and quantifying leaf surface injury from digital images

Plants exposed to stress due to pollution, disease or nutrient deficiency often develop visible symptoms on leaves such as spots, colour changes and necrotic regions. Early symptom detection is important for precision agriculture, environmental monitoring using bio-indicators and quality assessment of leafy vegetables. Leaf injury is usually assessed by visual inspection, which is labour-intensive and to a consid- erable extent subjective. In this study, methods for classifying individual pixels as healthy or injured from images of clover leaves exposed to the air pollutant ozone were tested and compared. RGB images of the leaves were acquired under controlled conditions in a laboratory using a standard digital SLR camera. Different feature vectors were extracted from the images by including different colour and texture (spa- tial) information. Four approaches to classification were evaluated: (1) Fit to a Pattern Multivariate Image Analysis (FPM) combined with T2 statistics (FPM-T2) or (2) Residual Sum of Squares statistics (FPM-RSS), (3) linear discriminant analysis (LDA) and (4) K-means clustering. The predicted leaf pixel classifications were trained from and compared to manually segmented images to evaluate classification performance. The LDA classifier outperformed the three other approaches in pixel identification with significantly higher accuracy, precision, true positive rate and F-score and significantly lower false positive rate and computation time. A feature vector of single pixel colour channel intensities was sufficient for capturing the information relevant for pixel identification. Including neighbourhood pixel information in the feature vector did not improve performance, but significantly increased the computation time. The LDA classifier was robust with 95% mean accuracy, 83% mean true positive rate and 2% mean false positive rate, indicating that it has potential for real-time applications.Opstad Kruse, OM.; Prats Montalbán, JM.; Indahl, UG.; Kvaal, K.; Ferrer Riquelme, AJ.; Futsaether, CM. (2014). Pixel classification methods for identifying and quantifying leaf surface injury from digital images. Computers and Electronics in Agriculture. 108:155-165. doi:10.1016/j.compag.2014.07.010S15516510

Statistical Process Control based on Multivariate Image Analysis: A new proposal for monitoring and defect detection

The monitoring, fault detection and visualization of defects are a strategic issue for product quality. This paper presents a novel methodology based on the integration of textural Multivariate image analysis (MIA) and multivariate statistical process control (MSPC) for process monitoring. The proposed approach combines MIA and p-control charts, as well as T2 and RSS images for defect location and visualization. Simulated images of steel plates are used to illustrate the monitoring performance of it. Both approaches are also applied on real clover images.The authors want to thank Ole Mathis Kruse and Prof. Cecilia Futsaether, from the Norwegian University of Life Sciences (Dept. of Mathematic Sciences and Technology), for providing the real image data set. This research work was partially supported by the Spanish Ministry of Economy and Competitiveness under the project DPI 2011-28112-C04-02.Prats Montalbán, JM.; Ferrer Riquelme, AJ. (2014). Statistical Process Control based on Multivariate Image Analysis: A new proposal for monitoring and defect detection. Computers and Chemical Engineering. 71:501-511. https://doi.org/10.1016/j.compchemeng.2014.09.014S5015117

Segmentation techniques in image analysis: A comparative study

[EN] Nowadays, the detection, localization, and quantification of different kinds of features in an RGB image (segmentation) is extremely helpful for, e.g., process monitoring or customer product acceptance. In this article, some of the most commonly used RGB image segmentation approaches are compared in an orange quality control case study. Analysis of variance and correspondence analysis are combined for determining their most relevant differences and highlighting their pros and cons.Spanish Ministry of Economy and Competitiveness, Grant/Award Number: DPI2014-55276-C5-1R; Spanish National Institute for Agricultural and Food Research and Technology (INIA), Grant/Award Number: RTA2012-00062-C04-01; European Regional Development Fund (FEDER); Shell Global Solutions International B.V.Vitale, R.; Prats-Montalbán, JM.; López García, F.; Blasco Ivars, J.; Ferrer, A. (2016). Segmentation techniques in image analysis: A comparative study. Journal of Chemometrics. 30(12):749-758. https://doi.org/10.1002/cem.2854S7497583012Prats-Montalbán, J. M., de Juan, A., & Ferrer, A. (2011). Multivariate image analysis: A review with applications. Chemometrics and Intelligent Laboratory Systems, 107(1), 1-23. doi:10.1016/j.chemolab.2011.03.002Bevilacqua, M., Bucci, R., Magrì, A. D., Magrì, A. L., Nescatelli, R., & Marini, F. (2013). Classification and Class-Modelling. Chemometrics in Food Chemistry, 171-233. doi:10.1016/b978-0-444-59528-7.00005-3Manning, C. D., Raghavan, P., & Schutze, H. (2008). Introduction to Information Retrieval. doi:10.1017/cbo9780511809071MacQueen J Some methods for classification and analysis of multivariate observations Proceedings of the Berkeley Symposium on Mathematical Statistics and Probability Berkeley, CA University of California Press 1967 281 297Haralick, R. M. (1979). Statistical and structural approaches to texture. Proceedings of the IEEE, 67(5), 786-804. doi:10.1109/proc.1979.11328Felzenszwalb, P. F., & Huttenlocher, D. P. (2004). Efficient Graph-Based Image Segmentation. International Journal of Computer Vision, 59(2), 167-181. doi:10.1023/b:visi.0000022288.19776.77Barker, M., & Rayens, W. (2003). Partial least squares for discrimination. Journal of Chemometrics, 17(3), 166-173. doi:10.1002/cem.785Postma, G. J., Krooshof, P. W. T., & Buydens, L. M. C. (2011). Opening the kernel of kernel partial least squares and support vector machines. Analytica Chimica Acta, 705(1-2), 123-134. doi:10.1016/j.aca.2011.04.025Vitale, R., de Noord, O. E., & Ferrer, A. (2014). A kernel-based approach for fault diagnosis in batch processes. Journal of Chemometrics, 28(8), S697-S707. doi:10.1002/cem.2629Prats-Montalbán, J. M., & Ferrer, A. (2007). Integration of colour and textural information in multivariate image analysis: defect detection and classification issues. Journal of Chemometrics, 21(1-2), 10-23. doi:10.1002/cem.1026Prats-Montalbán J Control estadístico de procesos mediante análisis multivariante de imágenes Ph.D. Thesis 2005López, F., Prats, J. M., Ferrer, A., & Valiente, J. M. (2006). Defect Detection in Random Colour Textures Using the MIA T2 Defect Maps. Image Analysis and Recognition, 752-763. doi:10.1007/11867661_68Ho, P.-G. (Ed.). (2011). Image Segmentation. doi:10.5772/628Pal, N. R., & Pal, S. K. (1993). A review on image segmentation techniques. Pattern Recognition, 26(9), 1277-1294. doi:10.1016/0031-3203(93)90135-jMATLAB R2012b (8.0.0.783), Natick, USA: The Mathworks IncWold, S., Esbensen, K., & Geladi, P. (1987). Principal component analysis. Chemometrics and Intelligent Laboratory Systems, 2(1-3), 37-52. doi:10.1016/0169-7439(87)80084-9Geladi, P., & Kowalski, B. R. (1986). Partial least-squares regression: a tutorial. Analytica Chimica Acta, 185, 1-17. doi:10.1016/0003-2670(86)80028-9Cao, D.-S., Liang, Y.-Z., Xu, Q.-S., Hu, Q.-N., Zhang, L.-X., & Fu, G.-H. (2011). Exploring nonlinear relationships in chemical data using kernel-based methods. Chemometrics and Intelligent Laboratory Systems, 107(1), 106-115. doi:10.1016/j.chemolab.2011.02.004Vitale, R., de Noord, O. E., & Ferrer, A. (2015). Pseudo-sample based contribution plots: innovative tools for fault diagnosis in kernel-based batch process monitoring. Chemometrics and Intelligent Laboratory Systems, 149, 40-52. doi:10.1016/j.chemolab.2015.09.013Hirschfeld, H. O. (1935). A Connection between Correlation and Contingency. Mathematical Proceedings of the Cambridge Philosophical Society, 31(4), 520-524. doi:10.1017/s030500410001351

Novel chemometric proposals for advanced multivariate data analysis, processing and interpretation

The present Ph.D. thesis, primarily conceived to support and reinforce the relation between academic and industrial worlds, was developed in collaboration with Shell Global Solutions (Amsterdam, The Netherlands) in the endeavour of applying and possibly extending well-established latent variable-based approaches (i.e. Principal Component Analysis - PCA - Partial Least Squares regression - PLS - or Partial Least Squares Discriminant Analysis - PLSDA) for complex problem solving not only in the fields of manufacturing troubleshooting and optimisation, but also in the wider environment of multivariate data analysis. To this end, novel efficient algorithmic solutions are proposed throughout all chapters to address very disparate tasks, from calibration transfer in spectroscopy to real-time modelling of streaming flows of data. The manuscript is divided into the following six parts, focused on various topics of interest: Part I - Preface, where an overview of this research work, its main aims and justification is given together with a brief introduction on PCA, PLS and PLSDA; Part II - On kernel-based extensions of PCA, PLS and PLSDA, where the potential of kernel techniques, possibly coupled to specific variants of the recently rediscovered pseudo-sample projection, formulated by the English statistician John C. Gower, is explored and their performance compared to that of more classical methodologies in four different applications scenarios: segmentation of Red-Green-Blue (RGB) images, discrimination of on-/off-specification batch runs, monitoring of batch processes and analysis of mixture designs of experiments; Part III - On the selection of the number of factors in PCA by permutation testing, where an extensive guideline on how to accomplish the selection of PCA components by permutation testing is provided through the comprehensive illustration of an original algorithmic procedure implemented for such a purpose; Part IV - On modelling common and distinctive sources of variability in multi-set data analysis, where several practical aspects of two-block common and distinctive component analysis (carried out by methods like Simultaneous Component Analysis - SCA - DIStinctive and COmmon Simultaneous Component Analysis - DISCO-SCA - Adapted Generalised Singular Value Decomposition - Adapted GSVD - ECO-POWER, Canonical Correlation Analysis - CCA - and 2-block Orthogonal Projections to Latent Structures - O2PLS) are discussed, a new computational strategy for determining the number of common factors underlying two data matrices sharing the same row- or column-dimension is described, and two innovative approaches for calibration transfer between near-infrared spectrometers are presented; Part V - On the on-the-fly processing and modelling of continuous high-dimensional data streams, where a novel software system for rational handling of multi-channel measurements recorded in real time, the On-The-Fly Processing (OTFP) tool, is designed; Part VI - Epilogue, where final conclusions are drawn, future perspectives are delineated, and annexes are included.La presente tesis doctoral, concebida principalmente para apoyar y reforzar la relación entre la academia y la industria, se desarrolló en colaboración con Shell Global Solutions (Amsterdam, Países Bajos) en el esfuerzo de aplicar y posiblemente extender los enfoques ya consolidados basados en variables latentes (es decir, Análisis de Componentes Principales - PCA - Regresión en Mínimos Cuadrados Parciales - PLS - o PLS discriminante - PLSDA) para la resolución de problemas complejos no sólo en los campos de mejora y optimización de procesos, sino también en el entorno más amplio del análisis de datos multivariados. Con este fin, en todos los capítulos proponemos nuevas soluciones algorítmicas eficientes para abordar tareas dispares, desde la transferencia de calibración en espectroscopia hasta el modelado en tiempo real de flujos de datos. El manuscrito se divide en las seis partes siguientes, centradas en diversos temas de interés: Parte I - Prefacio, donde presentamos un resumen de este trabajo de investigación, damos sus principales objetivos y justificaciones junto con una breve introducción sobre PCA, PLS y PLSDA; Parte II - Sobre las extensiones basadas en kernels de PCA, PLS y PLSDA, donde presentamos el potencial de las técnicas de kernel, eventualmente acopladas a variantes específicas de la recién redescubierta proyección de pseudo-muestras, formulada por el estadista inglés John C. Gower, y comparamos su rendimiento respecto a metodologías más clásicas en cuatro aplicaciones a escenarios diferentes: segmentación de imágenes Rojo-Verde-Azul (RGB), discriminación y monitorización de procesos por lotes y análisis de diseños de experimentos de mezclas; Parte III - Sobre la selección del número de factores en el PCA por pruebas de permutación, donde aportamos una guía extensa sobre cómo conseguir la selección de componentes de PCA mediante pruebas de permutación y una ilustración completa de un procedimiento algorítmico original implementado para tal fin; Parte IV - Sobre la modelización de fuentes de variabilidad común y distintiva en el análisis de datos multi-conjunto, donde discutimos varios aspectos prácticos del análisis de componentes comunes y distintivos de dos bloques de datos (realizado por métodos como el Análisis Simultáneo de Componentes - SCA - Análisis Simultáneo de Componentes Distintivos y Comunes - DISCO-SCA - Descomposición Adaptada Generalizada de Valores Singulares - Adapted GSVD - ECO-POWER, Análisis de Correlaciones Canónicas - CCA - y Proyecciones Ortogonales de 2 conjuntos a Estructuras Latentes - O2PLS). Presentamos a su vez una nueva estrategia computacional para determinar el número de factores comunes subyacentes a dos matrices de datos que comparten la misma dimensión de fila o columna y dos planteamientos novedosos para la transferencia de calibración entre espectrómetros de infrarrojo cercano; Parte V - Sobre el procesamiento y la modelización en tiempo real de flujos de datos de alta dimensión, donde diseñamos la herramienta de Procesamiento en Tiempo Real (OTFP), un nuevo sistema de manejo racional de mediciones multi-canal registradas en tiempo real; Parte VI - Epílogo, donde presentamos las conclusiones finales, delimitamos las perspectivas futuras, e incluimos los anexos.La present tesi doctoral, concebuda principalment per a recolzar i reforçar la relació entre l'acadèmia i la indústria, es va desenvolupar en col·laboració amb Shell Global Solutions (Amsterdam, Països Baixos) amb l'esforç d'aplicar i possiblement estendre els enfocaments ja consolidats basats en variables latents (és a dir, Anàlisi de Components Principals - PCA - Regressió en Mínims Quadrats Parcials - PLS - o PLS discriminant - PLSDA) per a la resolució de problemes complexos no solament en els camps de la millora i optimització de processos, sinó també en l'entorn més ampli de l'anàlisi de dades multivariades. A aquest efecte, en tots els capítols proposem noves solucions algorítmiques eficients per a abordar tasques dispars, des de la transferència de calibratge en espectroscopia fins al modelatge en temps real de fluxos de dades. El manuscrit es divideix en les sis parts següents, centrades en diversos temes d'interès: Part I - Prefaci, on presentem un resum d'aquest treball de recerca, es donen els seus principals objectius i justificacions juntament amb una breu introducció sobre PCA, PLS i PLSDA; Part II - Sobre les extensions basades en kernels de PCA, PLS i PLSDA, on presentem el potencial de les tècniques de kernel, eventualment acoblades a variants específiques de la recentment redescoberta projecció de pseudo-mostres, formulada per l'estadista anglés John C. Gower, i comparem el seu rendiment respecte a metodologies més clàssiques en quatre aplicacions a escenaris diferents: segmentació d'imatges Roig-Verd-Blau (RGB), discriminació i monitorització de processos per lots i anàlisi de dissenys d'experiments de mescles; Part III - Sobre la selecció del nombre de factors en el PCA per proves de permutació, on aportem una guia extensa sobre com aconseguir la selecció de components de PCA a través de proves de permutació i una il·lustració completa d'un procediment algorítmic original implementat per a la finalitat esmentada; Part IV - Sobre la modelització de fonts de variabilitat comuna i distintiva en l'anàlisi de dades multi-conjunt, on discutim diversos aspectes pràctics de l'anàlisis de components comuns i distintius de dos blocs de dades (realitzat per mètodes com l'Anàlisi Simultània de Components - SCA - Anàlisi Simultània de Components Distintius i Comuns - DISCO-SCA - Descomposició Adaptada Generalitzada en Valors Singulars - Adapted GSVD - ECO-POWER, Anàlisi de Correlacions Canòniques - CCA - i Projeccions Ortogonals de 2 blocs a Estructures Latents - O2PLS). Presentem al mateix temps una nova estratègia computacional per a determinar el nombre de factors comuns subjacents a dues matrius de dades que comparteixen la mateixa dimensió de fila o columna, i dos plantejaments nous per a la transferència de calibratge entre espectròmetres d'infraroig proper; Part V - Sobre el processament i la modelització en temps real de fluxos de dades d'alta dimensió, on dissenyem l'eina de Processament en Temps Real (OTFP), un nou sistema de tractament racional de mesures multi-canal registrades en temps real; Part VI - Epíleg, on presentem les conclusions finals, delimitem les perspectives futures, i incloem annexos.Vitale, R. (2017). Novel chemometric proposals for advanced multivariate data analysis, processing and interpretation [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/90442TESI

Pattern Recognition

Pattern recognition is a very wide research field. It involves factors as diverse as sensors, feature extraction, pattern classification, decision fusion, applications and others. The signals processed are commonly one, two or three dimensional, the processing is done in real- time or takes hours and days, some systems look for one narrow object class, others search huge databases for entries with at least a small amount of similarity. No single person can claim expertise across the whole field, which develops rapidly, updates its paradigms and comprehends several philosophical approaches. This book reflects this diversity by presenting a selection of recent developments within the area of pattern recognition and related fields. It covers theoretical advances in classification and feature extraction as well as application-oriented works. Authors of these 25 works present and advocate recent achievements of their research related to the field of pattern recognition

New algorithms for the analysis of live-cell images acquired in phase contrast microscopy

La détection et la caractérisation automatisée des cellules constituent un enjeu important dans de nombreux domaines de recherche tels que la cicatrisation, le développement de l'embryon et des cellules souches, l’immunologie, l’oncologie, l'ingénierie tissulaire et la découverte de nouveaux médicaments. Étudier le comportement cellulaire in vitro par imagerie des cellules vivantes et par le criblage à haut débit implique des milliers d'images et de vastes quantités de données. Des outils d'analyse automatisés reposant sur la vision numérique et les méthodes non-intrusives telles que la microscopie à contraste de phase (PCM) sont nécessaires. Comme les images PCM sont difficiles à analyser en raison du halo lumineux entourant les cellules et de la difficulté à distinguer les cellules individuelles, le but de ce projet était de développer des algorithmes de traitement d'image PCM dans Matlab® afin d’en tirer de l’information reliée à la morphologie cellulaire de manière automatisée. Pour développer ces algorithmes, des séries d’images de myoblastes acquises en PCM ont été générées, en faisant croître les cellules dans un milieu avec sérum bovin (SSM) ou dans un milieu sans sérum (SFM) sur plusieurs passages. La surface recouverte par les cellules a été estimée en utilisant un filtre de plage de valeurs, un seuil et une taille minimale de coupe afin d'examiner la cinétique de croissance cellulaire. Les résultats ont montré que les cellules avaient des taux de croissance similaires pour les deux milieux de culture, mais que celui-ci diminue de façon linéaire avec le nombre de passages. La méthode de transformée par ondelette continue combinée à l’analyse d'image multivariée (UWT-MIA) a été élaborée afin d’estimer la distribution de caractéristiques morphologiques des cellules (axe majeur, axe mineur, orientation et rondeur). Une analyse multivariée réalisée sur l’ensemble de la base de données (environ 1 million d’images PCM) a montré d'une manière quantitative que les myoblastes cultivés dans le milieu SFM étaient plus allongés et plus petits que ceux cultivés dans le milieu SSM. Les algorithmes développés grâce à ce projet pourraient être utilisés sur d'autres phénotypes cellulaires pour des applications de criblage à haut débit et de contrôle de cultures cellulaires.Automated cell detection and characterization is important in many research fields such as wound healing, embryo development, immune system studies, cancer research, parasite spreading, tissue engineering, stem cell research and drug research and testing. Studying in vitro cellular behavior via live-cell imaging and high-throughput screening involves thousands of images and vast amounts of data, and automated analysis tools relying on machine vision methods and non-intrusive methods such as phase contrast microscopy (PCM) are a necessity. However, there are still some challenges to overcome, since PCM images are difficult to analyze because of the bright halo surrounding the cells and blurry cell-cell boundaries when they are touching. The goal of this project was to develop image processing algorithms to analyze PCM images in an automated fashion, capable of processing large datasets of images to extract information related to cellular viability and morphology. To develop these algorithms, a large dataset of myoblasts images acquired in live-cell imaging (in PCM) was created, growing the cells in either a serum-supplemented (SSM) or a serum-free (SFM) medium over several passages. As a result, algorithms capable of computing the cell-covered surface and cellular morphological features were programmed in Matlab®. The cell-covered surface was estimated using a range filter, a threshold and a minimum cut size in order to look at the cellular growth kinetics. Results showed that the cells were growing at similar paces for both media, but their growth rate was decreasing linearly with passage number. The undecimated wavelet transform multivariate image analysis (UWT-MIA) method was developed, and was used to estimate cellular morphological features distributions (major axis, minor axis, orientation and roundness distributions) on a very large PCM image dataset using the Gabor continuous wavelet transform. Multivariate data analysis performed on the whole database (around 1 million PCM images) showed in a quantitative manner that myoblasts grown in SFM were more elongated and smaller than cells grown in SSM. The algorithms developed through this project could be used in the future on other cellular phenotypes for high-throughput screening and cell culture control applications