Computer Vision System for Non-Destructive and Contactless Evaluation of Quality Traits in Fresh Rocket Leaves (Diplotaxis Tenuifolia L.)

La tesi di dottorato è incentrata sull'analisi di tecnologie non distruttive per il controllo della qualità dei prodotti agroalimentari, lungo l'intera filiera agroalimentare. In particolare, la tesi riguarda l'applicazione del sistema di visione artificiale per valutare la qualità delle foglie di rucola fresh-cut. La tesi è strutturata in tre parti (introduzione, applicazioni sperimentali e conclusioni) e in cinque capitoli, rispettivamente il primo e il secondo incentrati sulle tecnologie non distruttive e in particolare sui sistemi di computer vision per il monitoraggio della qualità dei prodotti agroalimentari. Il terzo, quarto e quinto capitolo mirano a valutare le foglie di rucola sulla base della stima di parametri qualitativi, considerando diversi aspetti: (i) la variabilità dovuta alle diverse pratiche agricole, (ii) la senescenza dei prodotti confezionati e non, e (iii) lo sviluppo e sfruttamento dei vantaggi di nuovi modelli più semplici rispetto al machine learning utilizzato negli esperimenti precedenti. Il lavoro di ricerca di questa tesi di dottorato è stato svolto dall'Università di Foggia, dall'Istituto di Scienze delle Produzioni Alimentari (ISPA) e dall'Istituto di Tecnologie e Sistemi Industriali Intelligenti per le Manifatture Avanzate (STIIMA) del Consiglio Nazionale delle Ricerche (CNR). L’attività di ricerca è stata condotta nell'ambito del Progetto SUS&LOW (Sustaining Low-impact Practices in Horticulture through Non-destructive Approach to Provide More Information on Fresh Produce History & Quality), finanziato dal MUR-PRIN 2017, e volto a sostenere la qualità della produzione e dell'ambiente utilizzando pratiche agricole a basso input e la valutazione non distruttiva della qualità di prodotti ortofrutticoli.The doctoral thesis focused on the analysis of non-destructive technologies available for the control quality of agri-food products, along the whole supply chain. In particular, the thesis concerns the application of computer vision system to evaluate the quality of fresh rocket leaves. The thesis is structured in three parts (introduction, experimental applications and conclusions) and in 5 chapters, the first and second focused on non-destructive technologies and in particular on computer vision systems for monitoring the quality of agri-food products, respectively. The third, quarter, and fifth chapters aim to assess the rocket leaves based on the estimation of quality aspects, considering different aspects: (i) the variability due to the different agricultural practices, (ii) the senescence of packed and unpacked products, and (iii) development and exploitation of the advantages of new models simpler than the machine learning used in the previous experiments. The research work of this doctoral thesis was carried out by the University of Foggia, the Institute of Science of Food Production (ISPA) and the Institute of Intelligent Industrial Technologies and Systems for Advanced Manufacturing (STIIMA) of National Research Council (CNR). It was conducted within the Project SUS&LOW (Sustaining Low-impact Practices in Horticulture through Non-destructive Approach to Provide More Information on Fresh Produce History & Quality), funded by MUR- PRIN 2017, and aimed at sustaining quality of production and of the environment using low input agricultural practices and non-destructive quality evaluation

Combining omnidirectional vision with polarization vision for robot navigation

La polarisation est le phénomène qui décrit les orientations des oscillations des ondes lumineuses qui sont limitées en direction. La lumière polarisée est largement utilisée dans le règne animal,à partir de la recherche de nourriture, la défense et la communication et la navigation. Le chapitre (1) aborde brièvement certains aspects importants de la polarisation et explique notre problématique de recherche. Nous visons à utiliser un capteur polarimétrique-catadioptrique car il existe de nombreuses applications qui peuvent bénéficier d'une telle combinaison en vision par ordinateur et en robotique, en particulier pour l'estimation d'attitude et les applications de navigation. Le chapitre (2) couvre essentiellement l'état de l'art de l'estimation d'attitude basée sur la vision.Quand la lumière non-polarisée du soleil pénètre dans l'atmosphère, l'air entraine une diffusion de Rayleigh, et la lumière devient partiellement linéairement polarisée. Le chapitre (3) présente les motifs de polarisation de la lumière naturelle et couvre l'état de l'art des méthodes d'acquisition des motifs de polarisation de la lumière naturelle utilisant des capteurs omnidirectionnels (par exemple fisheye et capteurs catadioptriques). Nous expliquons également les caractéristiques de polarisation de la lumière naturelle et donnons une nouvelle dérivation théorique de son angle de polarisation.Notre objectif est d'obtenir une vue omnidirectionnelle à 360 associée aux caractéristiques de polarisation. Pour ce faire, ce travail est basé sur des capteurs catadioptriques qui sont composées de surfaces réfléchissantes et de lentilles. Généralement, la surface réfléchissante est métallique et donc l'état de polarisation de la lumière incidente, qui est le plus souvent partiellement linéairement polarisée, est modifiée pour être polarisée elliptiquement après réflexion. A partir de la mesure de l'état de polarisation de la lumière réfléchie, nous voulons obtenir l'état de polarisation incident. Le chapitre (4) propose une nouvelle méthode pour mesurer les paramètres de polarisation de la lumière en utilisant un capteur catadioptrique. La possibilité de mesurer le vecteur de Stokes du rayon incident est démontré à partir de trois composants du vecteur de Stokes du rayon réfléchi sur les quatre existants.Lorsque les motifs de polarisation incidents sont disponibles, les angles zénithal et azimutal du soleil peuvent être directement estimés à l'aide de ces modèles. Le chapitre (5) traite de l'orientation et de la navigation de robot basées sur la polarisation et différents algorithmes sont proposés pour estimer ces angles dans ce chapitre. A notre connaissance, l'angle zénithal du soleil est pour la première fois estimé dans ce travail à partir des schémas de polarisation incidents. Nous proposons également d'estimer l'orientation d'un véhicule à partir de ces motifs de polarisation.Enfin, le travail est conclu et les possibles perspectives de recherche sont discutées dans le chapitre (6). D'autres exemples de schémas de polarisation de la lumière naturelle, leur calibrage et des applications sont proposées en annexe (B).Notre travail pourrait ouvrir un accès au monde de la vision polarimétrique omnidirectionnelle en plus des approches conventionnelles. Cela inclut l'orientation bio-inspirée des robots, des applications de navigation, ou bien la localisation en plein air pour laquelle les motifs de polarisation de la lumière naturelle associés à l'orientation du soleil à une heure précise peuvent aboutir à la localisation géographique d'un véhiculePolarization is the phenomenon that describes the oscillations orientations of the light waves which are restricted in direction. Polarized light has multiple uses in the animal kingdom ranging from foraging, defense and communication to orientation and navigation. Chapter (1) briefly covers some important aspects of polarization and explains our research problem. We are aiming to use a polarimetric-catadioptric sensor since there are many applications which can benefit from such combination in computer vision and robotics specially robot orientation (attitude estimation) and navigation applications. Chapter (2) mainly covers the state of art of visual based attitude estimation.As the unpolarized sunlight enters the Earth s atmosphere, it is Rayleigh-scattered by air, and it becomes partially linearly polarized. This skylight polarization provides a signi cant clue to understanding the environment. Its state conveys the information for obtaining the sun orientation. Robot navigation, sensor planning, and many other applications may bene t from using this navigation clue. Chapter (3) covers the state of art in capturing the skylight polarization patterns using omnidirectional sensors (e.g fisheye and catadioptric sensors). It also explains the skylight polarization characteristics and gives a new theoretical derivation of the skylight angle of polarization pattern. Our aim is to obtain an omnidirectional 360 view combined with polarization characteristics. Hence, this work is based on catadioptric sensors which are composed of reflective surfaces and lenses. Usually the reflective surface is metallic and hence the incident skylight polarization state, which is mostly partially linearly polarized, is changed to be elliptically polarized after reflection. Given the measured reflected polarization state, we want to obtain the incident polarization state. Chapter (4) proposes a method to measure the light polarization parameters using a catadioptric sensor. The possibility to measure the incident Stokes is proved given three Stokes out of the four reflected Stokes. Once the incident polarization patterns are available, the solar angles can be directly estimated using these patterns. Chapter (5) discusses polarization based robot orientation and navigation and proposes new algorithms to estimate these solar angles where, to the best of our knowledge, the sun zenith angle is firstly estimated in this work given these incident polarization patterns. We also propose to estimate any vehicle orientation given these polarization patterns. Finally the work is concluded and possible future research directions are discussed in chapter (6). More examples of skylight polarization patterns, their calibration, and the proposed applications are given in appendix (B). Our work may pave the way to move from the conventional polarization vision world to the omnidirectional one. It enables bio-inspired robot orientation and navigation applications and possible outdoor localization based on the skylight polarization patterns where given the solar angles at a certain date and instant of time may infer the current vehicle geographical location.DIJON-BU Doc.électronique (212319901) / SudocSudocFranceF

Do-it-yourself instruments and data processing methods for developing marine citizen observatories

La consulta íntegra de la tesi, inclosos els articles no comunicats públicament per drets d'autor, es pot realitzar prèvia petició a l'Arxiu de la UPCWater is the most important resource for living on planet Earth, covering more than 70% of its surface. The oceans represent more than 97% of the planet total water and they are where more than the 99.5% of the living beings are concentrated. A great number of ecosystems depend on the health of these oceans; their study and protection are necessary. Large datasets over long periods of time and over wide geographical areas can be required to assess the health of aquatic ecosystems. The funding needed for data collection is considerable and limited, so it is important to look at new cost-effective ways of obtaining and processing marine environmental data. The feasible solution at present is to develop observational infrastructures that may increase significantly the conventional sampling capabilities. In this study we promote to achieve this solution with the implementation of Citizen Observatories, based on volunteer participation. Citizen observatories are platforms that integrate the latest information technologies to digitally connect citizens, improving observation skills for developing a new type of research known as Citizen Science. Citizen science has the potential to increase the knowledge of the environment, and aquatic ecosystems in particular, through the use of people with no specific scientific training to collect and analyze large data sets. We believe that citizen science based tools -open source software coupled with low-cost do-it-yourself hardware- can help to close the gap between science and citizens in the oceanographic field. As the public is actively engaged in the analysis of data, the research also provides a strong avenue for public education. This is the objective of this thesis, to demonstrate how open source software and low-cost do-it-yourself hardware are effectively applied to oceanographic research and how can it develop into citizen science. We analyze four different scenarios where this idea is demonstrated: an example of using open source software for video analysis where lobsters were monitored; a demonstration of using similar video processing techniques on in-situ low-cost do-it-yourself hardware for submarine fauna monitoring; a study using open source machine learning software as a method to improve biological observations; and last but not least, some preliminar results, as proof of concept, of how manual water sampling could be replaced by low-cost do-it-yourself hardware with optical sensors.L’aigua és el recurs més important per la vida al planeta Terra, cobrint més del 70% de la seva superfície. Els oceans representen més del 70% de tota l'aigua del planeta, i és on estan concentrats més del 99.5% dels éssers vius. Un gran nombre d'ecosistemes depenen de la salut d'aquests oceans; el seu estudi i protecció són necessaris. Grans conjunts de dades durant llargs períodes de temps i al llarg d’amples àrees geogràfiques poden ser necessaris per avaluar la salut dels ecosistemes aquàtics. El finançament necessari per aquesta recol·lecció de dades és considerable però limitat, i per tant és important trobar noves formes més rendibles d’obtenir i processar dades mediambientals marines. La solució factible actualment és la de desenvolupar infraestructures observacionals que puguin incrementar significativament les capacitats de mostreig convencionals. En aquest estudi promovem que es pot assolir aquesta solució amb la implementació d’Observatoris Ciutadans, basats en la participació de voluntaris. Els observatoris ciutadans són plataformes que integren les últimes tecnologies de la informació amb ciutadans digitalment connectats, millorant les capacitats d’observació, per desenvolupar un nou tipus de recerca coneguda com a Ciència Ciutadana. La ciència ciutadana té el potencial d’incrementar el coneixement del medi ambient, i dels ecosistemes aquàtics en particular, mitjançant l'ús de persones sense coneixement científic específic per recollir i analitzar grans conjunts de dades. Creiem que les eines basades en ciència ciutadana -programari lliure juntament amb maquinari de baix cost i del tipus "fes-ho tu mateix" (do-it-yourself en anglès)- poden ajudar a apropar la ciència del camp oceanogràfic als ciutadans. A mesura que el gran públic participa activament en l'anàlisi de dades, la recerca esdevé també una nova via d’educació pública. Aquest és l’objectiu d’aquesta tesis, demostrar com el programari lliure i el maquinari de baix cost "fes-ho tu mateix" s’apliquen de forma efectiva a la recerca oceanogràfica i com pot desenvolupar-se cap a ciència ciutadana. Analitzem quatre escenaris diferents on es demostra aquesta idea: un exemple d’ús de programari lliure per anàlisi de vídeos de monitoratge de llagostes; una demostració utilitzant tècniques similars de processat de vídeo en un dispositiu in-situ de baix cost "fes-ho tu mateix" per monitoratge de fauna submarina; un estudi utilitzant programari lliure d’aprenentatge automàtic (machine learning en anglès) com a mètode per millorar observacions biològiques; i finalment uns resultats preliminars, com a prova de la seva viabilitat, de com un mostreig manual de mostres d’aigua podria ser reemplaçat per maquinari de baix cost "fes-ho tu mateix" amb sensors òptics.Postprint (published version

NASA Tech Briefs, November 2002

Topics include: a technology focus on engineering materials, electronic components and systems, software, mechanics, machinery/automation, manufacturing, bio-medical, physical sciences, information sciences book and reports, and a special section of Photonics Tech Briefs

Do-it-yourself instruments and data processing methods for developing marine citizen observatories

Water is the most important resource for living on planet Earth, covering more than 70% of its surface. The oceans represent more than 97% of the planet total water and they are where more than the 99.5% of the living beings are concentrated. A great number of ecosystems depend on the health of these oceans; their study and protection are necessary. Large datasets over long periods of time and over wide geographical areas can be required to assess the health of aquatic ecosystems. The funding needed for data collection is considerable and limited, so it is important to look at new cost-effective ways of obtaining and processing marine environmental data. The feasible solution at present is to develop observational infrastructures that may increase significantly the conventional sampling capabilities. In this study we promote to achieve this solution with the implementation of Citizen Observatories, based on volunteer participation. Citizen observatories are platforms that integrate the latest information technologies to digitally connect citizens, improving observation skills for developing a new type of research known as Citizen Science. Citizen science has the potential to increase the knowledge of the environment, and aquatic ecosystems in particular, through the use of people with no specific scientific training to collect and analyze large data sets. We believe that citizen science based tools -open source software coupled with low-cost do-it-yourself hardware- can help to close the gap between science and citizens in the oceanographic field. As the public is actively engaged in the analysis of data, the research also provides a strong avenue for public education. This is the objective of this thesis, to demonstrate how open source software and low-cost do-it-yourself hardware are effectively applied to oceanographic research and how can it develop into citizen science. We analyze four different scenarios where this idea is demonstrated: an example of using open source software for video analysis where lobsters were monitored; a demonstration of using similar video processing techniques on in-situ low-cost do-it-yourself hardware for submarine fauna monitoring; a study using open source machine learning software as a method to improve biological observations; and last but not least, some preliminar results, as proof of concept, of how manual water sampling could be replaced by low-cost do-it-yourself hardware with optical sensors.L’aigua és el recurs més important per la vida al planeta Terra, cobrint més del 70% de la seva superfície. Els oceans representen més del 70% de tota l'aigua del planeta, i és on estan concentrats més del 99.5% dels éssers vius. Un gran nombre d'ecosistemes depenen de la salut d'aquests oceans; el seu estudi i protecció són necessaris. Grans conjunts de dades durant llargs períodes de temps i al llarg d’amples àrees geogràfiques poden ser necessaris per avaluar la salut dels ecosistemes aquàtics. El finançament necessari per aquesta recol·lecció de dades és considerable però limitat, i per tant és important trobar noves formes més rendibles d’obtenir i processar dades mediambientals marines. La solució factible actualment és la de desenvolupar infraestructures observacionals que puguin incrementar significativament les capacitats de mostreig convencionals. En aquest estudi promovem que es pot assolir aquesta solució amb la implementació d’Observatoris Ciutadans, basats en la participació de voluntaris. Els observatoris ciutadans són plataformes que integren les últimes tecnologies de la informació amb ciutadans digitalment connectats, millorant les capacitats d’observació, per desenvolupar un nou tipus de recerca coneguda com a Ciència Ciutadana. La ciència ciutadana té el potencial d’incrementar el coneixement del medi ambient, i dels ecosistemes aquàtics en particular, mitjançant l'ús de persones sense coneixement científic específic per recollir i analitzar grans conjunts de dades. Creiem que les eines basades en ciència ciutadana -programari lliure juntament amb maquinari de baix cost i del tipus "fes-ho tu mateix" (do-it-yourself en anglès)- poden ajudar a apropar la ciència del camp oceanogràfic als ciutadans. A mesura que el gran públic participa activament en l'anàlisi de dades, la recerca esdevé també una nova via d’educació pública. Aquest és l’objectiu d’aquesta tesis, demostrar com el programari lliure i el maquinari de baix cost "fes-ho tu mateix" s’apliquen de forma efectiva a la recerca oceanogràfica i com pot desenvolupar-se cap a ciència ciutadana. Analitzem quatre escenaris diferents on es demostra aquesta idea: un exemple d’ús de programari lliure per anàlisi de vídeos de monitoratge de llagostes; una demostració utilitzant tècniques similars de processat de vídeo en un dispositiu in-situ de baix cost "fes-ho tu mateix" per monitoratge de fauna submarina; un estudi utilitzant programari lliure d’aprenentatge automàtic (machine learning en anglès) com a mètode per millorar observacions biològiques; i finalment uns resultats preliminars, com a prova de la seva viabilitat, de com un mostreig manual de mostres d’aigua podria ser reemplaçat per maquinari de baix cost "fes-ho tu mateix" amb sensors òptics

Pulse Oxigraphy: And other new in-depth perspectives through the near infrared window

The aim of this thesis was to investigate the feasability of contactless imaging pulse oximetry (proposed term: pulse oxigraphy). The patent disclosed in chapter 2 claims that such pulse oxigraphy can be achieved with camera-derived photoplethysmographic pulse waves at three wavelengths, preferably being 660, 810 and 940nm. From the absorption curves of hemoglobin and oxyhemoglobin it can be easily derived that two of these wavelengths (660 and 940nm) contain oxygenation-related information, and they have proven to be useful for conventional pulse oximetry (in transmission- mode as well as in reflectance-mode). The additional third wavelength (810nm) lies at a so-called isobestic point where the absorption curves of hemoglobin and oxyhemoglobin intersect. Thus, images and/or plethysmographic pulse waves recorded at 810nm do not contain oxygenation-related information, which is useful for reference purposes when dealing with shadows, reflections, movement artifacts and variations in geometry. With regard to pulse oxigraphy the following results were obtained: In chapter 3 we proved that it is possible to derive photoplethysmographic pulse waves containing the heart rythm of a living person at all three required wavelengths from camera recordings collected at a distance of 72 cm. To investigate and validate the capabilities for pulse oxigraphy with this set up, direct measurements on volunteers were sub optimal, because of: Signal-to-noise issues, sequentially recorded heartbeats for oxygen saturation calculations, and lack of a method to induce prolonged stable and adjustable oxygen saturation levels

Clinical fluorescence spectroscopy and imaging for the detection of early carcinoma by autofluorescence bronchoscopy and the study of the protoporphyrin IX pharmacokinetics in the endometrium

The aim of this thesis is to optimise and gain fundamental information on two applications of photomedicine using fluorescence imaging and spectrofluorometry: (1) the detection of early bronchial cancer by autofluorescence imaging and (2) the endometrial ablation by photodynamic therapy (PDT) based on the use of Protoporphyrin IX (PpIX). Fluorescence imaging and spectroscopy require a fluorochrome localised within the tissue. The fluorochrome can either be endogenous (naturally synthesised in the body), endogenously induced (synthesised in the body from an administered drug), or exogenous (synthesised outside the body). This thesis concentrates on the clinical applications of the endogenous and an endogenously induced fluorochrome (PpIX). Therefore, this work has been divided into two parts according to the type of fluorochromes. The numerous endogenous fluorochromes occur naturally. They are collectively responsible for the fluorescence properties of biological tissues. This tissue's intrinsic fluorescence is also referred to as autofluorescence (AF). The AF of bronchial tissues, change when they become dysplastic or neoplastic. Early neoplastic or dysplastic lesions show an overall decrease in the AF intensity as well as a distorsion of the spectral shape. Endoscopic imaging devices rely on this principle to detect early neoplastic lesions in the tracheo-bronchial tree. The first part of this thesis describes our efforts to improve the performance of AFB and to gather insight into the mechanisms at the origin of the AF contrast in the bronchi. For this purpose, we conducted a number of clinical and ex vivo studies using imaging and spectrofluorometry. Our initial clinical imaging study revealed that the detection of a red background image instead of the red AF image increased the lesion-to-healthy tissue contrast by a factor of 2. This improvement has been implemented in an AFB device that is currently commercialised by the Richard Wolf Endoskope GmbH. In a seperate clinical imaging study we investigated the influence of the excitation wavelength on the AF contrast. Using a narrowband (6 nm FWHM) excitation around 410 nm resulted in a 1.5 times higher lesion-to-healthy tissue intensity contrast than observed with a comparable broadband (80 nm FWHM) excitation. A supplemental study showed that short wavelength blue backscattered light around 430 nm has the potential to discriminate true positive lesions (i.e. early neoplastic lesions detected positive with the AFB system) from false positive lesions (i.e. benign tissue changes detected positive with the AFB system). A spectrofluorometric ex vivo study was performed to gain insight on the mechanisms at the origin of these contrasts. Five principal mechanisms are discussed, namely changes of: (1) the fluorochrome's concentration, (2) the fluorochrome's metabolic status, (3) the fluorochrome's physico-chemical microenvironment, (4) the tissue architecture such as thickening of the epithelium, and (5) the concentration of light absorbing chromophores such as haemoglobin. We measured formalin fixed human bronchial tissue samples with an optical fibre based spectrofluorometer. The formalin fixed bronchial tissue samples showed a general decrease of the AF of early lesions compared to the healthy tissues. However, no distortion of the lesions' AF spectra with respect to that of the healthy tissues was observed. These results were confirmed by imaging of the tissue samples with our AFB system. The observations from these ex vivo studies together with results obtained in clinics with our imaging system lead us to conclude that the AF contrast can be attributed to a combined effect induced by: (1) changes in the architecture of superficial tissues and (2) the concentration and spatial distribution of haemoglobin in the submucosa. Furthermore, we investigated inter-patient variations of the bronchial AF to estimate their impact on the spectral/photonic design of AFB systems. An endoscopic reference with tissue-like optical and spectral properties was designed for this purpose. Surprisingly, the AF intensities in spectroscopy of the human bronchi showed only minor (< 30 %) variations from one individual to another. The exogenously induced fluorochrome Protoporphyrin IX (PpIX) is synthesised from 5-aminolaevulinic acid (5-ALA) in the haeme biosynthetic pathway. PpIX is widely used in PDT and fluoresence detection for both malignant and benign, lesions. The second part of this thesis deals with the pharmacokinetics of 5-ALA induced PpIX in the endometrium. The final goal of this study was the optimisation of the treatment protocol for photodynamic endometrial ablation to treat menorrhagia and hypermenorrhea. The PpIX build-up in the human endometrium was measured in vivo by spectrofluorometry following intra-uterine instillation of 5-ALA. An intra-uterine optical-fibre based probe was designed for this purpose. The PpIX pharmacokinetics showed important inter-patient and intra-patient variations regarding the time interval between the drug instillation and the maximal PpIX fluorescence. Indeed, we have found that this time interval ranges between 0.5 and 5 hours. The maximal measured PpIX fluorescence intensities varied by one order of magnitude from one patient to another. Finally, no correlation was found between the characteristics of the PpIX build-up and the patient's hormonal status