3D image acquisition system based on shape from focus technique

agent Agrosup Dijon de l'UMREcolDurGEAPSIThis paper describes the design of a 3D image acquisition system dedicated to natural complex scenes composed of randomly distributed objects with spatial discontinuities. In agronomic sciences, the 3D acquisition of natural scene is difficult due to the complex nature of the scenes. Our system is based on the Shape from Focus technique initially used in the microscopic domain. We propose to adapt this technique to the macroscopic domain and we detail the system as well as the image processing used to perform such technique. The Shape from Focus technique is a monocular and passive 3D acquisition method that resolves the occlusion problem affecting the multi-cameras systems. Indeed, this problem occurs frequently in natural complex scenes like agronomic scenes. The depth information is obtained by acting on optical parameters and mainly the depth of field. A focus measure is applied on a 2D image stack previously acquired by the system. When this focus measure is performed, we can create the depth map of the scene

A proposal for automatic fruit harvesting by combining a low cost stereovision camera and a robotic arm

This paper proposes the development of an automatic fruit harvesting system by combining a low cost stereovision camera and a robotic arm placed in the gripper tool. The stereovision camera is used to estimate the size, distance and position of the fruits whereas the robotic arm is used to mechanically pickup the fruits. The low cost stereovision system has been tested in laboratory conditions with a reference small object, an apple and a pear at 10 different intermediate distances from the camera. The average distance error was from 4% to 5%, and the average diameter error was up to 30% in the case of a small object and in a range from 2% to 6% in the case of a pear and an apple. The stereovision system has been attached to the gripper tool in order to obtain relative distance, orientation and size of the fruit. The harvesting stage requires the initial fruit location, the computation of the inverse kinematics of the robotic arm in order to place the gripper tool in front of the fruit, and a final pickup approach by iteratively adjusting the vertical and horizontal position of the gripper tool in a closed visual loop. The complete system has been tested in controlled laboratory conditions with uniform illumination applied to the fruits. As a future work, this system will be tested and improved in conventional outdoor farming conditions

Application of image processing methodologies for fruit detection and analysis

En aquesta memòria es presenten diversos treballs d'investigació centrats en l’automatització d’operacions agrícoles mitjançant l’aplicació de diverses tècniques de processament d’imatge. En primer lloc es presenta un mètode desenvolupat per detectar i comptar raïms mitjançant la localització de pics d'intensitat en superfícies esfèriques. En segon lloc es desenvolupa un sistema de recol•lecció automàtica de fruita mitjançant la combinació d'una càmera estereoscòpica de baix cost i un braç robòtic. En tercer lloc es proposa una aplicació en què es desenvolupa un mètode basat en l'ús de la informació de color per a la verificació d'una varietat de nectarines de forma automàtica i individual en una línia d’embalatge de fruita. Finalment s’han estudiat les correlacions entre els paràmetres de qualitat de la fruita i el espectre visible de la seva pell amb l’objectiu de controlar la seva qualitat de forma no destructiva durant el seu emmagatzematge.En esta memoria se presentan diversos trabajos de investigación centrados en la automatización de operaciones agrícolas mediante la aplicación de distintas técnicas de procesado de imágenes. En primer lugar se presenta un método desarrollado para detectar y contar uvas rojas mediante la identificación de picos de intensidad en las superficies esféricas. En segundo lugar se desarrolla un sistema de recolección automática de fruta mediante la combinación de una cámara estereoscópica de bajo coste y un brazo robótico. En tercer lugar se propone una aplicación en la que se desarrolla un método de procesamiento de imágenes basado en el uso de la información de color para la verificación de una variedad de nectarinas de forma automática e individual en una línea de envasado de fruta. Finalmente se han estudiado las correlaciones entre los parámetros de calidad de la fruta y el espectro visible de su piel con el fin de controlar su calidad de forma no destructiva durante el almacenamiento.This memory introduces several research works developed to automate agricultural tasks by applying image processing techniques. In the first place a new image processing method is proposed for detecting and counting red grapes by identifying specular reflection peaks from spherical surfaces. The proposal of the second application is to develop an automatic fruit harvesting system by combining a low cost stereovision camera and a robotic arm. The third application proposed is to develop a novel image processing method based on the use of color information to verify an in-line automatic and individual nectarine variety verification in a fruitpacking line. Finally, a study focused on assessing correlations between post-storage fruit quality indices and the visible spectra of the skin of the fruit is proposed in order to control fruit quality in a non-destructive way during the storage

Simultaneous Multispectral Imaging: Using Multiview Computational Compressive Sensing

Multispectral imaging is traditionally performed using a combination of an imaging device with a filter bank such as a filter wheel or a form of tunable filter, or a combination of many imaging devices with various spectral beam splitting optics. The complexity and size of these devices seem to be the limiting factor of their adoption and use in various fields that could potentially benefit from this imaging modality. With the advent of nanophotonics, there has been a surge in single camera, snapshot, multispectral imaging exploiting the capabilities of nanotechnology to devise pixel-based spectral filters. This new form of sensing, which can be classified as compressive sensing, has its limitations. One example is the laborious process of fabricating the filter bank and installing it into a detector since the detector fabrication process is completely removed from the filter fabrication process. The work presented here will describe an optical design that would enable a single-camera, simultaneous multispectral imaging via multiview computational compressive sensing. A number of points-of-view (POVs) of the field-of-view (FOV) of the camera are generated and directed through an assortment of spectral pre-filters en route to the camera. The image of each of the POVs is then captured on a different spatial location on the detector. With the spectral response of the detector pixels well characterized, spatial and spectral compressive sensing is performed as the images are recorded. Various computational techniques are used in this work which would: register the images captured from multiple views resulting in even more sparsely sensed images; perform spatial interpolation of the sparsely sampled spectral images; implement hyper-focusing of the images from all POVs captured as some defocusing will happen as the result of the discrepancy in the optical paths in each view; execute numerical dimensionality reduction analysis to extract information from the multispectral images. The spectral imaging capabilities of the device are tested with a collection of fluorescent microspheres. The spectral sensing capability of the device is examined by measuring the fluorescent spectra of adulterated edible oils and demonstrating the ability of the imaging system to differentiate between various types of oil as well as various levels of contamination. Lastly, the system is used to scrutinize samples of black ink from different pen manufacturers, and is able to discriminate between the different inks

Crop plant reconstruction and feature extraction based on 3-D vision

3-D imaging is increasingly affordable and offers new possibilities for a more efficient agricul-tural practice with the use of highly advances technological devices. Some reasons contrib-uting to this possibility include the continuous increase in computer processing power, the de-crease in cost and size of electronics, the increase in solid state illumination efficiency and the need for greater knowledge and care of the individual crops. The implementation of 3-D im-aging systems in agriculture is impeded by the economic justification of using expensive de-vices for producing relative low-cost seasonal products. However, this may no longer be true since low-cost 3-D sensors, such as the one used in this work, with advance technical capabili-ties are already available. The aim of this cumulative dissertation was to develop new methodologies to reconstruct the 3-D shape of agricultural environment in order to recognized and quantitatively describe struc-tures, in this case: maize plants, for agricultural applications such as plant breeding and preci-sion farming. To fulfil this aim a comprehensive review of the 3-D imaging systems in agricul-tural applications was done to select a sensor that was affordable and has not been fully inves-tigated in agricultural environments. A low-cost TOF sensor was selected to obtain 3-D data of maize plants and a new adaptive methodology was proposed for point cloud rigid registra-tion and stitching. The resulting maize 3-D point clouds were highly dense and generated in a cost-effective manner. The validation of the methodology showed that the plants were recon-structed with high accuracies and the qualitative analysis showed the visual variability of the plants depending on the 3-D perspective view. The generated point cloud was used to obtain information about the plant parameters (stem position and plant height) in order to quantita-tively describe the plant. The resulting plant stem positions were estimated with an average mean error and standard deviation of 27 mm and 14 mm, respectively. Additionally, meaning-ful information about the plant height profile was also provided, with an average overall mean error of 8.7 mm. Since the maize plants considered in this research were highly heterogeneous in height, some of them had folded leaves and were planted with standard deviations that emulate the real performance of a seeder; it can be said that the experimental maize setup was a difficult scenario. Therefore, a better performance, for both, plant stem position and height estimation could be expected for a maize field in better conditions. Finally, having a 3-D re-construction of the maize plants using a cost-effective sensor, mounted on a small electric-motor-driven robotic platform, means that the cost (either economic, energetic or time) of gen-erating every point in the point cloud is greatly reduced compared with previous researches.Die 3D-Bilderfassung ist zunehmend kostengünstiger geworden und bietet neue Möglichkeiten für eine effizientere landwirtschaftliche Praxis durch den Einsatz hochentwickelter technologischer Geräte. Einige Gründe, die diese ermöglichen, ist das kontinuierliche Wachstum der Computerrechenleistung, die Kostenreduktion und Miniaturisierung der Elektronik, die erhöhte Beleuchtungseffizienz und die Notwendigkeit einer besseren Kenntnis und Pflege der einzelnen Pflanzen. Die Implementierung von 3-D-Sensoren in der Landwirtschaft wird durch die wirtschaftliche Rechtfertigung der Verwendung teurer Geräte zur Herstellung von kostengünstigen Saisonprodukten verhindert. Dies ist jedoch nicht mehr länger der Fall, da kostengünstige 3-D-Sensoren, bereits verfügbar sind. Wie derjenige dier in dieser Arbeit verwendet wurde. Das Ziel dieser kumulativen Dissertation war, neue Methoden für die Visualisierung die 3-D-Form der landwirtschaftlichen Umgebung zu entwickeln, um Strukturen quantitativ zu beschreiben: in diesem Fall Maispflanzen für landwirtschaftliche Anwendungen wie Pflanzenzüchtung und Precision Farming zu erkennen. Damit dieses Ziel erreicht wird, wurde eine umfassende Überprüfung der 3D-Bildgebungssysteme in landwirtschaftlichen Anwendungen durchgeführt, um einen Sensor auszuwählen, der erschwinglich und in landwirtschaftlichen Umgebungen noch nicht ausgiebig getestet wurde. Ein kostengünstiger TOF-Sensor wurde ausgewählt, um 3-D-Daten von Maispflanzen zu erhalten und eine neue adaptive Methodik wurde für die Ausrichtung von Punktwolken vorgeschlagen. Die resultierenden Mais-3-D-Punktwolken hatten eine hohe Punktedichte und waren in einer kosteneffektiven Weise erzeugt worden. Die Validierung der Methodik zeigte, dass die Pflanzen mit hoher Genauigkeit rekonstruiert wurden und die qualitative Analyse die visuelle Variabilität der Pflanzen in Abhängigkeit der 3-D-Perspektive zeigte. Die erzeugte Punktwolke wurde verwendet, um Informationen über die Pflanzenparameter (Stammposition und Pflanzenhöhe) zu erhalten, die die Pflanze quantitativ beschreibt. Die resultierenden Pflanzenstammpositionen wurden mit einem durchschnittlichen mittleren Fehler und einer Standardabweichung von 27 mm bzw. 14 mm berechnet. Zusätzlich wurden aussagekräftige Informationen zum Pflanzenhöhenprofil mit einem durchschnittlichen Gesamtfehler von 8,7 mm bereitgestellt. Da die untersuchten Maispflanzen in der Höhe sehr heterogen waren, hatten einige von ihnen gefaltete Blätter und wurden mit Standardabweichungen gepflanzt, die die tatsächliche Genauigkeit einer Sämaschine nachahmen. Man kann sagen, dass der experimentelle Versuch ein schwieriges Szenario war. Daher könnte für ein Maisfeld unter besseren Bedingungen eine besseres Resultat sowohl für die Pflanzenstammposition als auch für die Höhenschätzung erwartet werden. Schließlich bedeutet eine 3D-Rekonstruktion der Maispflanzen mit einem kostengünstigen Sensor, der auf einer kleinen elektrischen, motorbetriebenen Roboterplattform montiert ist, dass die Kosten (entweder wirtschaftlich, energetisch oder zeitlich) für die Erzeugung jedes Punktes in den Punktwolken im Vergleich zu früheren Untersuchungen stark reduziert werden

Biometric Presentation Attack Detection for Mobile Devices Using Gaze Information

Facial recognition systems are among the most widely deployed in biometric applications. However, such systems are vulnerable to presentation attacks (spoofing), where a person tries to disguise as someone else by mimicking their biometric data and thereby gaining access to the system. Significant research attention has been directed toward developing robust strategies for detecting such attacks and thus assuring the security of these systems in real-world applications. This thesis is focused on presentation attack detection for face recognition systems using a gaze tracking approach. The proposed challenge-response presentation attack detection system assesses the gaze of the user in response to a randomly moving stimulus on the screen. The user is required to track the moving stimulus with their gaze with natural head/eye movements. If the response is adequately similar to the challenge, the access attempt is seen as genuine. The attack scenarios considered in this work included the use of hand held displayed photos, 2D masks, and 3D masks. Due to the nature of the proposed challenge-response approaches for presentation attack detection, none of the existing public databases were appropriate and a new database has been collected. The Kent Gaze Dynamics Database (KGDD) consists of 2,400 sets of genuine and attack-based presentation attempts collected from 80 participants. The use of a mobile device were simulated on a desktop PC for two possible geometries corresponding to mobile phone and tablet devices. Three different types of challenge trajectories were used in this data collection exercise. A number of novel gaze-based features were explored to develop the presentation attack detection algorithm. Initial experiments using the KGDD provided an encouraging indication of the potential of the proposed system for attack detection. In order to explore the feasibility of the scheme on a real hand held device, another database, the Mobile KGDD (MKGDD), was collected from 30 participants using a single mobile device (Google Nexus 6), to test the proposed features. Comprehensive experimental analysis has been performed on the two collected databases for each of the proposed features. Performance evaluation results indicate that the proposed gaze-based features are effective in discriminating between genuine and presentation attack attempts

Conception d'un dispositif d'acquisition d'images agronomiques 3D en extérieur et développement des traitements associés pour la détection et la reconnaissance de plantes et de maladies

Dans le cadre de l'acquisition de l'information de profondeur de scènes texturées, un processus d'estimation de la profondeur basé sur la méthode de reconstruction 3D Shape from Focus est présenté dans ce manuscrit. Les deux étapes fondamentales de cette approche sont l'acquisition de la séquence d'images de la scène par sectionnement optique et l'évaluation de la netteté locale pour chaque pixel des images acquises. Deux systèmes d'acquisition de cette séquence d'images sont présentés ainsi que les traitements permettant d'exploiter celle-ci pour la suite du processus d'estimation de la profondeur. L'étape d'évaluation de la netteté des pixels passe par la comparaison des différents opérateurs de mesure de netteté. En plus des opérateurs usuels, deux nouveaux opérateurs basés sur les descripteurs généralisés de Fourier sont proposés. Une méthode nouvelle et originale de comparaison est développée et permet une analyse approfondie de la robustesse à différents paramètres des divers opérateurs. Afin de proposer une automatisation du processus de reconstruction, deux méthodes d'évaluation automatique de la netteté sont détaillées. Finalement, le processus complet de reconstruction est appliqué à des scènes agronomiques, mais également à une problématique du domaine de l'analyse de défaillances de circuits intégrés afin d'élargir les domaines d'utilisationIn the context of the acquisition of depth information for textured scenes, a depth estimation process based on a 3D reconstruction method called "shape from focus" is proposed in this thesis. The two crucial steps of this approach are the image sequence acquisition of the scene by optical sectioning and the local sharpness evaluation for each pixel of the acquired images. Two acquisition systems have been developed and are presented as well as different image processing techniques that enable the image exploitation for the depth estimation process. The pixel sharpness evaluation requires comparison of different focus measure operators in order to determine the most appropriate ones. In addition to the usual focus measure operators, two news operators based on generalized Fourier descriptors are presented. A new and original comparison method is developped and provides a further analysis of the robustness to various parameters of the focus measure operators. In order to provide an automatic version of the reconstruction process, two automatic sharpness evaluation methods are detailed. Finally, the whole reconstruction process is applied to agronomic scenes, but also to a problematic in failure analysis domain aiming to expand to other applicationsDIJON-BU Doc.électronique (212319901) / SudocSudocFranceF