Plants Detection, Localization and Discrimination using 3D Machine Vision for Robotic Intra-row Weed Control

Weed management is vitally important in crop production systems. However, conventional herbicide-based weed control can lead to negative environmental impacts. Manual weed control is laborious and impractical for large scale production. Robotic weeding offers a possibility of controlling weeds precisely, particularly for weeds growing close to or within crop rows. The fusion of two-dimensional textural images and three-dimensional spatial images to recognize and localize crop plants at different growth stages were investigated. Images of different crop plants at different growth stages with weeds were acquired. Feature extraction algorithms were developed, and different features were extracted and used to train plant and background classifiers, which also addressed the problems of canopy occlusion and leaf damage. Then, the efficacy and accuracy of the proposed methods in classification were demonstrated by experiments. Currently, the algorithms were only developed and tested for broccoli and lettuce. For broccoli plants, the crop plants detection true positive rate was 93.1%, and the false discover rate was 1.1%, with the average crop-plant-localization error of 15.9 mm. For lettuce plants, the crop plants detection true positive rate was 92.3%, and the false discover rate was 4.0%, with the average crop-plant-localization error of 8.5 mm. The results have shown that 3D imaging based plant recognition algorithms are effective and reliable for crop/weed differentiation

Pushing the envelope for estimating poses and actions via full 3D reconstruction

Estimating poses and actions of human bodies and hands is an important task in the computer vision community due to its vast applications, including human computer interaction, virtual reality and augmented reality, medical image analysis. Challenges: There are many in-the-wild challenges in this task (see chapter 1). Among them, in this thesis, we focused on two challenges which could be relieved by incorporating the 3D geometry: (1) inherent 2D-to-3D ambiguity driven by the non-linear 2D projection process when capturing 3D objects. (2) lack of sufficient and quality annotated datasets due to the high-dimensionality of subjects' attribute space and inherent difficulty in annotating 3D coordinate values. Contributions: We first tried to jointly tackle the 2D-to-3D ambiguity and insufficient data issues by (1) explicitly reconstructing 2.5D and 3D samples and use them as new training data to train a pose estimator. Next, we tried to (2) encode 3D geometry in the training process of the action recognizer to reduce the 2D-to-3D ambiguity. In appendix, we proposed a (3) new hand pose synthetic dataset that can be used for more complete attribute changes and multi-modal experiments in the future. Experiments: Throughout experiments, we found interesting facts: (1) 2.5D depth map reconstruction and data augmentation can improve the accuracy of the depth-based hand pose estimation algorithm, (2) 3D mesh reconstruction can be used to generate a new RGB data and it improves the accuracy of RGB-based dense hand pose estimation algorithm, (3) 3D geometry from 3D poses and scene layouts could be successfully utilized to reduce the 2D-to-3D ambiguity in the action recognition problem.Open Acces

Capturing Hand-Object Interaction and Reconstruction of Manipulated Objects

Hand motion capture with an RGB-D sensor gained recently a lot of research attention, however, even most recent approaches focus on the case of a single isolated hand. We focus instead on hands that interact with other hands or with a rigid or articulated object. Our framework successfully captures motion in such scenarios by combining a generative model with discriminatively trained salient points, collision detection and physics simulation to achieve a low tracking error with physically plausible poses. All components are unified in a single objective function that can be optimized with standard optimization techniques. We initially assume a-priori knowledge of the object’s shape and skeleton. In case of unknown object shape there are existing 3d reconstruction methods that capitalize on distinctive geometric or texture features. These methods though fail for textureless and highly symmetric objects like household articles, mechanical parts or toys. We show that extracting 3d hand motion for in-hand scanning e↵ectively facilitates the reconstruction of such objects and we fuse the rich additional information of hands into a 3d reconstruction pipeline. Finally, although shape reconstruction is enough for rigid objects, there is a lack of tools that build rigged models of articulated objects that deform realistically using RGB-D data. We propose a method that creates a fully rigged model consisting of a watertight mesh, embedded skeleton and skinning weights by employing a combination of deformable mesh tracking, motion segmentation based on spectral clustering and skeletonization based on mean curvature flow

Proceedings of the Third Annual Symposium on Mathematical Pattern Recognition and Image Analysis

Topics addressed include: multivariate spline method; normal mixture analysis applied to remote sensing; image data analysis; classifications in spatially correlated environments; probability density functions; graphical nonparametric methods; subpixel registration analysis; hypothesis integration in image understanding systems; rectification of satellite scanner imagery; spatial variation in remotely sensed images; smooth multidimensional interpolation; and optimal frequency domain textural edge detection filters

Décomposition volumique d'images pour l'étude de la microstructure de la neige

Les avalanches de neige sont des phénomènes naturels complexes dont l'occurrence s'explique principalement par la structure et les propriétés du manteau neigeux. Afin de mieux comprendre les évolutions de ces propriétés au cours du temps, il est important de pouvoir caractériser la microstructure de la neige, notamment en termes de grains et de ponts de glace les reliant. Dans ce contexte, l'objectif de cette thèse est la décomposition d'échantillons de neige en grains individuels à partir d'images 3-D de neige obtenues par microtomographie X. Nous présentons ici deux méthodes de décomposition utilisant des algorithmes de géométrie discrète. Sur la base des résultats de ces segmentations, certains paramètres, comme la surface spécifique et la surface spécifique de contact entre grains sont ensuite estimés sur des échantillons de neiges variées. Ces méthodes de segmentation ouvrent de nouvelles perspectives pour la caractérisation de la microstructure de la neige, de ses propriétés, ainsi que de leur évolution au cours du temps.Snow avalanches are complex natural phenomena whose occurrence is mainly due to the structure and properties of the snowpack. To better understand the evolution of these properties over time, it is important to characterize the microstructure of snow, especially in terms of grains and ice necks that connect them. In this context, the objective of this thesis is the decomposition of snow samples into individual grains from 3-D images of snow obtained by X-ray microtomography. We present two decomposition methods using algorithms of discrete geometry. Based on the results of these segmentations, some parameters such as the specific surface area and the specific contact area between grains are then estimated from samples of several snow types. These segmentation methods offer new outlooks for the characterization of the microstructure of snow, its properties, and its time evolution