Measuring and Modeling Apple Trees Using Time-of-Flight Data for Automation of Dormant Pruning Applications

Dormant pruning is one of the most expensive, labor-intensive, but, unavoidable procedure in the field of horticulture to ensure quality crop production. During winter, skilled farmers remove certain branches that are connected directly with the trunk of a tree carefully using a set of predefined rules. In order to reduce this dependence on a large manpower, our goal is to automate this pruning process by building 3D models of dormant apple trees, which eventually would be fed to an intelligent robotic system. In this paper, we present a semicircle fitting based robust 3D reconstruction scheme for modeling the trunk and primary branches of apple trees. The method involves estimating the diameter-error, creating semicircle fit model of the tree from a single depth image, and reconstructing the final 3D model of the tree by aligning a sequence of depth images. Analysis of the qualitative as well as the quantitative evaluations of our algorithm on five different dormant apple trees from our dataset under various indoor and outdoor environments demonstrate the effectiveness of the proposed framework for automatic 3D reconstruction. The results show that on an average, the proposed schemes provide a performance of 89.4% for correctly estimating the diameters of the primary branches with a tolerance of 5 mm and 100%c for correctly identifying the branches

Computer Vision Problems in 3D Plant Phenotyping

In recent years, there has been significant progress in Computer Vision based plant phenotyping (quantitative analysis of biological properties of plants) technologies. Traditional methods of plant phenotyping are destructive, manual and error prone. Due to non-invasiveness and non-contact properties as well as increased accuracy, imaging techniques are becoming state-of-the-art in plant phenotyping. Among several parameters of plant phenotyping, growth analysis is very important for biological inference. Automating the growth analysis can result in accelerating the throughput in crop production. This thesis contributes to the automation of plant growth analysis. First, we present a novel system for automated and non-invasive/non-contact plant growth measurement. We exploit the recent advancements of sophisticated robotic technologies and near infrared laser scanners to build a 3D imaging system and use state-of-the-art Computer Vision algorithms to fully automate growth measurement. We have set up a gantry robot system having 7 degrees of freedom hanging from the roof of a growth chamber. The payload is a range scanner, which can measure dense depth maps (raw 3D coordinate points in mm) on the surface of an object (the plant). The scanner can be moved around the plant to scan from different viewpoints by programming the robot with a specific trajectory. The sequence of overlapping images can be aligned to obtain a full 3D structure of the plant in raw point cloud format, which can be triangulated to obtain a smooth surface (triangular mesh), enclosing the original plant. We show the capability of the system to capture the well known diurnal pattern of plant growth computed from the surface area and volume of the plant meshes for a number of plant species. Second, we propose a technique to detect branch junctions in plant point cloud data. We demonstrate that using these junctions as feature points, the correspondence estimation can be formulated as a subgraph matching problem, and better matching results than state-of-the-art can be achieved. Also, this idea removes the requirement of a priori knowledge about rotational angles between adjacent scanning viewpoints imposed by the original registration algorithm for complex plant data. Before, this angle information had to be approximately known. Third, we present an algorithm to classify partially occluded leaves by their contours. In general, partial contour matching is a NP-hard problem. We propose a suboptimal matching solution and show that our method outperforms state-of-the-art on 3 public leaf datasets. We anticipate using this algorithm to track growing segmented leaves in our plant range data, even when a leaf becomes partially occluded by other plant matter over time. Finally, we perform some experiments to demonstrate the capability and limitations of the system and highlight the future research directions for Computer Vision based plant phenotyping