Structured Light-Based 3D Reconstruction System for Plants.

Camera-based 3D reconstruction of physical objects is one of the most popular computer vision trends in recent years. Many systems have been built to model different real-world subjects, but there is lack of a completely robust system for plants. This paper presents a full 3D reconstruction system that incorporates both hardware structures (including the proposed structured light system to enhance textures on object surfaces) and software algorithms (including the proposed 3D point cloud registration and plant feature measurement). This paper demonstrates the ability to produce 3D models of whole plants created from multiple pairs of stereo images taken at different viewing angles, without the need to destructively cut away any parts of a plant. The ability to accurately predict phenotyping features, such as the number of leaves, plant height, leaf size and internode distances, is also demonstrated. Experimental results show that, for plants having a range of leaf sizes and a distance between leaves appropriate for the hardware design, the algorithms successfully predict phenotyping features in the target crops, with a recall of 0.97 and a precision of 0.89 for leaf detection and less than a 13-mm error for plant size, leaf size and internode distance

3D Perception-based Collision-Free Robotic Leaf Probing for Automated Indoor Plant Phenotyping

Various instrumentation devices for plant physiology study such as spectrometer, chlorophyll fluorimeter, and Raman spectroscopy sensor require accurate placement of their sensor probes toward the leaf surface to meet specific requirements of probe-to-target distance and orientation. In this work, a Kinect V2 sensor, a high-precision 2D laser profilometer, and a six-axis robotic manipulator were used to automate the leaf probing task. The relatively wide field of view and high resolution of Kinect V2 allowed rapid capture of the full 3D environment in front of the robot. The location and size of each plant were estimated by k-means clustering where “k” was the user-defined number of plants. A real-time collision-free motion planning framework based on Probabilistic Roadmaps was adapted to maneuver the robotic manipulator without colliding with the plants. Each plant was scanned from the top with the short-range profilometer to obtain high-precision 3D point cloud data. Potential leaf clusters were extracted by a 3D region growing segmentation scheme. Each leaf segment was further partitioned into small patches by a Voxel Cloud Connectivity Segmentation method. Only the patches with low root mean square errors of plane fitting were used to compute leaf probing poses of the robot. Experiments conducted inside a growth chamber mock-up showed that the developed robotic leaf probing system achieved an average motion planning time of 0.4 seconds with an average end-effector travel distance of 1.0 meter. To examine the probing accuracy, a square surface was scanned at different angles, and its centroid was probed perpendicularly. The average absolute probing errors of distance and angle were 1.5 mm and 0.84 degrees, respectively. These results demonstrate the utility of the proposed robotic leaf probing system for automated non-contact deployment of spectroscopic sensor probes for indoor plant phenotyping under controlled environmental conditions

Automatic morphological trait characterization for corn plants via 3D holographic reconstruction

Plant breeding is an extremely important route to genetic improvements that can increase yield and plant adaptability. Genetic improvement requires careful measurement of plant phenotypes or plant trait characteristics, but phenotype measurement is a tedious and error-prone task for humans to perform. High-throughput phenotyping aims to eliminate the problems of manual phenotype measurement. In this paper, we propose and demonstrate the efficacy of an automatic corn plant phenotyping system based on 3D holographic reconstruction. Point cloud image data were acquired from a time-of-flight 3D camera, which was integrated with a plant rotating table to form a screening station. Our method has five main steps: point cloud data filtering and merging, stem segmentation, leaf segmentation, phenotypic data extraction, and 3D holographic visualization. In an experimental study with five corn plants at their early growth stage (V3), we obtained promising results with accurate 3D holographic reconstruction. The average measurement error rate for stem major axis, stem minor axis, stem height, leaf area, leaf length and leaf angle were at 7.92%, 15.20%, 7.45%, 21.89%, 10.25% and 11.09%, respectively. The most challenging trait to measure was leaf area due to partial occlusions and rolling of some leaves. In future work, we plan to extend and evaluate the usability of the system in an industrial plant breeding setting

Robotic 3D Plant Perception and Leaf Probing with Collision-Free Motion Planning for Automated Indoor Plant Phenotyping

Various instrumentation devices for plant physiology study such as chlorophyll fluorimeter and Raman spectrometer require leaf probing with accurate probe positioning and orientation with respect to leaf surface. In this work, we aimed to automate this process with a Kinect V2 sensor, a high-precision 2D laser profilometer, and a 6-axis robotic manipulator in a high-throughput manner. The relatively wide field of view and high resolution of Kinect V2 allowed rapid capture of the full 3D environment in front of the robot. Given the number of plants, the location and size of each plant were estimated by K-means clustering. A real-time collision-free motion planning framework based on Probabilistic Roadmap was adopted to maneuver the robotic manipulator without colliding with the plants. Each plant was scanned from top with the short-range profilometer to obtain a high-precision point cloud where potential leaf clusters were extracted by region growing segmentation. Each leaf segment was further partitioned into small patches by Voxel Cloud Connectivity Segmentation. Only the small patches with low root mean square values of plane fitting were used to compute probing poses. To evaluate probing accuracy, a square surface was scanned at various angles and its centroid was probed perpendicularly with a probing position error of 1.5 mm and a probing angle error of 0.84 degrees on average. Our growth chamber leaf probing experiment showed that the average motion planning time was 0.4 seconds and the average traveled distance of tool center point was 1 meter

Individual tree point clouds and tree measurements from multi-platform laser scanning in German forests

Laser scanning from different acquisition platforms enables the collection of 3D point clouds from different perspectives and with varying resolutions. These point clouds allow us to retrieve detailed information on the individual tree and forest structure. We conducted airborne laser scanning (ALS), uncrewed aerial vehicle (UAV)-borne laser scanning (ULS) and terrestrial laser scanning (TLS) in two German mixed forests with species typical of central Europe. We provide the spatially overlapping, georeferenced point clouds for 12 forest plots. As a result of individual tree extraction, we furthermore present a comprehensive database of tree point clouds and corresponding tree metrics. Tree metrics were derived from the point clouds and, for half of the plots, also measured in the field. Our dataset may be used for the creation of 3D tree models for radiative transfer modeling or lidar simulation studies or to fit allometric equations between point cloud metrics and forest inventory variables. It can further serve as a benchmark dataset for different algorithms and machine learning tasks, in particular automated individual tree segmentation, tree species classification or forest inventory metric prediction. The dataset and supplementary metadata are available for download, hosted by the PANGAEA data publisher at https://doi.org/10.1594/PANGAEA.942856 (Weiser et al., 2022a)

Low-Cost Three-Dimensional Modeling of Crop Plants

Plant modeling can provide a more detailed overview regarding the basis of plant development throughout the life cycle. Three-dimensional processing algorithms are rapidly expanding in plant phenotyping programmes and in decision-making for agronomic management. Several methods have already been tested, but for practical implementations the trade-off between equipment cost, computational resources needed and the fidelity and accuracy in the reconstruction of the end-details needs to be assessed and quantified. This study examined the suitability of two low-cost systems for plant reconstruction. A low-cost Structure from Motion (SfM) technique was used to create 3D models for plant crop reconstruction. In the second method, an acquisition and reconstruction algorithm using an RGB-Depth Kinect v2 sensor was tested following a similar image acquisition procedure. The information was processed to create a dense point cloud, which allowed the creation of a 3D-polygon mesh representing every scanned plant. The selected crop plants corresponded to three different crops (maize, sugar beet and sunflower) that have structural and biological differences. The parameters measured from the model were validated with ground truth data of plant height, leaf area index and plant dry biomass using regression methods. The results showed strong consistency with good correlations between the calculated values in the models and the ground truth information. Although, the values obtained were always accurately estimated, differences between the methods and among the crops were found. The SfM method showed a slightly better result with regard to the reconstruction the end-details and the accuracy of the height estimation. Although the use of the processing algorithm is relatively fast, the use of RGB-D information is faster during the creation of the 3D models. Thus, both methods demonstrated robust results and provided great potential for use in both for indoor and outdoor scenarios. Consequently, these low-cost systems for 3D modeling are suitable for several situations where there is a need for model generation and also provide a favourable time-cost relationship

Insights into tree morphology and canopy space occupation under the influence of local neighbourhood interactions in mature temperate forests using laser scanning technology

Mounting evidence suggests that tree species richness promotes ecosystem functioning in forests. However, the mechanisms driving positive biodiversity ecosystem functioning relationships remain largely unclear. This also holds for the previously proposed key mechanisms of resource partitioning in canopy space. Until recently, surveying and hence the study of crown space was very time-consuming and the images low resolution. The application of high-resolution laser scanning, however, now enables a fast and precise recording of entire forests. This thesis presents how the abandonment of management strongly alters the individual tree structure from the wood distribution along the trunk to the crown, a tree species-rich neighbourhood can increase the wood volume and crown dimension of individual trees as well as the productivity of large-sized trees, mobile laser scanning in forests is suitable for the acquisition of high-quality point clouds and determination of relevant management parameters, and the direction and strength of the relationship between tree species richness and canopy occupation depends on the definition of both canopy and species richness. These results reinforce the influence of species richness on ecosystem functions in oldgrowth forests and underline the importance of laser scanning for forest ecology research. The findings of the comparative analyses further highlight the importance of underlying definitions for the results obtained