Leveraging Image Analysis to Compute 3D Plant Phenotypes Based on Voxel-Grid Plant Reconstruction

High throughput image-based plant phenotyping facilitates the extraction of morphological and biophysical traits of a large number of plants non-invasively in a relatively short time. It facilitates the computation of advanced phenotypes by considering the plant as a single object (holistic phenotypes) or its components, i.e., leaves and the stem (component phenotypes). The architectural complexity of plants increases over time due to variations in self-occlusions and phyllotaxy, i.e., arrangements of leaves around the stem. One of the central challenges to computing phenotypes from 2-dimensional (2D) single view images of plants, especially at the advanced vegetative stage in presence of self-occluding leaves, is that the information captured in 2D images is incomplete, and hence, the computed phenotypes are inaccurate. We introduce a novel algorithm to compute 3-dimensional (3D) plant phenotypes from multiview images using voxel-grid reconstruction of the plant (3DPhenoMV). The paper also presents a novel method to reliably detect and separate the individual leaves and the stem from the 3D voxel-grid of the plant using voxel overlapping consistency check and point cloud clustering techniques. To evaluate the performance of the proposed algorithm, we introduce the University of Nebraska-Lincoln 3D Plant Phenotyping Dataset (UNL-3DPPD). A generic taxonomy of 3D image-based plant phenotypes are also presented to promote 3D plant phenotyping research. A subset of these phenotypes are computed using computer vision algorithms with discussion of their significance in the context of plant science. The central contributions of the paper are (a) an algorithm for 3D voxel-grid reconstruction of maize plants at the advanced vegetative stages using images from multiple 2D views; (b) a generic taxonomy of 3D image-based plant phenotypes and a public benchmark dataset, i.e., UNL-3DPPD, to promote the development of 3D image-based plant phenotyping research; and (c) novel voxel overlapping consistency check and point cloud clustering techniques to detect and isolate individual leaves and stem of the maize plants to compute the component phenotypes. Detailed experimental analyses demonstrate the efficacy of the proposed method, and also show the potential of 3D phenotypes to explain the morphological characteristics of plants regulated by genetic and environmental interactions

Phenomenal: a software framework for model-assisted analysis of high throughput plant phenotyping data

International audiencePlant high-throughput phenotyping aims at capturing the genetic variability of plant response to environmental factors for thousands of plants, hence identifying heritable traits for genomic selection and predicting the genetic values of allelic combinations in different environment. This first implies the automation of the measurement of a large number of traits to characterize plant growth, plant development and plant functioning. It also requires a fluent and versatile interaction between data and continuously evolving plant response models, that are essential in the analysis of the marker x environment interaction and in the integration of processes for predicting crop performance [1]. In the frame of the Phenome high throughput phenotyping infrastructure, we develop Phenomenal: a software framework dedicated to the analysis of high throughput phenotyping data and models. It is based on the OpenAlea platform [2] that provides methods and softwares for the modelling of plants, together with a user-friendly interface for the design and execution of scientific workflows. OpenAlea is also part of the InfraPhenoGrid infrastructure that allows high throughput computation and recording of provenance during the execution [3]. Figure 1: The 3D plant reconstruction and segmentation pipeline. Muti-view plants images from PhenoArch are binarised and used to reconstruct plants in3D. The 3D skeleton is extracted and separated into stem (central vertical elements) and leaves. 3D voxels are segmented by propagating skeleton segmentation

Automatic morphological trait characterization for corn plants via 3D holographic reconstruction

Plant breeding is an extremely important route to genetic improvements that can increase yield and plant adaptability. Genetic improvement requires careful measurement of plant phenotypes or plant trait characteristics, but phenotype measurement is a tedious and error-prone task for humans to perform. High-throughput phenotyping aims to eliminate the problems of manual phenotype measurement. In this paper, we propose and demonstrate the efficacy of an automatic corn plant phenotyping system based on 3D holographic reconstruction. Point cloud image data were acquired from a time-of-flight 3D camera, which was integrated with a plant rotating table to form a screening station. Our method has five main steps: point cloud data filtering and merging, stem segmentation, leaf segmentation, phenotypic data extraction, and 3D holographic visualization. In an experimental study with five corn plants at their early growth stage (V3), we obtained promising results with accurate 3D holographic reconstruction. The average measurement error rate for stem major axis, stem minor axis, stem height, leaf area, leaf length and leaf angle were at 7.92%, 15.20%, 7.45%, 21.89%, 10.25% and 11.09%, respectively. The most challenging trait to measure was leaf area due to partial occlusions and rolling of some leaves. In future work, we plan to extend and evaluate the usability of the system in an industrial plant breeding setting

Structured Light-Based 3D Reconstruction System for Plants.

Camera-based 3D reconstruction of physical objects is one of the most popular computer vision trends in recent years. Many systems have been built to model different real-world subjects, but there is lack of a completely robust system for plants. This paper presents a full 3D reconstruction system that incorporates both hardware structures (including the proposed structured light system to enhance textures on object surfaces) and software algorithms (including the proposed 3D point cloud registration and plant feature measurement). This paper demonstrates the ability to produce 3D models of whole plants created from multiple pairs of stereo images taken at different viewing angles, without the need to destructively cut away any parts of a plant. The ability to accurately predict phenotyping features, such as the number of leaves, plant height, leaf size and internode distances, is also demonstrated. Experimental results show that, for plants having a range of leaf sizes and a distance between leaves appropriate for the hardware design, the algorithms successfully predict phenotyping features in the target crops, with a recall of 0.97 and a precision of 0.89 for leaf detection and less than a 13-mm error for plant size, leaf size and internode distance

A model-based approach to recovering the structure of a plant from images

We present a method for recovering the structure of a plant directly from a small set of widely-spaced images. Structure recovery is more complex than shape estimation, but the resulting structure estimate is more closely related to phenotype than is a 3D geometric model. The method we propose is applicable to a wide variety of plants, but is demonstrated on wheat. Wheat is made up of thin elements with few identifiable features, making it difficult to analyse using standard feature matching techniques. Our method instead analyses the structure of plants using only their silhouettes. We employ a generate-and-test method, using a database of manually modelled leaves and a model for their composition to synthesise plausible plant structures which are evaluated against the images. The method is capable of efficiently recovering accurate estimates of plant structure in a wide variety of imaging scenarios, with no manual intervention