ARIGAN: Synthetic Arabidopsis Plants using Generative Adversarial Network

In recent years, there has been an increasing interest in image-based plant phenotyping, applying state-of-the-art machine learning approaches to tackle challenging problems, such as leaf segmentation (a multi-instance problem) and counting. Most of these algorithms need labelled data to learn a model for the task at hand. Despite the recent release of a few plant phenotyping datasets, large annotated plant image datasets for the purpose of training deep learning algorithms are lacking. One common approach to alleviate the lack of training data is dataset augmentation. Herein, we propose an alternative solution to dataset augmentation for plant phenotyping, creating artificial images of plants using generative neural networks. We propose the Arabidopsis Rosette Image Generator (through) Adversarial Network: a deep convolutional network that is able to generate synthetic rosette-shaped plants, inspired by DCGAN (a recent adversarial network model using convolutional layers). Specifically, we trained the network using A1, A2, and A4 of the CVPPP 2017 LCC dataset, containing Arabidopsis Thaliana plants. We show that our model is able to generate realistic 128x128 colour images of plants. We train our network conditioning on leaf count, such that it is possible to generate plants with a given number of leaves suitable, among others, for training regression based models. We propose a new Ax dataset of artificial plants images, obtained by our ARIGAN. We evaluate this new dataset using a state-of-the-art leaf counting algorithm, showing that the testing error is reduced when Ax is used as part of the training data.Comment: 8 pages, 6 figures, 1 table, ICCV CVPPP Workshop 201

Leveraging Image Analysis for High-Throughput Plant Phenotyping

The complex interaction between a genotype and its environment controls the biophysical properties of a plant, manifested in observable traits, i.e., plant’s phenome, which influences resources acquisition, performance, and yield. High-throughput automated image-based plant phenotyping refers to the sensing and quantifying plant traits non-destructively by analyzing images captured at regular intervals and with precision. While phenomic research has drawn significant attention in the last decade, extracting meaningful and reliable numerical phenotypes from plant images especially by considering its individual components, e.g., leaves, stem, fruit, and flower, remains a critical bottleneck to the translation of advances of phenotyping technology into genetic insights due to various challenges including lighting variations, plant rotations, and self-occlusions. The paper provides (1) a framework for plant phenotyping in a multimodal, multi-view, time-lapsed, high-throughput imaging system; (2) a taxonomy of phenotypes that may be derived by image analysis for better understanding of morphological structure and functional processes in plants; (3) a brief discussion on publicly available datasets to encourage algorithm development and uniform comparison with the state-of-the-art methods; (4) an overview of the state-of-the-art image-based high-throughput plant phenotyping methods; and (5) open problems for the advancement of this research field

Watching plants grow:A position paper on computer vision and Arabidopsis thaliana

The authors present a comprehensive overview of image processing and analysis work done to support research into the model flowering plant Arabidopsis thaliana. Beside the plant's importance in biological research, using image analysis to obtain experimental measurements of it is an interesting vision problem in its own right, involving the segmentation and analysis of sequences of images of objects whose shape varies between individual specimens and also changes over time. While useful measurements can be obtained by segmenting a whole plant from the background, they suggest that the increased range and precision of measurements made available by leaf‐level segmentation makes this a problem well worth solving. A variety of approaches have been tried by biologists as well as computer vision researchers. This is an interdisciplinary area and the computer vision community has an important contribution to make. They suggest that there is a need for publicly available datasets with ground truth annotations to enable the evaluation of new approaches and to support the building of training data for modern data‐driven computer vision approaches, which are those most likely to result in the kind of fully automated systems that will be of use to biologists

OSC-CO2: coattention and cosegmentation framework for plant state change with multiple features

Cosegmentation and coattention are extensions of traditional segmentation methods aimed at detecting a common object (or objects) in a group of images. Current cosegmentation and coattention methods are ineffective for objects, such as plants, that change their morphological state while being captured in different modalities and views. The Object State Change using Coattention-Cosegmentation (OSC-CO2) is an end-to-end unsupervised deep-learning framework that enhances traditional segmentation techniques, processing, analyzing, selecting, and combining suitable segmentation results that may contain most of our target object’s pixels, and then displaying a final segmented image. The framework leverages coattention-based convolutional neural networks (CNNs) and cosegmentation-based dense Conditional Random Fields (CRFs) to address segmentation accuracy in high-dimensional plant imagery with evolving plant objects. The efficacy of OSC-CO2 is demonstrated using plant growth sequences imaged with infrared, visible, and fluorescence cameras in multiple views using a remote sensing, high-throughput phenotyping platform, and is evaluated using Jaccard index and precision measures. We also introduce CosegPP+, a dataset that is structured and can provide quantitative information on the efficacy of our framework. Results show that OSC-CO2 out performed state-of-the art segmentation and cosegmentation methods by improving segementation accuracy by 3% to 45%

Pheno-Deep Counter: a unified and versatile deep learning architecture for leaf counting

Direct observation of morphological plant traits is tedious and a bottleneck for high‐throughput phenotyping. Hence, interest in image‐based analysis is increasing, with the requirement for software that can reliably extract plant traits, such as leaf count, preferably across a variety of species and growth conditions. However, current leaf counting methods do not work across species or conditions and therefore may lack broad utility. In this paper, we present Pheno‐Deep Counter, a single deep network that can predict leaf count in two‐dimensional (2D) plant images of different species with a rosette‐shaped appearance. We demonstrate that our architecture can count leaves from multi‐modal 2D images, such as visible light, fluorescence and near‐infrared. Our network design is flexible, allowing for inputs to be added or removed to accommodate new modalities. Furthermore, our architecture can be used as is without requiring dataset‐specific customization of the internal structure of the network, opening its use to new scenarios. Pheno‐Deep Counter is able to produce accurate predictions in many plant species and, once trained, can count leaves in a few seconds. Through our universal and open source approach to deep counting we aim to broaden utilization of machine learning‐based approaches to leaf counting. Our implementation can be downloaded at https://bitbucket.org/tuttoweb/pheno-deep-counter

Leveraging Image Analysis for High-Throughput Plant Phenotyping

The complex interaction between a genotype and its environment controls the biophysical properties of a plant, manifested in observable traits, i.e., plant's phenome, which influences resources acquisition, performance, and yield. High-throughput automated image-based plant phenotyping refers to the sensing and quantifying plant traits non-destructively by analyzing images captured at regular intervals and with precision. While phenomic research has drawn significant attention in the last decade, extracting meaningful and reliable numerical phenotypes from plant images especially by considering its individual components, e.g., leaves, stem, fruit, and flower, remains a critical bottleneck to the translation of advances of phenotyping technology into genetic insights due to various challenges including lighting variations, plant rotations, and self-occlusions. The paper provides (1) a framework for plant phenotyping in a multimodal, multi-view, time-lapsed, high-throughput imaging system; (2) a taxonomy of phenotypes that may be derived by image analysis for better understanding of morphological structure and functional processes in plants; (3) a brief discussion on publicly available datasets to encourage algorithm development and uniform comparison with the state-of-the-art methods; (4) an overview of the state-of-the-art image-based high-throughput plant phenotyping methods; and (5) open problems for the advancement of this research field

High resolution imaging of maize (Zea mays) leaf temperature in the field: the key role of the regions of interest

Abstract. The use of remote sensors (thermometers and cameras) to analyse crop water status in field conditions is fraught with several difficulties. In particular, average canopy temperature measurements are affected by the mixture of soil and green regions, the mutual shading of leaves and the variability of absorbed radiation. The aim of the study was to analyse how the selection of different 'regions of interest' (ROI) in canopy images affect the variability of the resulting temperature averages. Using automated image segmentation techniques we computed the average temperature in four nested ROI of decreasing size, from the whole image down to the sunlit fraction of a leaf located in the upper part of the canopy. The study was conducted on maize (Zea mays L.) at the flowering stage, for its large leaves and well structured canopy. Our results suggest that, under these conditions, the ROI comprising the sunlit fraction of a leaf located in the upper part of the canopy should be analogous to the single leaf approach (in controlled conditions) that allows the estimation of stomatal conductance or plant water potential