Towards infield, live plant phenotyping using a reduced-parameter CNN

© 2019, The Author(s). There is an increase in consumption of agricultural produce as a result of the rapidly growing human population, particularly in developing nations. This has triggered high-quality plant phenotyping research to help with the breeding of high-yielding plants that can adapt to our continuously changing climate. Novel, low-cost, fully automated plant phenotyping systems, capable of infield deployment, are required to help identify quantitative plant phenotypes. The identification of quantitative plant phenotypes is a key challenge which relies heavily on the precise segmentation of plant images. Recently, the plant phenotyping community has started to use very deep convolutional neural networks (CNNs) to help tackle this fundamental problem. However, these very deep CNNs rely on some millions of model parameters and generate very large weight matrices, thus making them difficult to deploy infield on low-cost, resource-limited devices. We explore how to compress existing very deep CNNs for plant image segmentation, thus making them easily deployable infield and on mobile devices. In particular, we focus on applying these models to the pixel-wise segmentation of plants into multiple classes including background, a challenging problem in the plant phenotyping community. We combined two approaches (separable convolutions and SVD) to reduce model parameter numbers and weight matrices of these very deep CNN-based models. Using our combined method (separable convolution and SVD) reduced the weight matrix by up to 95% without affecting pixel-wise accuracy. These methods have been evaluated on two public plant datasets and one non-plant dataset to illustrate generality. We have successfully tested our models on a mobile device

Cyber-Agricultural Systems for Crop Breeding and Sustainable Production

The Cyber-Agricultural System (CAS) Represents an overarching Framework of Agriculture that Leverages Recent Advances in Ubiquitous Sensing, Artificial Intelligence, Smart Actuators, and Scalable Cyberinfrastructure (CI) in Both Breeding and Production Agriculture. We Discuss the Recent Progress and Perspective of the Three Fundamental Components of CAS – Sensing, Modeling, and Actuation – and the Emerging Concept of Agricultural Digital Twins (DTs). We Also Discuss How Scalable CI is Becoming a Key Enabler of Smart Agriculture. in This Review We Shed Light on the Significance of CAS in Revolutionizing Crop Breeding and Production by Enhancing Efficiency, Productivity, Sustainability, and Resilience to Changing Climate. Finally, We Identify Underexplored and Promising Future Directions for CAS Research and Development

Deep learning sensor fusion in plant water stress assessment: A comprehensive review

Water stress is one of the major challenges to food security, causing a significant economic loss for the nation as well for growers. Accurate assessment of water stress will enhance agricultural productivity through optimization of plant water usage, maximizing plant breeding strategies, and preventing forest wildfire for better ecosystem management. Recent advancements in sensor technologies have enabled high-throughput, non-contact, and cost-efficient plant water stress assessment through intelligence system modeling. The advanced deep learning sensor fusion technique has been reported to improve the performance of the machine learning application for processing the collected sensory data. This paper extensively reviews the state-of-the-art methods for plant water stress assessment that utilized the deep learning sensor fusion approach in their application, together with future prospects and challenges of the application domain. Notably, 37 deep learning solutions fell under six main areas, namely soil moisture estimation, soil water modelling, evapotranspiration estimation, evapotranspiration forecasting, plant water status estimation and plant water stress identification. Basically, there are eight deep learning solutions compiled for the 3D-dimensional data and plant varieties challenge, including unbalanced data that occurred due to isohydric plants, and the effect of variations that occur within the same species but cultivated from different locations

Reviewing the essential roles of remote phenotyping, GWAS and explainable AI in practical marker-assisted selection for drought-tolerant winter wheat breeding

Marker-assisted selection (MAS) plays a crucial role in crop breeding improving the speed and precision of conventional breeding programmes by quickly and reliably identifying and selecting plants with desired traits. However, the efficacy of MAS depends on several prerequisites, with precise phenotyping being a key aspect of any plant breeding programme. Recent advancements in high-throughput remote phenotyping, facilitated by unmanned aerial vehicles coupled to machine learning, offer a non-destructive and efficient alternative to traditional, time-consuming, and labour-intensive methods. Furthermore, MAS relies on knowledge of marker-trait associations, commonly obtained through genome-wide association studies (GWAS), to understand complex traits such as drought tolerance, including yield components and phenology. However, GWAS has limitations that artificial intelligence (AI) has been shown to partially overcome. Additionally, AI and its explainable variants, which ensure transparency and interpretability, are increasingly being used as recognised problem-solving tools throughout the breeding process. Given these rapid technological advancements, this review provides an overview of state-of-the-art methods and processes underlying each MAS, from phenotyping, genotyping and association analyses to the integration of explainable AI along the entire workflow. In this context, we specifically address the challenges and importance of breeding winter wheat for greater drought tolerance with stable yields, as regional droughts during critical developmental stages pose a threat to winter wheat production. Finally, we explore the transition from scientific progress to practical implementation and discuss ways to bridge the gap between cutting-edge developments and breeders, expediting MAS-based winter wheat breeding for drought tolerance

Just-in-time Pastureland Trait Estimation for Silage Optimization, under Limited Data Constraints

To ensure that pasture-based farming meets production and environmental targets for a growing population under increasing resource constraints, producers need to know pastureland traits. Current proximal pastureland trait prediction methods largely rely on vegetation indices to determine biomass and moisture content. The development of new techniques relies on the challenging task of collecting labelled pastureland data, leading to small datasets. Classical computer vision has already been applied to weed identification and recognition of fruit blemishes using morphological features, but machine learning algorithms can parameterise models without the provision of explicit features, and deep learning can extract even more abstract knowledge although typically this is assumed to be based around very large datasets. This work hypothesises that through the advantages of state-of-the-art deep learning systems, pastureland crop traits can be accurately assessed in a just-in-time fashion, based on data retrieved from an inexpensive sensor platform, under the constraint of limited amounts of labelled data. However the challenges to achieve this overall goal are great, and for applications such as just-in-time yield and moisture estimation for farm-machinery, this work must bring together systems development, knowledge of good pastureland practice, and also techniques for handling low-volume datasets in a machine learning context. Given these challenges, this thesis makes a number of contributions. The first of these is a comprehensive literature review, relating pastureland traits to ruminant nutrient requirements and exploring trait estimation methods, from contact to remote sensing methods, including details of vegetation indices and the sensors and techniques required to use them. The second major contribution is a high-level specification of a platform for collecting and labelling pastureland data. This includes the collection of four-channel Blue, Green, Red and NIR (VISNIR) images, narrowband data, height and temperature differential, using inexpensive proximal sensors and provides a basis for holistic data analysis. Physical data platforms built around this specification were created to collect and label pastureland data, involving computer scientists, agricultural, mechanical and electronic engineers, and biologists from academia and industry, working with farmers. Using the developed platform and a set of protocols for data collection, a further contribution of this work was the collection of a multi-sensor multimodal dataset for pastureland properties. This was made up of four-channel image data, height data, thermal data, Global Positioning System (GPS) and hyperspectral data, and is available and labelled with biomass (Kg/Ha) and percentage dry matter, ready for use in deep learning. However, the most notable contribution of this work was a systematic investigation of various machine learning methods applied to the collected data in order to maximise model performance under the constraints indicated above. The initial set of models focused on collected hyperspectral datasets. However, due to their relative complexity in real-time deployment, the focus was instead on models that could best leverage image data. The main body of these models centred on image processing methods and, in particular, the use of the so-called Inception Resnet and MobileNet models to predict fresh biomass and percentage dry matter, enhancing performance using data fusion, transfer learning and multi-task learning. Images were subdivided to augment the dataset, using two different patch sizes, resulting in around 10,000 small patches of size 156 x 156 pixels and around 5,000 large patches of size 240 x 240 pixels. Five-fold cross validation was used in all analysis. Prediction accuracy was compared to older mechanisms, albeit using hyperspectral data collected, with no provision made for lighting, humidity or temperature. Hyperspectral labelled data did not produce accurate results when used to calculate Normalized Difference Vegetation Index (NDVI), or to train a neural network (NN), a 1D Convolutional Neural Network (CNN) or Long Short Term Memory (LSTM) models. Potential reasons for this are discussed, including issues around the use of highly sensitive devices in uncontrolled environments. The most accurate prediction came from a multi-modal hybrid model that concatenated output from an Inception ResNet based model, run on RGB data with ImageNet pre-trained RGB weights, output from a residual network trained on NIR data, and LiDAR height data, before fully connected layers, using the small patch dataset with a minimum validation MAPE of 28.23% for fresh biomass and 11.43% for dryness. However, a very similar prediction accuracy resulted from a model that omitted NIR data, thus requiring fewer sensors and training resources, making it more sustainable. Although NIR and temperature differential data were collected and used for analysis, neither improved prediction accuracy, with the Inception ResNet model’s minimum validation MAPE rising to 39.42% when NIR data was added. When both NIR data and temperature differential were added to a multi-task learning Inception ResNet model, it yielded a minimum validation MAPE of 33.32%. As more labelled data are collected, the models can be further trained, enabling sensors on mowers to collect data and give timely trait information to farmers. This technology is also transferable to other crops. Overall, this work should provide a valuable contribution to the smart agriculture research space

A comprehensive review of crop yield prediction using machine learning approaches with special emphasis on palm oil yield prediction

An early and reliable estimation of crop yield is essential in quantitative and financial evaluation at the field level for determining strategic plans in agricultural commodities for import-export policies and doubling farmer’s incomes. Crop yield predictions are carried out to estimate higher crop yield through the use of machine learning algorithms which are one of the challenging issues in the agricultural sector. Due to this developing significance of crop yield prediction, this article provides an exhaustive review on the use of machine learning algorithms to predict crop yield with special emphasis on palm oil yield prediction. Initially, the current status of palm oil yield around the world is presented, along with a brief discussion on the overview of widely used features and prediction algorithms. Then, the critical evaluation of the state-of-the-art machine learning-based crop yield prediction, machine learning application in the palm oil industry and comparative analysis of related studies are presented. Consequently, a detailed study of the advantages and difficulties related to machine learning-based crop yield prediction and proper identification of current and future challenges to the agricultural industry is presented. The potential solutions are additionally prescribed in order to alleviate existing problems in crop yield prediction. Since one of the major objectives of this study is to explore the future perspectives of machine learning-based palm oil yield prediction, the areas including application of remote sensing, plant’s growth and disease recognition, mapping and tree counting, optimum features and algorithms have been broadly discussed. Finally, a prospective architecture of machine learning-based palm oil yield prediction has been proposed based on the critical evaluation of existing related studies. This technology will fulfill its promise by performing new research challenges in the analysis of crop yield prediction and the development

Predicting plant environmental exposure using remote sensing

Wheat is one of the most important crops globally with 776.4 million tonnes produced in 2019 alone. However, 10% of all wheat yield is predicted to be lost to Septoria Tritici Blotch (STB) caused by Zymoseptoria tritici (Z. tritici). Throughout Europe farmers spend £0.9 billion annually on preventative fungicide regimes to protect wheat against Z. tritici. A preventative fungicide regime is used as Z. tritici has a 9-16 day asymptomatic latent phase which makes it difficult to detect before symptoms develop, after which point fungicide intervention is ineffective. In the second chapter of my thesis I use hyperspectral sensing and imaging techniques, analysed with machine learning to detect and predict symptomatic Z. tritici infection in winter wheat, in UK based field trials, with high accuracy. This has the potential to improve detection and monitoring of symptomatic Z. tritici infection and could facilitate precision agriculture methods, to use in the subsequent growing season, that optimise fungicide use and increase yield. In the third chapter of my thesis, I develop a multispectral imaging system which can detect and utilise none visible shifts in plant leaf reflectance to distinguish plants based on the nitrogen source applied. Currently, plants are treated with nitrogen sources to increase growth and yield, the most common being calcium ammonium nitrate. However, some nitrogen sources are used in illicit activities. Ammonium nitrate is used in explosive manufacture and ammonium sulphate in the cultivation and extraction of the narcotic cocaine from Erythroxylum spp. In my third chapter I show that hyperspectral sensing, multispectral imaging, and machine learning image analysis can be used to visualise and differentiate plants exposed to different nefarious nitrogen sources. Metabolomic analysis of leaves from plants exposed to different nitrogen sources reveals shifts in colourful metabolites that may contribute to altered reflectance signatures. This suggests that different nitrogen feeding regimes alter plant secondary metabolism leading to changes in plant leaf reflectance detectable via machine learning of multispectral data but not the naked eye. These results could facilitate the development of technologies to monitor illegal activities involving various nitrogen sources and further inform nitrogen application requirements in agriculture. In my fourth chapter I implement and adapt the hyperspectral sensing, multispectral imaging and machine learning image analysis developed in the third chapter to detect asymptomatic (and symptomatic) Z. tritici infection in winter wheat, in UK based field trials, with high accuracy. This has the potential to improve detection and monitoring of all stages of Z. tritici infection and could facilitate precision agriculture methods to be used during the current growing season that optimise fungicide use and increase yield.Open Acces