Using Deep Learning to Predict Plant Growth and Yield in Greenhouse Environments

Funding Information: This work is part of EU Interreg SMARTGREEN project (2017-2021). We would like to thank all the growers (UK & EU), for providing the data. Their valuable feedback, suggestions and comments are highly appreciated to increase the overall quality of this work.Postprin

합성곱 신경망과 완전 연결 계층의 앙상블 모델을 이용한 정밀한 파프리카 과실 발달 단계 예측

학위논문 (석사) -- 서울대학교 대학원 : 농업생명과학대학 농림생물자원학부, 2021. 2. 손정익.Accurate detection of individual fruits and prediction of their development stages enable growers to efficiently allocate labor and manage strategically. However, the prediction of the fruit development stage is challenging, especially in sweet peppers, because the fruit harvest is discrete and its immature stage is indistinguishable. An ensemble model of convolutional and fully connected neural networks was developed to detect sweet pepper (Capsicum annuum L.) fruits in images and predict their development stages. The plants were grown in four rows in a greenhouse, and images were collected in each row. Plant growth and environmental data were collected every minute and month, respectively. For predicting the fruit stage, an ensemble of convolutional neural network (CNN) and multilayer perceptron (MLP) models were used. The fruit development stage was classified into immature, breaking, and mature stages with a CNN using images. Moreover, the immature stage was internally divided into four stages with an MLP. The plant growth and environmental data and the information from the CNN output were used for the MLP input. That is, a total of six stages were classified using the CNN–MLP ensemble model. The ensemble model showed good agreement in predicting fruit development stages. The average accuracy of the six stages was F1 score = 0.77 and IoU = 0.86. The CNN-only model could classify the mature and breaking stages well, but the immature stages were not distinguished, while the MLP-only model could hardly classify the fruit stage except the immature stages. The most influential factors in classification were the data obtained from CNN and the plant growth and environment data, which contributed to the improvement of model accuracy. The ensemble models can help in appropriate labor allocation and strategic management by detecting individual fruits in images and predicting precise fruit development stages.온실에서는 고부가가치에 열매를 맺는 작물을 효율적으로 관리하는 것이 중요하다. 개별 과실을 감지하고 그것의 발달 단계를 예측함으로써 재배자가 노동력을 적재적소에 할당하고, 전략적인 관리를 할 수 있다. 그러나 파프리카의 과실 발달 단계를 예측하는 것은 과실 수확량이 불연속적이고, 미성숙 단계에서 과실 간 나타나는 외부적인 특징 차이를 구별하기 어렵기 때문에 쉽지 않다. 이 연구의 목적은 합성곱 신경망과 완전 연결 계층의 앙상블 모델을 이용하여 이미지에서 파프리카 과실을 찾아내고 과실 발달 단계를 예측하는 것이다. 실험용 온실에서 파프리카 (Capsicum annuum L.)를 4줄로 재배하였고, 각 줄에 양면에서 식물 이미지를 수집 하였다. 2020년 4월 6일부터 6월 24일까지 환경 데이터는 분 마다, 식물 생장 데이터는 월 마다 수집되었다. 과실 발달 단계는 이미지에서 합성곱 신경망을 이용하여 미성숙, 변화 중, 성숙 3 단계로 구분하였고, 미성숙 단계는 완전 연결 계층을 이용하여 다시 세부적으로 4 단계로 구분 하였다. 환경, 식물 생장 데이터 및 합성곱 신경망의 출력 정보가 완전 연결 계층에 입력으로 사용되었다. 즉, 총 6 개의 과실 발달 단계가 앙상블 모델을 이용하여 분류되었다. 앙상블 모델은 과실 발달 단계를 예측하는 데 좋은 성능을 보였다. 총 6 단계의 과실 발달 단계 분류에 평균 정확도는 F1 점수 = 0.77, IoU = 0.86이다. 합성곱 신경망만을 이용한 모델은 성숙 단계와 변화 중 단계를 잘 분류 할 수 있었지만 미성숙 단계를 구별하지 못하였다. 완전 연결 계층만을 이용한 모델은 미성숙 단계를 제외하고 과실 단계를 거의 분류 할 수 없었다. 과실 발달 단계의 분류에 가장 큰 영향을 미치는 요인은 합성곱 신경망의 출력 정보였고, 환경 및 식물 생장 데이터는 모델 정확도 향상에 기여했다. 본 연구 결과는 추후 다양한 환경에 이미지에서 개별 과실을 찾아내고, 정확한 과실 발달 단계를 예측함으로써 적절한 노동력 할당 및 전략적 관리에 도움이 될 수 있을 것으로 사료된다.ABSTRACT i CONTENTS iii LIST OF TABLES iv LIST OF FIGURES v INTRODUCTION 1 LITERATURE REVIEW 4 MATERIALS AND METHODS 9 RESULTS 24 DISCUSSION 34 CONCLUSION 39 LITERATURE CITED 40 ABSTRACT IN KOREAN 47Maste

Source-tracking cadmium in New Zealand agricultural soils: a stable isotope approach

Cadmium (Cd) is a toxic heavy metal, which is accumulated by plants and animals and therefore enters the human food chain. In New Zealand (NZ), where Cd mainly originates from the application of phosphate fertilisers, stable isotopes can be used to trace the fate of Cd in soils and potentially the wider environment due to the limited number of sources in this setting. Prior to 1997, extraneous Cd added to soils in P fertilisers was essentially limited to a single source, the small pacific island of Nauru. Analysis of Cd isotope ratios (ɛ114/110Cd) in Nauru rock phosphate, pre-1997 superphosphate fertilisers, and Canterbury (Lismore Stony Silt Loam) topsoils (Winchmore Research Farm) has demonstrated their close similarity with respect to ɛ114/110Cd. We report a consistent ɛ114/110Cd signature in fertiliser-derived Cd throughout the latter twentieth century. This finding is useful because it allows the application of mixing models to determine the proportions of fertiliser-derived Cd in the wider environment. We believe this approach has good potential because we also found the ɛ114/110Cd in fertilisers to be distinct from unfertilised Canterbury subsoils. In our analysis of the Winchmore topsoil series (1949-2015), the ɛ114/110Cd remained quite constant following the change from Nauru to other rock phosphate sources in 1997, despite a corresponding shift in fertiliser ɛ114/110Cd at this time. We can conclude that to the present day, the Cd in topsoil at Winchmore still mainly originates from historical phosphate fertilisers. One implication of this finding is that the current applications of P fertiliser are not resulting in further Cd accumulation. We aim to continue our research into Cd fate, mobility and transformations in the NZ environment by applying Cd isotopes in soils and aquatic environments across the country

딥러닝 방법론을 이용한 높은 적용성을 가진 수경재배 파프리카 대상 절차 기반 모델 개발

학위논문(박사) -- 서울대학교대학원 : 농업생명과학대학 농림생물자원학부, 2022. 8. 손정익.Many agricultural challenges are entangled in a complex interaction between crops and the environment. As a simplifying tool, crop modeling is a process of abstracting and interpreting agricultural phenomena. Understanding based on this interpretation can play a role in supporting academic and social decisions in agriculture. Process-based crop models have solved the challenges for decades to enhance the productivity and quality of crop production; the remaining objectives have led to demand for crop models handling multidirectional analyses with multidimensional information. As a possible milestone to satisfy this goal, deep learning algorithms have been introduced to the complicated tasks in agriculture. However, the algorithms could not replace existing crop models because of the research fragmentation and low accessibility of the crop models. This study established a developmental protocol for a process-based crop model with deep learning methodology. Literature Review introduced deep learning and crop modeling, and it explained the reasons for the necessity of this protocol despite numerous deep learning applications for agriculture. Base studies were conducted with several greenhouse data in Chapters 1 and 2: transfer learning and U-Net structure were utilized to construct an infrastructure for the deep learning application; HyperOpt, a Bayesian optimization method, was tested to calibrate crop models to compare the existing crop models with the developed model. Finally, the process-based crop model with full deep neural networks, DeepCrop, was developed with an attention mechanism and multitask decoders for hydroponic sweet peppers (Capsicum annuum var. annuum) in Chapter 3. The methodology for data integrity showed adequate accuracy, so it was applied to the data in all chapters. HyperOpt was able to calibrate food and feed crop models for sweet peppers. Therefore, the compared models in the final chapter were optimized using HyperOpt. DeepCrop was trained to simulate several growth factors with environment data. The trained DeepCrop was evaluated with unseen data, and it showed the highest modeling efficiency (=0.76) and the lowest normalized root mean squared error (=0.18) than the compared models. With the high adaptability of DeepCrop, it can be used for studies on various scales and purposes. Since all methods adequately solved the given tasks and underlay the DeepCrop development, the established protocol can be a high throughput for enhancing accessibility of crop models, resulting in unifying crop modeling studies.농업 시스템에서 발생하는 문제들은 작물과 환경의 상호작용 하에 복잡하게 얽혀 있다. 작물 모델링은 대상을 단순화하는 방법으로써, 농업에서 일어나는 현상을 추상화하고 해석하는 과정이다. 모델링을 통해 대상을 이해하는 것은 농업 분야의 학술적 및 사회적 결정을 지원할 수 있다. 지난 수년 간 절차 기반 작물 모델은 농업의 문제들을 해결하여 작물 생산성 및 품질을 증진시켰으며, 현재 작물 모델링에 남아있는 과제들은 다차원 정보를 다방향에서 분석할 수 있는 작물 모델을 필요로 하게 되었다. 이를 만족시킬 수 있는 지침으로써, 복잡한 농업적 과제들을 목표로 딥러닝 알고리즘이 도입되었다. 그러나, 이 알고리즘들은 낮은 데이터 완결성 및 높은 연구 다양성 때문에 기존의 작물 모델들을 대체하지는 못했다. 본 연구에서는 딥러닝 방법론을 이용하여 절차 기반 작물 모델을 구축하는 개발 프로토콜을 확립하였다. Literature Review에서는 딥러닝과 작물 모델에 대해 소개하고, 농업으로의 딥러닝 적용 연구가 많음에도 이 프로토콜이 필요한 이유를 설명하였다. 제1장과 2장에서는 국내 여러 지역의 데이터를 이용하여 전이 학습 및 U-Net 구조를 활용하여 딥러닝 모델 적용을 위한 기반을 마련하고, 베이지안 최적화 방법인 HyperOpt를 사용하여 기존 모델과 딥러닝 기반 모델을 비교하기 위해 시험적으로 WOFOST 작물 모델을 보정하는 등 모델 개발을 위한 기반 연구를 수행하였다. 마지막으로, 제3장에서는 주의 메커니즘 및 다중 작업 디코더를 가진 완전 심층 신경망 절차 기반 작물 모델인 DeepCrop을 수경재배 파프리카(Capsicum annuum var. annuum) 대상으로 개발하였다. 데이터 완결성을 위한 기술들은 적합한 정확도를 보여주었으며, 전체 챕터 데이터에 적용하였다. HyperOpt는 식량 및 사료 작물 모델들을 파프리카 대상으로 보정할 수 있었다. 따라서, 제3장의 비교 대상 모델들에 대해 HyperOpt를 사용하였다. DeepCrop은 환경 데이터를 이용하고 여러 생육 지표를 예측하도록 학습되었다. 학습에 사용하지 않은 데이터를 이용하여 학습된 DeepCrop를 평가하였으며, 이 때 비교 모델들 중 가장 높은 모형 효율(EF=0.76)과 가장 낮은 표준화 평균 제곱근 오차(NRMSE=0.18)를 보여주었다. DeepCrop은 높은 적용성을 기반으로 다양한 범위와 목적을 가진 연구에 사용될 수 있을 것이다. 모든 방법들이 주어진 작업을 적절히 풀어냈고 DeepCrop 개발의 근거가 되었으므로, 본 논문에서 확립한 프로토콜은 작물 모델의 접근성을 향상시킬 수 있는 획기적인 방향을 제시하였고, 작물 모델 연구의 통합에 기여할 수 있을 것으로 기대한다.LITERATURE REVIEW 1 ABSTRACT 1 BACKGROUND 3 REMARKABLE APPLICABILITY AND ACCESSIBILITY OF DEEP LEARNING 12 DEEP LEARNING APPLICATIONS FOR CROP PRODUCTION 17 THRESHOLDS TO APPLY DEEP LEARNING TO CROP MODELS 18 NECESSITY TO PRIORITIZE DEEP-LEARNING-BASED CROP MODELS 20 REQUIREMENTS OF THE DEEP-LEARNING-BASED CROP MODELS 21 OPENING REMARKS AND THESIS OBJECTIVES 22 LITERATURE CITED 23 Chapter 1 34 Chapter 1-1 35 ABSTRACT 35 INTRODUCTION 37 MATERIALS AND METHODS 40 RESULTS 50 DISCUSSION 59 CONCLUSION 63 LITERATURE CITED 64 Chapter 1-2 71 ABSTRACT 71 INTRODUCTION 73 MATERIALS AND METHODS 75 RESULTS 84 DISCUSSION 92 CONCLUSION 101 LITERATURE CITED 102 Chapter 2 108 ABSTRACT 108 NOMENCLATURE 110 INTRODUCTION 112 MATERIALS AND METHODS 115 RESULTS 124 DISCUSSION 133 CONCLUSION 137 LITERATURE CITED 138 Chapter 3 144 ABSTRACT 144 INTRODUCTION 146 MATERIALS AND METHODS 149 RESULTS 169 DISCUSSION 182 CONCLUSION 187 LITERATURE CITED 188 GENERAL DISCUSSION 196 GENERAL CONCLUSION 201 ABSTRACT IN KOREAN 203 APPENDIX 204박

PREDICTION OF CROP YIELDS ACROSS FOUR CLIMATE ZONES IN GERMANY: AN ARTIFICIAL NEURAL NETWORK APPROACH

This paper shows the ability of artificial neural network technology to be used for the approximation and prediction of crop yields at rural district and federal state scales in different climate zones based on reported daily weather data. The method may later be used to construct regional time series of agricultural output under climate change, based on the highly resolved output of the global circulation models and regional models. Three 30-year combined historical data sets of rural district yields (oats, spring barley and silage maize), daily temperatures (mean, maximum, dewpoint) and precipitation were constructed. They were used with artificial neural network technology to investigate, simulate and predict historical time series of crop yields in four climate zones of Germany. Final neural networks, trained with data sets of three climate zones and tested against an independent northern zone, have high predictive power (0.83global change, agriculture, artificial neural networks, yield prediction

Annual Report: 2012

I submit herewith the annual reports from the Agricultural and Forestry Experiment Station, School of Natural Resources and Agricultural Sciences, University of Alaska Fairbanks, for the period ending December 31, 2012. This is done in accordance with an act of Congress, approved March 2, 1887, entitled, “An act to establish agricultural experiment stations, in connection with the agricultural college established in the several states under the provisions of an act approved July 2, 1862, and under the acts supplementary thereto,” and also of the act of the Alaska Territorial Legislature, approved March 12, 1935, accepting the provisions of the act of Congress. The research reports are organized according to our strategic plan and by broad subject, focusing on geography, high-latitude agriculture, forest sciences, and the interaction of humans and the environment. Research conducted by our graduate and undergraduate students plays an important role in these grants and the impact they make on Alaska.Financial Statement -- Grants -- Students -- Research at SNRAS & AFES -- Publications -- Facult

In Vivo Human-Like Robotic Phenotyping of Leaf and Stem Traits in Maize and Sorghum in Greenhouse

In plant phenotyping, the measurement of morphological, physiological and chemical traits of leaves and stems is needed to investigate and monitor the condition of plants. The manual measurement of these properties is time consuming, tedious, error prone, and laborious. The use of robots is a new approach to accomplish such endeavors, which enables automatic monitoring with minimal human intervention. In this study, two plant phenotyping robotic systems were developed to realize automated measurement of plant leaf properties and stem diameter which could reduce the tediousness of data collection compare to manual measurements. The robotic systems comprised of a four degree of freedom (DOF) robotic manipulator and a Time-of-Flight (TOF) camera. Robotic grippers were developed to integrate an optical fiber cable (coupled to a portable spectrometer) for leaf spectral reflectance measurement, a thermistor for leaf temperature measurement, and a linear potentiometer for stem diameter measurement. An Image processing technique and deep learning method were used to identify grasping points on leaves and stems, respectively. The systems were tested in a greenhouse using maize and sorghum plants. The results from the leaf phenotyping robot experiment showed that leaf temperature measurements by the phenotyping robot were correlated with those measured manually by a human researcher (R2 = 0.58 for maize and 0.63 for sorghum). The leaf spectral measurements by the phenotyping robot predicted leaf chlorophyll, water content and potassium with moderate success (R2 ranged from 0.52 to 0.61), whereas the prediction for leaf nitrogen and phosphorus were poor. The total execution time to grasp and take measurements from one leaf was 35.5±4.4 s for maize and 38.5±5.7 s for sorghum. Furthermore, the test showed that the grasping success rate was 78% for maize and 48% for sorghum. The experimental results from the stem phenotyping robot demonstrated a high correlation between the manual and automated stem diameter measurements (R2 \u3e 0.98). The execution time for stem diameter measurement was 45.3 s. The system could successfully detect and localize, and also grasp the stem for all plants during the experiment. Both robots could decrease the tediousness of collecting phenotypes compare to manual measurements. The phenotyping robots can be useful to complement the traditional image-based high-throughput plant phenotyping in greenhouses by collecting in vivo morphological, physiological, and biochemical trait measurements for plant leaves and stems. Advisors: Yufeng Ge, Santosh Pitl

Developing affordable high-throughput plant phenotyping methods for breeding of cereals and tuber crops

High-throughput plant phenotyping (HTPP) is a fast, accurate, and non-destructive process for evaluating plants' health and environmental adaptability. HTPP accelerates the identification of agronomic traits of interest, eliminates subjectivism (which is innate to humans), and facilitates the development of adapted genotypes. Current HTPP methods often rely on imaging sensors and computer vision both in the field and under controlled (indoor) conditions. However, their use is limited by the costs and complexity of the necessary instrumentation, data analysis tools, and software. This issue could be overcome by developing more cost-efficient and user-friendly methods that let breeders, farmers, and stakeholders access the benefits of HTPP. To assist such efforts, this thesis presents an ensemble of dedicated affordable phenotyping methods using RGB imaging for a range of key applications under controlled conditions.  The affordable Phenocave imaging system for use in controlled conditions was developed to facilitate studies on the effects of abiotic stresses by gathering data on important plant characteristics related to growth, yield, and adaptation to growing conditions and cultivation systems. Phenocave supports imaging sensors including visible (RGB), spectroscopic (multispectral and hyperspectral), and thermal imaging. Additionally, a pipeline for RGB image analysis was implemented as a plugin for the free and easy-to-use software ImageJ. This plugin has since proven to be an accurate alternative to conventional measurements that produces highly reproducible results. A subsequent study was conducted to evaluate the effects of heat and drought stress on plant growth and grain nutrient composition in wheat, an important staple cereal in Sweden. The effects of stress on plant growth were evaluated using image analysis, while stress-induced changes in the abundance of key plant compounds were evaluated by analyzing the nutrient composition of grains via chromatography. This led to the discovery of genotypes whose harvest quality remains stable under heat and drought stress. The next objective was to evaluate biotic stress; for this case, the effect of the fungal disease Fusarium head blight (FHB) that affects grain development in wheat was investigated. For this purpose, seed phenotyping parameters were used to determine the components and settings of a statistical model, which predicts the occurrence of FHB. The results reveal that grain morphology evaluations, such as length and width, were found to be significantly affected by the disease. Another study was carried out to estimate the disease severity of the common scab (CS) in potatoes, a widely popular food source. CS occurs on the tubers and reduces their visual appeal, significantly affecting their market value. Tubers were analyzed by a deep learning-based method to estimate disease lesion areas caused by CS. Results showed a high correlation between the predictions and expert visual scorings of the disease and proved to be a potential tool for the selection of genotypes that fulfill the market standards and resistance to CS. Both case studies highlight the role of imaging in plant health monitoring and its integration into the larger picture of plant health management.  The methods presented in this work are a starting point for bridging the gap between costs and accessibility to imaging technology. These are affordable and user-friendly resources for generating pivotal knowledge on plant development and genotype selection. In the future, image acquisition of all the methods can be integrated into the Phenocave system, potentially allowing for a more automated and efficient plant health monitoring process, leading to the identification of tolerant genotypes to biotic and abiotic stresses

Using numerical plant models and phenotypic correlation space to design achievable ideotypes

Numerical plant models can predict the outcome of plant traits modifications resulting from genetic variations, on plant performance, by simulating physiological processes and their interaction with the environment. Optimization methods complement those models to design ideotypes, i.e. ideal values of a set of plant traits resulting in optimal adaptation for given combinations of environment and management, mainly through the maximization of a performance criteria (e.g. yield, light interception). As use of simulation models gains momentum in plant breeding, numerical experiments must be carefully engineered to provide accurate and attainable results, rooting them in biological reality. Here, we propose a multi-objective optimization formulation that includes a metric of performance, returned by the numerical model, and a metric of feasibility, accounting for correlations between traits based on field observations. We applied this approach to two contrasting models: a process-based crop model of sunflower and a functional-structural plant model of apple trees. In both cases, the method successfully characterized key plant traits and identified a continuum of optimal solutions, ranging from the most feasible to the most efficient. The present study thus provides successful proof of concept for this enhanced modeling approach, which identified paths for desirable trait modification, including direction and intensity.Comment: 25 pages, 5 figures, 2017, Plant, Cell and Environmen

Climate change, agriculture and Fairtrade: identifying the challenges and opportunities

This paper presents the findings of a study commissioned by the UK's Fairtrade Foundation on the implications of climate change for agricultural producers in Fairtrade value chains