Hybrid Neural Networks with Attention-based Multiple Instance Learning for Improved Grain Identification and Grain Yield Predictions

Agriculture is a critical part of the world's food production, being a vital aspect of all societies. Procedures need to be adjusted to their specific environment because of their climate and field condition disparity. Existing research has demonstrated the potential of grain yield predictions on Norwegian farms. However, this research is limited to regional analytics, which is unable to acquire sufficient plant growth factors influenced by field conditions and farmers' decisions. One factor critical for yield prediction is the crop type planted on a per-field basis. This research effort proposes a novel approach for improving crop yield predictions using a hybrid deep neural network utilizing temporal satellite imagery from a remote sensing system. Additionally, We apply a variety of data, including grain production, meteorological data, and geographical data. The crop yield prediction system is supported by a field-based crop type classification model, which supplies features related to crop type and field area. Our crop classification system takes advantage of both raw satellite images as well as carefully chosen vegetation indices. Further, we propose a multi-class attention-based deep multiple instance learning model to utilize semi-labeled datasets, fully benefiting Norwegian data acquisition. Our best crop classification model, which consists of a time distributed network and a gated recurrent unit, classifies crop types with an accuracy of 70\% and is currently state-of-the-art for country-wide crop type mapping in Norway. Lastly, our yield prediction system enables realistic in-season early predictions that could benefit actors in real-life scenarios

Mid-Season High-Resolution Satellite Imagery for Forecasting Site-Specific Corn Yield

Citation: Peralta, N.R.; Assefa, Y.; Du, J.; Barden, C.J.; Ciampitti, I.A. Mid-Season High-Resolution Satellite Imagery for Forecasting Site-Specific Corn Yield. Remote Sens. 2016, 8, 848.This technical note presents the first Sentinel-2 data service platform for obtaining atmospherically-corrected images and generating the corresponding value-added products for any land surface on Earth (http://s2.boku.eodc.eu/). Using the European Space Agency’s (ESA) Sen2Cor algorithm, the platform processes ESA’s Level-1C top-of-atmosphere reflectance to atmospherically-corrected bottom-of-atmosphere (BoA) reflectance (Level-2A). The processing runs on-demand, with a global coverage, on the Earth Observation Data Centre (EODC), which is a public-private collaborative IT infrastructure in Vienna (Austria) for archiving, processing, and distributing Earth observation (EO) data (http://www.eodc.eu). Using the data service platform, users can submit processing requests and access the results via a user-friendly web page or using a dedicated application programming interface (API). Building on the processed Level-2A data, the platform also creates value-added products with a particular focus on agricultural vegetation monitoring, such as leaf area index (LAI) and broadband hemispherical-directional reflectance factor (HDRF). An analysis of the performance of the data service platform, along with processing capacity, is presented. Some preliminary consistency checks of the algorithm implementation are included to demonstrate the expected product quality. In particular, Sentinel-2 data were compared to atmospherically-corrected Landsat-8 data for six test sites achieving a R2 = 0.90 and Root Mean Square Error (RMSE) = 0.031. LAI was validated for one test site using ground estimations. Results show a very good agreement (R2 = 0.83) and a RMSE of 0.32 m2/m2 (12% of mean value)

California Almond Yield Prediction at the Orchard Level With a Machine Learning Approach.

California's almond growers face challenges with nitrogen management as new legislatively mandated nitrogen management strategies for almond have been implemented. These regulations require that growers apply nitrogen to meet, but not exceed, the annual N demand for crop and tree growth and nut production. To accurately predict seasonal nitrogen demand, therefore, growers need to estimate block-level almond yield early in the growing season so that timely N management decisions can be made. However, methods to predict almond yield are not currently available. To fill this gap, we have developed statistical models using the Stochastic Gradient Boosting, a machine learning approach, for early season yield projection and mid-season yield update over individual orchard blocks. We collected yield records of 185 orchards, dating back to 2005, from the major almond growers in the Central Valley of California. A large set of variables were extracted as predictors, including weather and orchard characteristics from remote sensing imagery. Our results showed that the predicted orchard-level yield agreed well with the independent yield records. For both the early season (March) and mid-season (June) predictions, a coefficient of determination (R 2) of 0.71, and a ratio of performance to interquartile distance (RPIQ) of 2.6 were found on average. We also identified several key determinants of yield based on the modeling results. Almond yield increased dramatically with the orchard age until about 7 years old in general, and the higher long-term mean maximum temperature during April-June enhanced the yield in the southern orchards, while a larger amount of precipitation in March reduced the yield, especially in northern orchards. Remote sensing metrics such as annual maximum vegetation indices were also dominant variables for predicting the yield potential. While these results are promising, further refinement is needed; the availability of larger data sets and incorporation of additional variables and methodologies will be required for the model to be used as a fertilization decision support tool for growers. Our study has demonstrated the potential of automatic almond yield prediction to assist growers to manage N adaptively, comply with mandated requirements, and ensure industry sustainability

GeoAI approach to Vineyard Yield Estimation

A thesis submitted in partial fulfillment of the requirements for the degree of Doctor in Information Management, specialization in Geographic Information SystemsKnowing in advance vineyard yield is a key issue for growers, winemakers, policy makers, and regulators being fundamental to achieve the best balance between vegetative and reproductive growth, and to allow more informed decisions like thinning, irrigation and nutrient management, schedule harvest, optimize winemaking operations, program crop insurance, fraud detection and grape picking workforce demand. In a long-term scenario of perceived climate change, it is also essential for planning and regulatory purposes at the regional level. Estimating yield is complex and requires knowing driving factors related to climate, plant, and crop management that directly influence the number of clusters per vine, berries per cluster, and berry weight. These three yield components explain 60%, 30%, and 10% of the yield. The traditional methods are destructive, labor-demanding, and time-consuming, with low accuracy primarily due to operator errors and sparse sampling (compared to the inherent spatial variability in a production vineyard). Those are supported by manual sampling, where yield is estimated by sampling clusters weight and the number of clusters per vine, historical data, and extrapolation considering the number of vines in a plot. As the extensive research in the area clearly shows, improved applied methodologies are needed at different spatial scales. The methodological approaches for yield estimation based on indirect methods are primarily applicable at small scale and can provide better estimates than the traditional manual sampling. They mainly depend on computer vision and image processing algorithms, data-driven models based on vegetation indices and pollen data, and on relating climate, soil, vegetation, and crop management variables that can support dynamic crop simulation models. Despite surpassing the limitations assigned to traditional manual sampling methods with the same or better results on accuracy, they still lack a fundamental key aspect: the real application in commercial vineyards. Another gap is the lack of solutions for estimating yield at broader scales (e.g., regional level). The perception is that decisions are more likely to take place on a smaller scale, which in some cases is inaccurate. It might be the case in regulated areas and areas where support for small viticulturists is needed and made by institutions with proper resources and a large area of influence. This is corroborated by the fact that data-driven models based on Trellis Tension and Pollen traps are being used for yield estimation at regional scales in real environments in different regions of the world. The current dissertation consists of the first study to identify through a systematic literature review the research approaches for predicting yield in vineyards for wine production that can serve as an alternative to traditional estimation methods, to characterize the different new approaches identifying and comparing their applicability under field conditions, scalability concerning the objective, accuracy, advantages, and shortcomings. In the second study following the identified research gap, a yield estimation model based on Geospatial Artificial Intelligence (GeoAI) with remote sensing and climate data and a machine-learning approach was developed. Using a satellite-based time-series of Normalized Difference Vegetation Index (NDVI) calculated from Sentinel 2 images and climate data acquired by local automatic weather stations, a system for yield prediction based on a Long Short-Term Memory (LSTM) neural network was implemented. The results show that this approach makes it possible to estimate wine grape yield accurately in advance at different scales

Assessing the performance of MODIS NDVI and EVI for seasonal crop yield forecasting at the ecodistrict scale

Crop yield forecasting plays a vital role in coping with the challenges of the impacts of climate change on agriculture. Improvements in the timeliness and accuracy of yield forecasting by incorporating near real-time remote sensing data and the use of sophisticated statistical methods can improve our capacity to respond effectively to these challenges. The objectives of this study were (i) to investigate the use of derived vegetation indices for the yield forecasting of spring wheat (Triticum aestivum L.) from the Moderate resolution Imaging Spectroradiometer (MODIS) at the ecodistrict scale across Western Canada with the Integrated Canadian Crop Yield Forecaster (ICCYF); and (ii) to compare the ICCYF-model based forecasts and their accuracy across two spatial scales-the ecodistrict and Census Agricultural Region (CAR), namely in CAR with previously reported ICCYF weak performance. Ecodistricts are areas with distinct climate, soil, landscape and ecological aspects, whereas CARs are census-based/statistically-delineated areas. Agroclimate variables combined respectively with MODIS-NDVI and MODIS-EVI indices were used as inputs for the in-season yield forecasting of spring wheat during the 2000–2010 period. Regression models were built based on a procedure of a leave-one-year-out. The results showed that both agroclimate + MODIS-NDVI and agroclimate + MODIS-EVI performed equally well predicting spring wheat yield at the ECD scale. The mean absolute error percentages (MAPE) of the models selected from both the two data sets ranged from 2% to 33% over the study period. The model efficiency index (MEI) varied between -1.1 and 0.99 and -1.8 and 0.99, respectively for the agroclimate + MODIS-NDVI and agroclimate + MODIS-EVI data sets. Moreover, significant improvement in forecasting skill (with decreasing MAPE of 40% and 5 times increasing MEI, on average) was obtained at the finer, ecodistrict spatial scale, compared to the coarser CAR scale. Forecast models need to consider the distribution of extreme values of predictor variables to improve the selection of remote sensing indices. Our findings indicate that statistical-based forecasting error could be significantly reduced by making use of MODIS-EVI and NDVI indices at different times in the crop growing season and within different sub-regions

Exploring the invasion of rangelands by Acacia mearnsii (black wattle): biophysical characteristics and management implications

Australian acacias have spread to many parts of the world. In South Africa, species such as A. mearnsii and A. dealbata are invasive. Consequently, more effort has focused on their clearing. In a context of increasing clearing costs, it is crucial to develop innovative ways of managing invasions. Our aim was to understand the biophysical properties of A. mearnsii in grasslands as they relate to grass production and to explore management implications. Aboveground biomass (AGB) of A. mearnsii was determined using a published allometric equation in invaded grasslands of the northern Eastern Cape, South Africa. The relationships among the A. mearnsii leaf area index (LAI), normalised difference vegetation index (NDVI) and AGB were investigated. The influence of A. mearnsii LAI and terrain slope on grass cover was also investigated. Strong linear relationships between NDVI, LAI and AGB were developed. Acacia mearnsii canopy adversely impacted grass production more than terrain slope (p < 0.05) and when LAI approached 2.1, grass cover dropped to below 10% in infested areas. Reducing A. mearnsii canopy could promote grass production while encouraging carbon sequestration. Given the high AGB and clearing costs, it may be prudent to adopt the ‘novel ecosystems’ approach in managing infested landscapes.Keywords: grassland, invasive plants, landscape ecology, rangeland conditio

The Vegetation Outlook (VegOut): A New Method for Predicting Vegetation Seasonal Greenness

The vegetation outlook (VegOut) is a geospatial tool for predicting general vegetation condition patterns across large areas. VegOut predicts a standardized seasonal greenness (SSG) measure, which represents a general indicator of relative vegetation health. VegOut predicts SSG values at multiple time steps (two to six weeks into the future) based on the analysis of “historical patterns” (i.e., patterns at each 1 km grid cell and time of the year) of satellite, climate, and oceanic data over an 18-year period (1989 to 2006). The model underlying VegOut capitalizes on historical climate–vegetation interactions and ocean–climate teleconnections (such as El Niño and the Southern Oscillation, ENSO) expressed over the 18-year data record and also considers several environmental characteristics (e.g., land use/cover type and soils) that influence vegetation’s response to weather conditions to produce 1 km maps that depict future general vegetation conditions. VegOut provides regional level vegetation monitoring capabilities with local-scale information (e.g., county to sub-county level) that can complement more traditional remote sensing–based approaches that monitor “current” vegetation conditions. In this paper, the VegOut approach is discussed and a case study over the central United States for selected periods of the 2008 growing season is presented to demonstrate the potential of this new tool for assessing and predicting vegetation conditions

Enhancing Crop Yield Prediction Utilizing Machine Learning on Satellite-Based Vegetation Health Indices

Accurate crop yield forecasting is essential in the food industry’s decision-making process, where vegetation condition index (VCI) and thermal condition index (TCI) coupled with machine learning (ML) algorithms play crucial roles. The drawback, however, is that a one-fits-all prediction model is often employed over an entire region without considering subregional VCI and TCI’s spatial variability resulting from environmental and climatic factors. Furthermore, when using nonlinear ML, redundant VCI/TCI data present additional challenges that adversely affect the models’ output. This study proposes a framework that (i) employs higher-order spatial independent component analysis (sICA), and (ii), exploits a combination of the principal component analysis (PCA) and ML (i.e., PCA-ML combination) to deal with the two challenges in order to enhance crop yield prediction accuracy. The proposed framework consolidates common VCI/TCI spatial variability into their respective subregions, using Vietnam as an example. Compared to the one-fits-all approach, subregional rice yield forecasting models over Vietnam improved by an average level of 20% up to 60%. PCA-ML combination outperformed ML-only by an average of 18.5% up to 45%. The framework generates rice yield predictions 1 to 2 months ahead of the harvest with an average of 5% error, displaying its reliability

Remote sensing and machine learning for prediction of wheat growth in precision agriculture applications

This thesis focuses on remote sensing and machine learning for prediction of wheat growth in precision agriculture applications. Agriculture is the primary productive force, which plays an important role in human activities. Wheat, as one of the essential sources of food, is also a widely planted crop. The impact of weather and climate and some other uncertain factors on wheat production is crucial. Therefore, it is necessary to use reliable and statistically reasonable models for crop growth and yield prediction based on vegetation index variables and other factors, so as to obtain reliable prediction for efficient production. Applying certain artificial intelligence algorithms to the precision agriculture can significantly improve the efficiency of traditional agriculture in crop planting and reduce the consumption of human and natural resources. Remote sensing can objectively, accurately and timely provide a large amount of information for ecological environment and crop growth in agriculture applications. By combining the image and spectral data obtained by remote sensing technology with machine learning, information about wheat growth, yield and insect pests can be learned in time. This thesis focuses on its applications in agriculture, particularly using effective prediction models such as the back propagation neural network and some optimisation algorithms for predicting wheat growth, yield and aphid. The work presented in this thesis address the issues of wheat growth prediction, yield assessment and aphid validation by model building and machine learning algorithm optimisation by means of remote sensing data. Specifically, the following objectives are defined: 1. Analyse multiple vegetation indexes based on the TM 1-4 band data of Landsat satellite and use regression algorithms to train the models and predict wheat growth; 2. Analyse and compare multiple vegetation indexes models by means of spectral data and use regression algorithms to predict wheat yield; 3. Combine spectral vegetation indexes and multiple regression algorithms to predict wheat aphid; 4. Use accurate evaluation criteria for validating the efficacy of the various algorithms. In this thesis, the remote sensing data from the satellite has been applied instead of the airborne-based remote sensing data. Based on the TM 1-4 band image data of Landsat satellite, multiple vegetation indexes were used as the input of regression algorithms. After that, four kinds of regression algorithms such as the multiple linear regression (MR) algorithm, back propagation network (BPNN) algorithm, genetic algorithm (GA) optimised BPNN algorithm and particle swarm optimisation (PSO) optimised BPNN algorithm were used to train the model and predict the LAI and SPAD. The prediction results of each algorithm were compared with the ground truth information collected by hand held instruments on the ground. The relationship between wheat yield and spectral data has been studied. Based on the BPNN algorithm, four kinds of models such as visible hyperspectral index (VHI) model, hyperspectral vegetation index (HVI) model, difference hyperspectral index (DHI) model and normalized hyperspectral index (NHI) model have been utilized to predict wheat yield. For the optimal NHI model, three regression algorithms such as back propagation network (BPNN) algorithm, genetic algorithm (GA) optimised BPNN algorithm and particle swarm optimisation (PSO) optimised BPNN algorithm, were compared to predict wheat yield, and RMSE and R-square of the three algorithms were compared and analysed. Finally, the relationship between wheat aphid and spectral data has been investigated. Nine vegetation indexes related to aphid have been estimated from spectral data as the input of regression algorithms. Five kinds of regression algorithms such as back propagation network (BPNN) algorithm, genetic algorithm (GA) optimised BPNN algorithm, particle swarm optimisation (PSO) optimised BPNN algorithm, ant colony (ACO) optimisation algorithm optimised BPNN algorithm and cuckoo search (CS) optimised BPNN algorithm have been implemented to predict wheat aphid, which was validated with the ground truth information measured by hand-held instruments on the ground. The prediction results of each algorithm have been analysed. The major original contributions of this thesis are as follows: 1. A variety of optimisation algorithms are used to improve the regression analysis of the BPNN algorithm, so that the prediction results of each model for wheat growth, yield and aphid are more accurate. 2. The spectral characteristics of winter wheat canopy have been analysed. The correlation between the absorption band and the associated physical and chemical properties of crops, specially the red edge slope, with the crop yield and wheat aphid damage is established. 3. Adjusted MSE and un-centered R-square, as accurate evaluation criteria for practical applications, are used to compare the prediction results of the models under different dimensions of the observed data. 4. Improve algorithm training by using the cross-validation method to obtain reliable and stable models for the prediction of wheat growth, yield, and aphid. Through repeated cross-validation, a better model can be obtained in the last. Key word：Precision agriculture; BP network, wheat growth assessment; wheat yield prediction, wheat aphid validationThis thesis focuses on remote sensing and machine learning for prediction of wheat growth in precision agriculture applications. Agriculture is the primary productive force, which plays an important role in human activities. Wheat, as one of the essential sources of food, is also a widely planted crop. The impact of weather and climate and some other uncertain factors on wheat production is crucial. Therefore, it is necessary to use reliable and statistically reasonable models for crop growth and yield prediction based on vegetation index variables and other factors, so as to obtain reliable prediction for efficient production. Applying certain artificial intelligence algorithms to the precision agriculture can significantly improve the efficiency of traditional agriculture in crop planting and reduce the consumption of human and natural resources. Remote sensing can objectively, accurately and timely provide a large amount of information for ecological environment and crop growth in agriculture applications. By combining the image and spectral data obtained by remote sensing technology with machine learning, information about wheat growth, yield and insect pests can be learned in time. This thesis focuses on its applications in agriculture, particularly using effective prediction models such as the back propagation neural network and some optimisation algorithms for predicting wheat growth, yield and aphid. The work presented in this thesis address the issues of wheat growth prediction, yield assessment and aphid validation by model building and machine learning algorithm optimisation by means of remote sensing data. Specifically, the following objectives are defined: 1. Analyse multiple vegetation indexes based on the TM 1-4 band data of Landsat satellite and use regression algorithms to train the models and predict wheat growth; 2. Analyse and compare multiple vegetation indexes models by means of spectral data and use regression algorithms to predict wheat yield; 3. Combine spectral vegetation indexes and multiple regression algorithms to predict wheat aphid; 4. Use accurate evaluation criteria for validating the efficacy of the various algorithms. In this thesis, the remote sensing data from the satellite has been applied instead of the airborne-based remote sensing data. Based on the TM 1-4 band image data of Landsat satellite, multiple vegetation indexes were used as the input of regression algorithms. After that, four kinds of regression algorithms such as the multiple linear regression (MR) algorithm, back propagation network (BPNN) algorithm, genetic algorithm (GA) optimised BPNN algorithm and particle swarm optimisation (PSO) optimised BPNN algorithm were used to train the model and predict the LAI and SPAD. The prediction results of each algorithm were compared with the ground truth information collected by hand held instruments on the ground. The relationship between wheat yield and spectral data has been studied. Based on the BPNN algorithm, four kinds of models such as visible hyperspectral index (VHI) model, hyperspectral vegetation index (HVI) model, difference hyperspectral index (DHI) model and normalized hyperspectral index (NHI) model have been utilized to predict wheat yield. For the optimal NHI model, three regression algorithms such as back propagation network (BPNN) algorithm, genetic algorithm (GA) optimised BPNN algorithm and particle swarm optimisation (PSO) optimised BPNN algorithm, were compared to predict wheat yield, and RMSE and R-square of the three algorithms were compared and analysed. Finally, the relationship between wheat aphid and spectral data has been investigated. Nine vegetation indexes related to aphid have been estimated from spectral data as the input of regression algorithms. Five kinds of regression algorithms such as back propagation network (BPNN) algorithm, genetic algorithm (GA) optimised BPNN algorithm, particle swarm optimisation (PSO) optimised BPNN algorithm, ant colony (ACO) optimisation algorithm optimised BPNN algorithm and cuckoo search (CS) optimised BPNN algorithm have been implemented to predict wheat aphid, which was validated with the ground truth information measured by hand-held instruments on the ground. The prediction results of each algorithm have been analysed. The major original contributions of this thesis are as follows: 1. A variety of optimisation algorithms are used to improve the regression analysis of the BPNN algorithm, so that the prediction results of each model for wheat growth, yield and aphid are more accurate. 2. The spectral characteristics of winter wheat canopy have been analysed. The correlation between the absorption band and the associated physical and chemical properties of crops, specially the red edge slope, with the crop yield and wheat aphid damage is established. 3. Adjusted MSE and un-centered R-square, as accurate evaluation criteria for practical applications, are used to compare the prediction results of the models under different dimensions of the observed data. 4. Improve algorithm training by using the cross-validation method to obtain reliable and stable models for the prediction of wheat growth, yield, and aphid. Through repeated cross-validation, a better model can be obtained in the last. Key word：Precision agriculture; BP network, wheat growth assessment; wheat yield prediction, wheat aphid validatio