Machine learning to generate soil information

This thesis is concerned with the novel use of machine learning (ML) methods in soil science research. ML adoption in soil science has increased considerably, especially in pedometrics (the use of quantitative methods to study the variation of soils). In parallel, the size of the soil datasets has also increased thanks to projects of global impact that aim to rescue legacy data or new large extent surveys to collect new information. While we have big datasets and global projects, currently, modelling is mostly based on "traditional" ML approaches which do not take full advantage of these large data compilations. This compilation of these global datasets is severely limited by privacy concerns and, currently, no solution has been implemented to facilitate the process. If we consider the performance differences derived from the generality of global models versus the specificity of local models, there is still a debate on which approach is better. Either in global or local DSM, most applications are static. Even with the large soil datasets available to date, there is not enough soil data to perform a fully-empirical, space-time modelling. Considering these knowledge gaps, this thesis aims to introduce advanced ML algorithms and training techniques, specifically deep neural networks, for modelling large datasets at a global scale and provide new soil information. The research presented here has been successful at applying the latest advances in ML to improve upon some of the current approaches for soil modelling with large datasets. It has also created opportunities to utilise information, such as descriptive data, that has been generally disregarded. ML methods have been embraced by the soil community and their adoption is increasing. In the particular case of neural networks, their flexibility in terms of structure and training makes them a good candidate to improve on current soil modelling approaches

Rapid detection of soil carbonates by means of NIR spectroscopy, deep learning methods and phase quantification by powder Xray diffraction

Soil NIR spectral absorbance/reflectance libraries are utilized towards improving agricultural production and analysis of soil properties which are key prerequisite for agroecological balance and environmental sustainability. Carbonates in particular, represent a soil property which is mostly affected even by mild, let alone extreme, changes of environmental conditions during climate change. In this study we propose a rapid and efficient way to predict carbonates content in soil by means of FT NIR reflectance spectroscopy and by use of deep learning methods. We exploited multiple machine learning methods, such as: 1) a MLP Regressor and 2) a CNN and compare their performance with other traditional ML algorithms such as PLSR, Cubist and SVM on the combined dataset of two NIR spectral libraries: KSSL (USDA), a dataset of soil samples reflectance spectra collected nationwide, and LUCAS TopSoil (European Soil Library) which contains soil sample absorbance spectra from all over the European Union, and use them to predict carbonate content on never before seen soil samples. Soil samples in KSSL and in TopSoil spectral libraries were acquired in the spectral region of visNIR, however in this study, only the NIR spectral region was utilized. Quantification of carbonates by means of Xray Diffraction is in good agreement with the volumetric method and the MLP prediction. Our work contributes to rapid carbonates content prediction in soil samples in cases where: 1) no volumetric method is available and 2) only NIR spectra absorbance data are available. Up till now and to the best of our knowledge, there exists no other study, that presents a prediction model trained on such an extensive dataset with such promising results on unseen data, undoubtedly supporting the notion that deep learning models present excellent prediction tools for soil carbonates content.Comment: 39 pages, 5 figure

Spectroscopic Sensor Data Fusion to Improve the Prediction of Soil Nutrient Contents

This study aims to advance the understanding and application of spectroscopic sensor data fusion for improving soil nutrient content predictions. In addition to presenting an extensive review of studies on the spectroscopic sensor data fusion, a research investigation was conducted to assess the effectiveness of five fusion algorithms in predicting three primary nutrients (Nitrogen, Phosphorous, and Potassium) and two secondary nutrients (Calcium and Magnesium) in soil using Visible and Near-Infrared, Mid-Infrared, and X-ray Fluorescence data. Among the five fusion algorithms, one was a low-level fusion involving data concatenation. Two were mid-level fusions, incorporating feature extraction by applying (i) Principal Component reduction and (ii) Partial Least Squares reduction. The other two were high-level fusions, namely (i) Simple Averaging and (ii) Granger Ramanathan Averaging. The results indicate that Low-Level Fusion may not be suitable for inherently incompatible data. Mid-level fusion improved the R² by 0.1-18%, RMSE by 0-8%, and RPIQ by 0-11.5%, while high-level fusion enhanced the R² by 0-12.5%, RMSE by 0-12%, and RPIQ by 2.3-13.4%, depending on the nutrients and fusion algorithms. Despite these improvements, predictions were only satisfactory for primary nutrients, and none of the algorithms could notably enhance predictions for Phosphorous. The study also finds that fusion algorithms do not significantly improve bias. The study provided evidence of improvement in prediction accuracy with data fusion which can aid in delineating management zones for precision agriculture. It also encourages further research on novel approaches of sensor fusion and algorithms that can effectively handle non-linearity introduced due to the fusion of data. Advisor: Yufeng G

Estimation of soluble solids content and fruit temperature in 'rocha' pear using Vis-NIR spectroscopy and the spectraNet–32 deep learning architecture

Spectra-based methods are becoming increasingly important in Precision Agriculture as they offer non-destructive, quick tools for measuring the quality of produce. This study introduces a novel approach for esti-mating the soluble solids content (SSC) of 'Rocha' pears using the SpectraNet-32 deep learning architecture, which operates on 1D fruit spectra in the visible to near-infrared region (Vis-NIRS). This method was also able to estimate fruit temperatures, which improved the SSC prediction performance. The dataset consisted of 3300 spectra from 1650 'Rocha' pears collected from local markets over several weeks during the 2010 and 2011 seasons, which had varying edaphoclimatic conditions. Two types of partial least squares (PLS) feature selection methods, under various configurations, were applied to the input spectra to identify the most significant wavelengths for training SpectraNet-32. The model's robustness was also compared to a similar state-of-the-art deep learning architecture, DeepSpectra, as well as four other classical machine learning algorithms: PLS, multiple linear regression (MLR), support vector machine (SVM), and multi-layer perceptron (MLP). In total, 23 different experimental method configurations were assessed, with 150 neural networks each. SpectraNet-32 consistently outperformed other methods in several metrics. On average, it was 6.1% better than PLS in terms of the root mean square error of prediction (RMSEP, 1.08 vs. 1.15%), 7.7% better in prediction gain (PG, 1.67 vs. 1.55), 3.6% better in the coefficient of determination (R2, 0.58 vs. 0.56) and 5.8% better in the coefficient of variation (CV%, 8.35 vs. 8.86).info:eu-repo/semantics/publishedVersio