Comparison between random forests, artificial neural networks and gradient boosted machines methods of on-line vis-NIR spectroscopy measurements of soil total nitrogen and total carbon

Accurate and detailed spatial soil information about within-field variability is essential for variable-rate applications of farm resources. Soil total nitrogen (TN) and total carbon (TC) are important fertility parameters that can be measured with on-line (mobile) visible and near infrared (vis-NIR) spectroscopy. This study compares the performance of local farm scale calibrations with those based on the spiking of selected local samples from both fields into an European dataset for TN and TC estimation using three modelling techniques, namely gradient boosted machines (GBM), artificial neural networks (ANNs) and random forests (RF). The on-line measurements were carried out using a mobile, fiber type, vis-NIR spectrophotometer (305-2200 nm) (AgroSpec from tec5, Germany), during which soil spectra were recorded in diffuse reflectance mode from two fields in the UK. After spectra pre-processing, the entire datasets were then divided into calibration (75%) and prediction (25%) sets, and calibration models for TN and TC were developed using GBM, ANN and RF with leave-one-out cross-validation. Results of cross-validation showed that the effect of spiking of local samples collected from a field into an European dataset when combined with RF has resulted in the highest coefficients of determination (R-2) values of 0.97 and 0.98, the lowest root mean square error (RMSE) of 0.01% and 0.10%, and the highest residual prediction deviations (RPD) of 5.58 and 7.54, for TN and TC, respectively. Results for laboratory and on-line predictions generally followed the same trend as for cross-validation in one field, where the spiked European dataset-based RF calibration models outperformed the corresponding GBM and ANN models. In the second field ANN has replaced RF in being the best performing. However, the local field calibrations provided lower R-2 and RPD in most cases. Therefore, from a cost-effective point of view, it is recommended to adopt the spiked European dataset-based RF/ANN calibration models for successful prediction of TN and TC under on-line measurement conditions

Estimation of secondary soil properties by fusion of laboratory and on-line measured vis-NIR spectra

Visible and near infrared (vis-NIR) diffuse reflectance spectroscopy has made invaluable contributions to the accurate estimation of soil properties having direct and indirect spectral responses in NIR spectroscopy with measurements made in laboratory, in situ or using on-line (while the sensor is moving) platforms. Measurement accuracies vary with measurement type, for example, accuracy is higher for laboratory than on-line modes. On-line measurement accuracy deteriorates further for secondary (having indirect spectral response) soil properties. Therefore, the aim of this study is to improve on-line measurement accuracy of secondary properties by fusion of laboratory and on-line scanned spectra. Six arable fields were scanned using an on-line sensing platform coupled with a vis-NIR spectrophotometer (CompactSpec by Tec5 Technology for spectroscopy, Germany), with a spectral range of 305-1700 nm. A total of 138 soil samples were collected and used to develop five calibration models: (i) standard, using 100 laboratory scanned samples; (ii) hybrid-1, using 75 laboratory and 25 on-line samples; (iii) hybrid-2, using 50 laboratory and 50 on-line samples; (iv) hybrid-3, using 25 laboratory and 75 on-line samples, and (v) real-time using 100 on-line samples. Partial least squares regression (PLSR) models were developed for soil pH, available potassium (K), magnesium (Mg), calcium (Ca), and sodium (Na) and quality of models were validated using an independent prediction dataset (38 samples). Validation results showed that the standard models with laboratory scanned spectra provided poor to moderate accuracy for on-line prediction, and the hybrid-3 and real-time models provided the best prediction results, although hybrid-2 model with 50% on-line spectra provided equally good results for all properties except for pH and Na. These results suggest that either the real-time model with exclusively on-line spectra or the hybrid model with fusion up to 50% (except for pH and Na) and 75% on-line scanned spectra allows significant improvement of on-line prediction accuracy for secondary soil properties using vis-NIR spectroscopy

Development of a framework for the evaluation of the environmental benefits of controlled traffic farming

Although controlled traffic farming (CTF) is an environmentally friendly soil management system, no quantitative evaluation of environmental benefits is available. This paper aims at establishing a framework for quantitative evaluation of the environmental benefits of CTF, considering a list of environmental benefits, namely, reducing soil compaction, runoff/erosion, energy requirement and greenhouse gas emission (GHG), conserving organic matter, enhancing soil biodiversity and fertiliser use efficiency. Based on a comprehensive literature review and the European Commission Soil Framework Directive, the choice of and the weighting of the impact of each of the environmental benefits were made. The framework was validated using data from three selected farms. For Colworth farm (Unilever, UK), the framework predicted the largest overall environmental benefit of 59.3% of the theoretically maximum achievable benefits (100%), as compared to the other two farms in Scotland (52%) and Australia (47.3%). This overall benefit could be broken down into: reducing soil compaction (24%), tillage energy requirement (10%) and GHG emissions (3%), enhancing soil biodiversity (7%) and erosion control (6%), conserving organic matter (6%), and improving fertiliser use efficiency (3%). Similar evaluation can be performed for any farm worldwide, providing that data on soil properties, topography, machinery, and weather are available

Online measurement of soil organic carbon as correlated with wheat normalised difference vegetation index in a vertisol field

This study explores the potential of visible and near infrared (vis-NIR) spectroscopy for onlinemeasurement of soil organic carbon (SOC). It also attempts to explore correlations and similarities between the spatial distribution of SOC and normalized differential vegetation index (NDVI) of a wheat crop. The online measurement was carried out in a clay vertisol field covering 10 ha of area in Karacabey, Bursa, Turkey. Kappa statistics were carried out between different SOC and NDVI data to investigate potential similarities. Calibration model of SOC in full cross-validationresulted in a good accuracy ( 2 = 0.75, root mean squares error of prediction (RMSEP) = 0.17%, and ratio of prediction deviation (RPD) = 1.81). The validation of the calibration model using laboratory spectra provided comparatively better prediction accuracy ( 2 = 0.70, RMSEP = 0.15%, and RPD = 1.78), as compared to the online measured spectra ( 2 = 0.60, RMSEP = 0.20%, and RPD = 1.41). Although visual similarity was clear, low similarity indicated by a lowKappa value of 0.259 was observed between the online vis-NIR predicted full-point (based on all pointsmeasured in the field, e.g., 6486 points)map of SOC and NDVI map

Machine learning based on-line prediction of soil organic carbon after removal of soil moisture effect

It is well-documented in the visible and near-infrared reflectance spectroscopy (VNIRS) studies that soil moisture content (SMC) negatively affects the prediction accuracy of soil attributes. This work was undertaken to remove the negative effect of SMC on the on-line prediction of soil organic carbon (SOC). A mobile VNIR spectrophotometer with a spectral range of 305-1700 nm and spectral resolution of 1 nm (CompactSpec, Tec5 Technology, Germany) was used for the spectral measurements at four farms in Flanders, Belgium. A total of 381 fresh soil samples were collected and divided into a calibration set (264) and a validation set (117). The validation samples were processed (air-dried and grind) and scanned with the same spectrophotometer in the laboratory. Three SMC correction methods, namely, external parameter orthogonalization (EPO), piecewise direct standardization (PDS), and orthogonal signal correction (OSC) were used to correct the on-line fresh spectra based-on its corresponding laboratory spectra. Then, the Cubist machine learning method was used to develop calibration models of SOC using the on-line spectra (after correction) of the calibration set. Results indicated that the EPO-Cubist outperformed the PDS-Cubist and the OSC-Cubist, with considerable improvements in the prediction results of SOC (coefficient of determination (R-2) = 0.76, ratio of performance to deviation (RPD) = 2.08, and root mean square error of prediction (RMSEP) = 0.12%), compared with the corresponding uncorrected on-line spectra (R-2 = 0.55, RPD = 1.24, and RMSEP = 0.20%). It can be concluded that SOC can be accurately predicted on-line using the Cubist machine learning method, after removing the negative effect of SMC with the EPO method