Visible and near infrared spectroscopy in soil science

This chapter provides a review on the state of soil visible–near infrared (vis–NIR) spectroscopy. Our intention is for the review to serve as a source of up-to date information on the past and current role of vis–NIR spectroscopy in soil science. It should also provide critical discussion on issues surrounding the use of vis–NIR for soil analysis and on future directions. To this end, we describe the fundamentals of visible and infrared diffuse reflectance spectroscopy and spectroscopic multivariate calibrations. A review of the past and current role of vis–NIR spectroscopy in soil analysis is provided, focusing on important soil attributes such as soil organic matter (SOM), minerals, texture, nutrients, water, pH, and heavy metals. We then discuss the performance and generalization capacity of vis–NIR calibrations, with particular attention on sample pre-tratments, co-variations in data sets, and mathematical data preprocessing. Field analyses and strategies for the practical use of vis–NIR are considered. We conclude that the technique is useful to measure soil water and mineral composition and to derive robust calibrations for SOM and clay content. Many studies show that we also can predict properties such as pH and nutrients, although their robustness may be questioned. For future work we recommend that research should focus on: (i) moving forward with more theoretical calibrations, (ii) better understanding of the complexity of soil and the physical basis for soil reflection, and (iii) applications and the use of spectra for soil mapping and monitoring, and for making inferences about soils quality, fertility and function. To do this, research in soil spectroscopy needs to be more collaborative and strategic. The development of the Global Soil Spectral Library might be a step in the right direction

Comparison of different multivariate calibrations and ensemble methods for estimating selected soil properties with vis-NIR reflectance spectroscopy

Sustainable soil management requires a correct assessment of soil chemi- cal and physical properties. Historically, this has been gained through conventional laboratory analyses, which are considered costly and time-consuming, particularly when a large number of soil samples need to be analysed. An alternative, faster and less expensive, approach is based on the use of reflectance spectroscopy in the vis- NIR domain. This approach implies the calibration of predictive models that relate the spectral reflectance to soil properties. The goodness of the models can be partic- ularly influenced by the multivariate methods used. In this article, we compare the performance of different multivariate and statistical ensemble methods for estimating some basic soil properties, such as sand, silt, clay, and organic carbon in the specific pedo-environmental conditions of an important agricultural area in southern Italy

Rapid identification of oil contaminated soils using visible near infrared diffuse reflectance spectroscopy

Initially, 46 petroleum contaminated and non-contaminated soil samples were collected and scanned using visible near-infrared diffuse reflectance spectroscopy (VisNIR DRS) at three combinations of moisture content and pretreatment. The VisNIR spectra of soil samples were used to predict total petroleum hydrocarbon (TPH) content using partial least squares (PLS) regression and boosted regression tree (BRT) models. The field-moist intact scan proved best for predicting TPH content with a validation r2 of 0.64 and relative percent difference (RPD) of 1.70. Those 46 samples were used to calibrate a penalized spline (PS) model. Subsequently, the PS model was used to predict soil TPH content for 128 soil samples collected over an 80 ha study site. An exponential semivariogram using PS predictions revealed strong spatial dependence among soil TPH [r2 = 0.76, range = 52 m, nugget = 0.001 (log10 mg kg-1)2, and sill 1.044 (log10 mg kg-1)2]. An ordinary block kriging map produced from the data showed that TPH distribution matched the expected TPH variability of the study site. Another study used DRS to measure reflectance patterns of 68 artificially constructed samples with different clay content, organic carbon levels, petroleum types, and different levels of contamination per type. Both first derivative of reflectance and discrete wavelet transformations were used to preprocess the spectra. Principal component analysis (PCA) was applied for qualitative VisNIR discrimination of variable soil types, organic carbon levels, petroleum types, and concentration levels. Soil types were separated with 100% accuracy, and organic carbon levels were separated with 96% accuracy by linear discriminant analysis. The support vector machine produced 82% classification accuracy for organic carbon levels by repeated random splitting of the whole dataset. However, spectral absorptions for each petroleum hydrocarbon overlapped with each other and could not be separated with any classification scheme when contaminations were mixed. Wavelet-based multiple linear regression performed best for predicting petroleum amount with the highest residual prediction deviation (RPD) of 3.97. While using the first derivative of reflectance spectra, PS regression performed better (RPD = 3.3) than the PLS (RPD= 2.5) model. Specific calibrations considering additional soil physicochemical variability are recommended to produce improved predictions

Proximal sensing in soil profiles

Objective and quantitative soil information is crucial for pedological investigations and to inform diverse decision making processes. New techniques are required so that soil information can be ascertained in a timely manner to support sampling at finer spatial and temporal resolutions. Currently, no single technique can provide information on all of the properties of interest. This research investigated the conjoint use of visible near-infrared diffuse reflectance spectroscopy (VisNIR) and portable X-ray fluorescence spectroscopy (pXRF) for the in situ investigation of soil properties, profile variability and description. Fifteen soil pits across New South Wales, Australia, were selected for their diverse representation of soil properties. Sampling at these sites involved scanning three vertical with sensor readings taken at 2.5 cm intervals to a depth of 1 m within each transect. Soils were described by traditional pit description techniques and horizon based sampling was conducted to characterise the soil in terms of mineral composition, OC, TC, TN, CEC, EC, pH and PSA. A data fusion approach involving model averaging, and a mass balance was implemented to characterise the mineral composition of soils, including phyllosilicates sesquioxides, carbonate, gypsum, quartz and feldspars. Results were validated against X-ray diffraction analysis. To explore the predictive capability of scans taken in situ, existing spectral libraries were used to calibrate VisNIR and pXRF models and identify the best use of proximal sensor data to maximise soil information gain. As not all properties of interest have detectable spectral activity by either VisNIR or pXRF, a spectral soil inference system (SPEC-SINFERS) to augment the number of predicted properties. This system involved the propagation of sensor and model uncertainties through one hundred independent simulations for each calculation, and allowed the integration of both regression models and machine learning techniques

Machine learning based prediction of soil total nitrogen, organic carbon and moisture content by using VIS-NIR spectroscopy

It is widely known that the visible and near infrared (VIS-NIR) spectroscopy has the potential of estimating soil total nitrogen (TN), organic carbon (OC) and moisture content (MC) due to the direct spectral responses these properties have in the near infrared (NIR) region. However, improving the prediction accuracy requires advanced modelling techniques, particularly when measurement is planned for fresh (wet and un-processed) soil samples. The aim of this work is to compare the predictive performance of two linear multivariate and two machine learning methods for TN, OC and MC. The two multivariate methods investigated included principal component regression (PCR) and partial least squares regression (PLSR), whereas the machine learning methods included least squares support vector machines (LS-SVM), and Cubist. A mobile, fibre type, VIS-NIR spectrophotometer was utilised to collect soil spectra (305–2200 nm) in diffuse reflectance mode from 140 wet soil samples collected from one field in Germany. The results indicate that machine learning methods are capable of tackling non-linear problems in the dataset. LS-SVMs and the Cubist method out-performed the linear multivariate methods for the prediction of all three soil properties studied. LS-SVM provided the best prediction for MC (root mean square error of prediction (RMSEP) = 0.457% and residual prediction deviation (RPD) = 2.24) and OC (RMSEP = 0.062% and RPD = 2.20), whereas the Cubist method provided the best prediction for TN (RMSEP = 0.071 and RPD = 1.96)

Rapid detection of soil carbonates by means of NIR spectroscopy, deep learning methods and phase quantification by powder Xray diffraction

Soil NIR spectral absorbance/reflectance libraries are utilized towards improving agricultural production and analysis of soil properties which are key prerequisite for agroecological balance and environmental sustainability. Carbonates in particular, represent a soil property which is mostly affected even by mild, let alone extreme, changes of environmental conditions during climate change. In this study we propose a rapid and efficient way to predict carbonates content in soil by means of FT NIR reflectance spectroscopy and by use of deep learning methods. We exploited multiple machine learning methods, such as: 1) a MLP Regressor and 2) a CNN and compare their performance with other traditional ML algorithms such as PLSR, Cubist and SVM on the combined dataset of two NIR spectral libraries: KSSL (USDA), a dataset of soil samples reflectance spectra collected nationwide, and LUCAS TopSoil (European Soil Library) which contains soil sample absorbance spectra from all over the European Union, and use them to predict carbonate content on never before seen soil samples. Soil samples in KSSL and in TopSoil spectral libraries were acquired in the spectral region of visNIR, however in this study, only the NIR spectral region was utilized. Quantification of carbonates by means of Xray Diffraction is in good agreement with the volumetric method and the MLP prediction. Our work contributes to rapid carbonates content prediction in soil samples in cases where: 1) no volumetric method is available and 2) only NIR spectra absorbance data are available. Up till now and to the best of our knowledge, there exists no other study, that presents a prediction model trained on such an extensive dataset with such promising results on unseen data, undoubtedly supporting the notion that deep learning models present excellent prediction tools for soil carbonates content.Comment: 39 pages, 5 figure