LED based soil spectroscopy

Soil is a medium for plant roots to grow, absorb water and necessary solutes for growth. Soil macronutrient testing is helpful for determining the nutrient content in soil before applying fertilizer for quality and process controls of agricultural productivity and soil fertility. Spectroscopy is an emerging technology which is rapid and simple has been widely used in agricultural and food analysis processes. The capability of spectroscopy to characterize material from the transmission or absorbance has been used in this paper to measure nitrogen ( N ), phosphorus ( P ) and potassium ( K ) content in organic soil. The paper details preliminary characterization of soil spectroscopy with a Deuterium - Halogen lamp and spectrometer to measure the absorbance level of the macronutrients. The extracted nutrients were mixed with the colour reagent and specific colour ed solution was developed. The result shows high absorbance level for N and P are at 970 nm in wavelength. In addition, N gi ve absorbance at wavelength 450 nm and P yield absorbance at 800 nm wavelength . K was measured high at 620 nm. Further experiments were conducted to measure the absorbance characteristic of N , P and K for 20 minute period. The result shows that P and K has constant value of absorbance for 20 minutes duration while N , have stable absorbance value after 10 minutes being illuminated by 470 nm blue light - emitting diode ( LED ) . For future works, the optical measurements will be implemented using visible and near infrared LED and the photodetector in order to replace the spectrometer usage for soil spectroscopy. This would lead to achieve the primary objective of this research in developing a simple and low cost spectroscopy uses LED

Visible and near infrared spectroscopy in soil science

This chapter provides a review on the state of soil visible–near infrared (vis–NIR) spectroscopy. Our intention is for the review to serve as a source of up-to date information on the past and current role of vis–NIR spectroscopy in soil science. It should also provide critical discussion on issues surrounding the use of vis–NIR for soil analysis and on future directions. To this end, we describe the fundamentals of visible and infrared diffuse reflectance spectroscopy and spectroscopic multivariate calibrations. A review of the past and current role of vis–NIR spectroscopy in soil analysis is provided, focusing on important soil attributes such as soil organic matter (SOM), minerals, texture, nutrients, water, pH, and heavy metals. We then discuss the performance and generalization capacity of vis–NIR calibrations, with particular attention on sample pre-tratments, co-variations in data sets, and mathematical data preprocessing. Field analyses and strategies for the practical use of vis–NIR are considered. We conclude that the technique is useful to measure soil water and mineral composition and to derive robust calibrations for SOM and clay content. Many studies show that we also can predict properties such as pH and nutrients, although their robustness may be questioned. For future work we recommend that research should focus on: (i) moving forward with more theoretical calibrations, (ii) better understanding of the complexity of soil and the physical basis for soil reflection, and (iii) applications and the use of spectra for soil mapping and monitoring, and for making inferences about soils quality, fertility and function. To do this, research in soil spectroscopy needs to be more collaborative and strategic. The development of the Global Soil Spectral Library might be a step in the right direction

A global soil spectral calibration library and estimation service

There is growing global interest in the potential for soil reflectance spectroscopy to fill an urgent need for more data on soil properties for improved decision-making on soil security at local to global scales. This is driven by the capability of soil spectroscopy to estimate a wide range of soil properties from a rapid, inexpensive, and highly reproducible measurement using only light. However, several obstacles are preventing wider adoption of soil spectroscopy. The biggest obstacles are the large variation in the soil analytical methods and operating procedures used in different laboratories, poor reproducibility of analyses within and amongst laboratories and a lack of soil physical archives. In addition, adoption is hindered by the expense and complexity of building soil spectral libraries and calibration models. The Global Soil Spectral Calibration Library and Estimation Service is proposed to overcome these obstacles by providing a freely available estimation service based on an open, high quality and diverse spectral calibration library and the extensive soil archives of the Kellogg Soil Survey Laboratory (KSSL) of the Natural Resources Conservation Service of the United States Department of Agriculture (USDA). The initiative is supported by the Global Soil Laboratory Network (GLOSOLAN) of the Global Soil Partnership and the Soil Spectroscopy for Global Good network, which provide additional support through dissemination of standards, capacity development and research. This service is a global public good which stands to benefit soil assessments globally, but especially developing countries where soil data and resources for conventional soil analyses are most limited

Outcome Evaluation of the work of the CGIAR Research Program on Water, Land and Ecosystems (WLE) on soil and water management in Ethiopia

In 2019, the CGIAR Research Program on Water, Land and Ecosystems (WLE) Leadership chose to evaluate WLE’s work in Ethiopia as one of its countries where it has had most success. The objectives of the evaluation are: To determine how and in what ways WLE contributed to the achievement of intended/unintended outcomes; Based on the findings of the evaluation, make recommendations of how WLE (and its partners) can become more effective in supporting soil and water management in Ethiopia; To serve as a participatory learning experience for WLE and its partners. This report describes the evaluation process, findings, conclusions and recommendations

An interlaboratory comparison of mid-infrared spectra acquisition: Instruments and procedures matter

Diffuse reflectance spectroscopy has been extensively employed to deliver timely and cost-effective predictions of a number of soil properties. However, although several soil spectral laboratories have been established worldwide, the distinct characteristics of instruments and operations still hamper further integration and interoperability across mid-infrared (MIR) soil spectral libraries. In this study, we conducted a large-scale ring trial experiment to understand the lab-to-lab variability of multiple MIR instruments. By developing a systematic evaluation of different mathematical treatments with modeling algorithms, including regular preprocessing and spectral standardization, we quantified and evaluated instruments' dissimilarity and how this impacts internal and shared model performance. We found that all instruments delivered good predictions when calibrated internally using the same instruments' characteristics and standard operating procedures by solely relying on regular spectral preprocessing that accounts for light scattering and multiplicative/additive effects, e.g., using standard normal variate (SNV). When performing model transfer from a large public library (the USDA NSSC-KSSL MIR library) to secondary instruments, good performance was also achieved by regular preprocessing (e.g., SNV) if both instruments shared the same manufacturer. However, significant differences between the KSSL MIR library and contrasting ring trial instruments responses were evident and confirmed by a semi-unsupervised spectral clustering. For heavily contrasting setups, spectral standardization was necessary before transferring prediction models. Non-linear model types like Cubist and memory-based learning delivered more precise estimates because they seemed to be less sensitive to spectral variations than global partial least square regression. In summary, the results from this study can assist new laboratories in building spectroscopy capacity utilizing existing MIR spectral libraries and support the recent global efforts to make soil spectroscopy universally accessible with centralized or shared operating procedures

In-field soil spectroscopy in Vis-NIR range for fast and reliable soil analysis: A review

In-field soil spectroscopy represents a promising opportunity for fast soil analysis, allowing the prediction of several soil properties from one spectral reading representing one soil sample. This facilitates data acquisition from large amounts of samples through its rapidity and the absence of required chemical processing. This is of particular interest in agriculture, where the chance to retrieve information from soils directly in the field is very appealing. This review is focused on in-field visible to near infrared (Vis-NIR) spectroscopy (350-2500 nm), aimed at analysing soils directly in the field through proximal sensing. The main scope was to explore the available knowledge to identify existing gaps limiting the reliability and robustness of in-field measurement, to foster future research and help transition towards the practical application of this technology. For this purpose, a literature review was performed, and surveyed information encompassed sensor range, carrier platforms in use, sensor type, distance to the soil sample, measurement methodology, measured soil properties and soil management, among many others. From this, we derived a list of tools in use with their spectral measurement properties, including the potential cross-calibration with soil spectral libraries from laboratory spectroscopy of soil samples and potential measured target soil properties. Different instruments and sensors used to measure at varying wavelength ranges and with different spectral qualities are available for a large range of prices. The most frequently analysed soil properties included soil carbon contents (soil organic carbon, soil organic matter, total carbon), texture (clay, silt, sand), total nitrogen, pH and cation exchange capacity. Future perspectives comprise the implementation of larger databases, including different instruments and cropping systems as well as methodologies combining existing knowledge regarding laboratory spectroscopy with in-field methods. The authors highlight the need for a broadly accepted measurement protocol for in-field soil spectroscopy, fostering harmonization and standardization and consequently a more robust application in practice

Diffuse reflectance spectroscopy for estimating soil properties: A technology for the 21st century

Spectroscopic measurements of soil samples are reliable because they are highly repeatable and reproducible. They characterise the samples' mineral-organic composition. Estimates of concentrations of soil constituents are inevitably less precise than estimates obtained conventionally by chemical analysis. But the cost of each spectroscopic estimate is at most one-tenth of the cost of a chemical determination. Spectroscopy is cost-effective when we need many data, despite the costs and errors of calibration. Soil spectroscopists understand the risks of over-fitting models to highly dimensional multivariate spectra and have command of the mathematical and statistical methods to avoid them. Machine learning has fast become an algorithmic alternative to statistical analysis for estimating concentrations of soil constituents from reflectance spectra. As with any modelling, we need judicious implementation of machine learning as it also carries the risk of over-fitting predictions to irrelevant elements of the spectra. To use the methods confidently, we need to validate the outcomes with appropriately sampled, independent data sets. Not all machine learning should be considered 'black boxes'. Their interpretability depends on the algorithm, and some are highly interpretable and explainable. Some are difficult to interpret because of complex transformations or their huge and complicated network of parameters. But there is rapidly advancing research on explainable machine learning, and these methods are finding applications in soil science and spectroscopy. In many parts of the world, soil and environmental scientists recognise the merits of soil spectroscopy. They are building spectral libraries on which they can draw to localise the modelling and derive soil information for new projects within their domains. We hope our article gives readers a more balanced and optimistic perspective of soil spectroscopy and its future. Highlights Spectroscopy is reliable because it is a highly repeatable and reproducible analytical technique. Spectra are calibrated to estimate concentrations of soil properties with known error. Spectroscopy is cost-effective for estimating soil properties. Machine learning is becoming ever more powerful for extracting accurate information from spectra, and methods for interpreting the models exist. Large libraries of soil spectra provide information that can be used locally to aid estimates from new samples

An imperative for soil spectroscopic modelling is to think global but fit local with transfer learning

Soil spectroscopy with machine learning (ML) can estimate soil properties. Extensive soil spectral libraries (SSLs) have been developed for this purpose. However, general models built with those SSLs do not generalize well on new ‘unseen’ local data. The main reason is the different characteristics of the observations in the SSL and the local data, which cause their conditional and marginal distributions to differ. This makes the modelling of soil properties with spectra challenging. General models developed using large ‘global’ SSLs offer broad, systematic information on the soil-spectra relationships. However, to accurately generalize in a local situation, they must be adjusted to capture the site-specific characteristics of the local observations. Most current methods for ‘localizing’ spectroscopic modelling report inconsistent results. An understanding of spectroscopic ‘localization’ is lacking, and there is no framework to guide further developments. Here, we review current localization methods and propose their reformulation as a transfer learning (TL) undertaking. We then demonstrate the implementation of instance-based TL with RS-LOCAL 2.0 for modelling the soil organic carbon (SOC) content of 12 sites representing fields, farms and regions from 10 countries on the seven continents. The method uses a small number of instances or observations (measured soil property values and corresponding spectra) from the local site to transfer relevant information from a large and diverse global SSL (GSSL 2.0) with more than 50,000 records. We found that with ≤ 30 local observations, RS-LOCAL 2.0 produces more accurate and stable estimates of SOC than modelling with only the local data. Using the information in the GSSL 2.0 and reducing the number of samples for laboratory analysis, the method improves the cost-efficiency and practicality of soil spectroscopy. We interpreted the transfer by analysing the data, models, and soil and environmental relationships of the local and the ‘transferred’ data to gain insight into the approach. Transferring instances from the GSSL 2.0 to the local sites helped to align their conditional and marginal distributions, making the spectra-SOC relationships in the models more robust. Finally, we propose directions for future research. The guiding principle for developing practical and cost-effective spectroscopy should be to think globally but fit locally. By reformulating the localization problem within a TL framework, we hope to have acquainted the soil science community with a set of methodologies that can inspire the development of new, innovative algorithms for soil spectroscopic modelling

Quantitative Mapping of Soil Property Based on Laboratory and Airborne Hyperspectral Data Using Machine Learning

Soil visible and near-infrared spectroscopy provides a non-destructive, rapid and low-cost approach to quantify various soil physical and chemical properties based on their reflectance in the spectral range of 400–2500 nm. With an increasing number of large-scale soil spectral libraries established across the world and new space-borne hyperspectral sensors, there is a need to explore methods to extract informative features from reflectance spectra and produce accurate soil spectroscopic models using machine learning. Features generated from regional or large-scale soil spectral data play a key role in the quantitative spectroscopic model for soil properties. The Land Use/Land Cover Area Frame Survey (LUCAS) soil library was used to explore PLS-derived components and fractal features generated from soil spectra in this study. The gradient-boosting method performed well when coupled with extracted features on the estimation of several soil properties. Transfer learning based on convolutional neural networks (CNNs) was proposed to make the model developed from laboratory data transferable for airborne hyperspectral data. The soil clay map was successfully derived using HyMap imagery and the fine-tuned CNN model developed from LUCAS mineral soils, as deep learning has the potential to learn transferable features that generalise from the source domain to target domain. The external environmental factors like the presence of vegetation restrain the application of imaging spectroscopy. The reflectance data can be transformed into a vegetation suppressed domain with a force invariance approach, the performance of which was evaluated in an agricultural area using CASI airborne hyperspectral data. However, the relationship between vegetation and acquired spectra is complicated, and more efforts should put on removing the effects of external factors to make the model transferable from one sensor to another.:Abstract I Kurzfassung III Table of Contents V List of Figures IX List of Tables XIII List of Abbreviations XV 1 Introduction 1 1.1 Motivation 1 1.2 Soil spectra from different platforms 2 1.3 Soil property quantification using spectral data 4 1.4 Feature representation of soil spectra 5 1.5 Objectives 6 1.6 Thesis structure 7 2 Combining Partial Least Squares and the Gradient-Boosting Method for Soil Property Retrieval Using Visible Near-Infrared Shortwave Infrared Spectra 9 2.1 Abstract 10 2.2 Introduction 10 2.3 Materials and methods 13 2.3.1 The LUCAS soil spectral library 13 2.3.2 Partial least squares algorithm 15 2.3.3 Gradient-Boosted Decision Trees 15 2.3.4 Calculation of relative variable importance 16 2.3.5 Assessment 17 2.4 Results 17 2.4.1 Overview of the spectral measurement 17 2.4.2 Results of PLS regression for the estimation of soil properties 19 2.4.3 Results of PLS-GBDT for the estimation of soil properties 21 2.4.4 Relative important variables derived from PLS regression and the gradient-boosting method 24 2.5 Discussion 28 2.5.1 Dimension reduction for high-dimensional soil spectra 28 2.5.2 GBDT for quantitative soil spectroscopic modelling 29 2.6 Conclusions 30 3 Quantitative Retrieval of Organic Soil Properties from Visible Near-Infrared Shortwave Infrared Spectroscopy Using Fractal-Based Feature Extraction 31 3.1 Abstract 32 3.2 Introduction 32 3.3 Materials and Methods 35 3.3.1 The LUCAS topsoil dataset 35 3.3.2 Fractal feature extraction method 37 3.3.3 Gradient-boosting regression model 37 3.3.4 Evaluation 41 3.4 Results 42 3.4.1 Fractal features for soil spectroscopy 42 3.4.2 Effects of different step and window size on extracted fractal features 45 3.4.3 Modelling soil properties with fractal features 47 3.4.3 Comparison with PLS regression 49 3.5 Discussion 51 3.5.1 The importance of fractal dimension for soil spectra 51 3.5.2 Modelling soil properties with fractal features 52 3.6 Conclusions 53 4 Transfer Learning for Soil Spectroscopy Based on Convolutional Neural Networks and Its Application in Soil Clay Content Mapping Using Hyperspectral Imagery 55 4.1 Abstract 55 4.2 Introduction 56 4.3 Materials and Methods 59 4.3.1 Datasets 59 4.3.2 Methods 62 4.3.3 Assessment 67 4.4 Results and Discussion 67 4.4.1 Interpretation of mineral and organic soils from LUCAS dataset 67 4.4.2 1D-CNN and spectral index for LUCAS soil clay content estimation 69 4.4.3 Application of transfer learning for soil clay content mapping using the pre-trained 1D-CNN model 72 4.4.4 Comparison between spectral index and transfer learning 74 4.4.5 Large-scale soil spectral library for digital soil mapping at the local scale using hyperspectral imagery 75 4.5 Conclusions 75 5 A Case Study of Forced Invariance Approach for Soil Salinity Estimation in Vegetation-Covered Terrain Using Airborne Hyperspectral Imagery 77 5.1 Abstract 78 5.2 Introduction 78 5.3 Materials and Methods 81 5.3.1 Study area of Zhangye Oasis 81 5.3.2 Data description 82 5.3.3 Methods 83 5.3.3 Model performance assessment 85 5.4 Results and Discussion 86 5.4.1 The correlation between NDVI and soil salinity 86 5.4.2 Vegetation suppression performance using the Forced Invariance Approach 86 5.4.3 Estimation of soil properties using airborne hyperspectral data 88 5.5 Conclusions 90 6 Conclusions and Outlook 93 Bibliography 97 Acknowledgements 11