Evaluation of Airborne HySpex and Spaceborne PRISMA Hyperspectral Remote Sensing Data for Soil Organic Matter and Carbonates Estimation

Remote sensing and soil spectroscopy applications are valuable techniques for soil property estimation. Soil organic matter (SOM) and calcium carbonate are important factors in soil quality, and although organic matter is well studied, calcium carbonates require more investigation. In this study, we validated the performance of laboratory soil spectroscopy for estimating the aforementioned properties with referenced in situ data. We also examined the performance of imaging spectroscopy sensors, such as the airborne HySpex and the spaceborne PRISMA. For this purpose, we applied four commonly used machine learning algorithms and six preprocessing methods for the evaluation of the best fitting algorithm.. The study took place over crop areas of Amyntaio in Northern Greece, where extensive soil sampling was conducted. This is an area with a very variable mineralogical environment (from lignite mine to mountainous area). The SOM results were very good at the laboratory scale and for both remote sensing sensors with R2 = 0.79 for HySpex and R2 = 0.76 for PRISMA. Regarding the calcium carbonate estimations, the remote sensing accuracy was R2 = 0.82 for HySpex and R2 = 0.36 for PRISMA. PRISMA was still in the commissioning phase at the time of the study, and therefore, the acquired image did not cover the whole study area. Accuracies for calcium carbonates may be lower due to the smaller sample size used for the modeling procedure. The results show the potential for using quantitative predictions of SOM and the carbonate content based on soil and imaging spectroscopy at the air and spaceborne scales and for future applications using larger datasets

An intelligent System for Soil Classification using Supervised Learning Approach

Agriculturist or farmers collect soil sample that are later analyzed for proper classification. This conventional procedure is labour intensive, time consuming and expensive. In this research work, an attempt was made to develop an intelligent system that can identify different types of soil in a particular location using the available hyperspectral data at such location with supervised learning approach. The system was developed using fuzzy –C means to identify the cluster centre. The cluster center was used as an input to train KSOM and generate soil prediction map as an output. ANFIS was eventually used to identify each class of the soil using the soil predictor map as an output during the training stage. The system was implemented using R programming Language. Keywords: Hyperspectral data, C-Means Clustering, KSOM, ANFIS, Supervised Learning, Intelligent System

Quantitative Mapping of Soil Property Based on Laboratory and Airborne Hyperspectral Data Using Machine Learning

Soil visible and near-infrared spectroscopy provides a non-destructive, rapid and low-cost approach to quantify various soil physical and chemical properties based on their reflectance in the spectral range of 400–2500 nm. With an increasing number of large-scale soil spectral libraries established across the world and new space-borne hyperspectral sensors, there is a need to explore methods to extract informative features from reflectance spectra and produce accurate soil spectroscopic models using machine learning. Features generated from regional or large-scale soil spectral data play a key role in the quantitative spectroscopic model for soil properties. The Land Use/Land Cover Area Frame Survey (LUCAS) soil library was used to explore PLS-derived components and fractal features generated from soil spectra in this study. The gradient-boosting method performed well when coupled with extracted features on the estimation of several soil properties. Transfer learning based on convolutional neural networks (CNNs) was proposed to make the model developed from laboratory data transferable for airborne hyperspectral data. The soil clay map was successfully derived using HyMap imagery and the fine-tuned CNN model developed from LUCAS mineral soils, as deep learning has the potential to learn transferable features that generalise from the source domain to target domain. The external environmental factors like the presence of vegetation restrain the application of imaging spectroscopy. The reflectance data can be transformed into a vegetation suppressed domain with a force invariance approach, the performance of which was evaluated in an agricultural area using CASI airborne hyperspectral data. However, the relationship between vegetation and acquired spectra is complicated, and more efforts should put on removing the effects of external factors to make the model transferable from one sensor to another.:Abstract I Kurzfassung III Table of Contents V List of Figures IX List of Tables XIII List of Abbreviations XV 1 Introduction 1 1.1 Motivation 1 1.2 Soil spectra from different platforms 2 1.3 Soil property quantification using spectral data 4 1.4 Feature representation of soil spectra 5 1.5 Objectives 6 1.6 Thesis structure 7 2 Combining Partial Least Squares and the Gradient-Boosting Method for Soil Property Retrieval Using Visible Near-Infrared Shortwave Infrared Spectra 9 2.1 Abstract 10 2.2 Introduction 10 2.3 Materials and methods 13 2.3.1 The LUCAS soil spectral library 13 2.3.2 Partial least squares algorithm 15 2.3.3 Gradient-Boosted Decision Trees 15 2.3.4 Calculation of relative variable importance 16 2.3.5 Assessment 17 2.4 Results 17 2.4.1 Overview of the spectral measurement 17 2.4.2 Results of PLS regression for the estimation of soil properties 19 2.4.3 Results of PLS-GBDT for the estimation of soil properties 21 2.4.4 Relative important variables derived from PLS regression and the gradient-boosting method 24 2.5 Discussion 28 2.5.1 Dimension reduction for high-dimensional soil spectra 28 2.5.2 GBDT for quantitative soil spectroscopic modelling 29 2.6 Conclusions 30 3 Quantitative Retrieval of Organic Soil Properties from Visible Near-Infrared Shortwave Infrared Spectroscopy Using Fractal-Based Feature Extraction 31 3.1 Abstract 32 3.2 Introduction 32 3.3 Materials and Methods 35 3.3.1 The LUCAS topsoil dataset 35 3.3.2 Fractal feature extraction method 37 3.3.3 Gradient-boosting regression model 37 3.3.4 Evaluation 41 3.4 Results 42 3.4.1 Fractal features for soil spectroscopy 42 3.4.2 Effects of different step and window size on extracted fractal features 45 3.4.3 Modelling soil properties with fractal features 47 3.4.3 Comparison with PLS regression 49 3.5 Discussion 51 3.5.1 The importance of fractal dimension for soil spectra 51 3.5.2 Modelling soil properties with fractal features 52 3.6 Conclusions 53 4 Transfer Learning for Soil Spectroscopy Based on Convolutional Neural Networks and Its Application in Soil Clay Content Mapping Using Hyperspectral Imagery 55 4.1 Abstract 55 4.2 Introduction 56 4.3 Materials and Methods 59 4.3.1 Datasets 59 4.3.2 Methods 62 4.3.3 Assessment 67 4.4 Results and Discussion 67 4.4.1 Interpretation of mineral and organic soils from LUCAS dataset 67 4.4.2 1D-CNN and spectral index for LUCAS soil clay content estimation 69 4.4.3 Application of transfer learning for soil clay content mapping using the pre-trained 1D-CNN model 72 4.4.4 Comparison between spectral index and transfer learning 74 4.4.5 Large-scale soil spectral library for digital soil mapping at the local scale using hyperspectral imagery 75 4.5 Conclusions 75 5 A Case Study of Forced Invariance Approach for Soil Salinity Estimation in Vegetation-Covered Terrain Using Airborne Hyperspectral Imagery 77 5.1 Abstract 78 5.2 Introduction 78 5.3 Materials and Methods 81 5.3.1 Study area of Zhangye Oasis 81 5.3.2 Data description 82 5.3.3 Methods 83 5.3.3 Model performance assessment 85 5.4 Results and Discussion 86 5.4.1 The correlation between NDVI and soil salinity 86 5.4.2 Vegetation suppression performance using the Forced Invariance Approach 86 5.4.3 Estimation of soil properties using airborne hyperspectral data 88 5.5 Conclusions 90 6 Conclusions and Outlook 93 Bibliography 97 Acknowledgements 11

Rapid detection of soil carbonates by means of NIR spectroscopy, deep learning methods and phase quantification by powder Xray diffraction

Soil NIR spectral absorbance/reflectance libraries are utilized towards improving agricultural production and analysis of soil properties which are key prerequisite for agroecological balance and environmental sustainability. Carbonates in particular, represent a soil property which is mostly affected even by mild, let alone extreme, changes of environmental conditions during climate change. In this study we propose a rapid and efficient way to predict carbonates content in soil by means of FT NIR reflectance spectroscopy and by use of deep learning methods. We exploited multiple machine learning methods, such as: 1) a MLP Regressor and 2) a CNN and compare their performance with other traditional ML algorithms such as PLSR, Cubist and SVM on the combined dataset of two NIR spectral libraries: KSSL (USDA), a dataset of soil samples reflectance spectra collected nationwide, and LUCAS TopSoil (European Soil Library) which contains soil sample absorbance spectra from all over the European Union, and use them to predict carbonate content on never before seen soil samples. Soil samples in KSSL and in TopSoil spectral libraries were acquired in the spectral region of visNIR, however in this study, only the NIR spectral region was utilized. Quantification of carbonates by means of Xray Diffraction is in good agreement with the volumetric method and the MLP prediction. Our work contributes to rapid carbonates content prediction in soil samples in cases where: 1) no volumetric method is available and 2) only NIR spectra absorbance data are available. Up till now and to the best of our knowledge, there exists no other study, that presents a prediction model trained on such an extensive dataset with such promising results on unseen data, undoubtedly supporting the notion that deep learning models present excellent prediction tools for soil carbonates content.Comment: 39 pages, 5 figure

Ensemble Identification of Spectral Bands Related to Soil Organic Carbon Levels over an Agricultural Field in Southern Ontario, Canada

The recent use of hyperspectral remote sensing imagery has introduced new opportunities for soil organic carbon (SOC) assessment and monitoring. These data enable monitoring of a wide variety of soil properties but pose important methodological challenges. Highly correlated hyperspectral spectral bands can affect the prediction and accuracy as well as the interpretability of the retrieval model. Therefore, the spectral dimension needs to be reduced through a selection of specific spectral bands or regions that are most helpful to describing SOC. This study evaluates the efficiency of visible near-infrared (VNIR) and shortwave near-infrared (SWIR) hyperspectral data to identify the most informative hyperspectral bands responding to SOC content in agricultural soils. Soil samples (111) were collected over an agricultural field in southern Ontario, Canada and analyzed against two hyperspectral datasets: An airborne Nano-Hyperspec imaging sensor with 270 bands (400–1000 nm) and a laboratory hyperspectral dataset (ASD FieldSpec 3) along the 1000–2500 nm range (NIR-SWIR). In parallel, a multimethod modeling approach consisting of random forest, support vector machine, and partial least squares regression models was used to conduct band selections and to assess the validity of the selected bands. The multimethod model resulted in a selection of optimal band or regions over the VNIR and SWIR sensitive to SOC and potentially for mapping. The bands that achieved the highest respective importance values were 711–715, 727, 986–998, and 433–435 nm regions (VNIR); and 2365–2373, 2481–2500, and 2198–2206 nm (NIR-SWIR). Some of these bands are in agreement with the absorption features of SOC reported in the literature, whereas others have not been reported before. Ultimately, the selection of optimal band and regions is of importance for quantification of agricultural SOC and would provide a new framework for creating optimized SOC-specific sensors

Sensors in agriculture and forestry

Agriculture and Forestry are two broad and promising areas demanding technological solutions with the aim of increasing production or accurate inventories for sustainability while the environmental impact is minimized by reducing the application of agro-chemicals and increasing the use of environmental friendly agronomical practices. In addition, the immediate consequence of this “trend” is the reduction of production costs. Sensors-based technologies provide appropriate tools to achieve the above mentioned goals. The explosive technological advances and development in recent years enormously facilitates the attainment of these objectives removing many barriers for their implementation, including the reservations expressed by the farmers themselves. Precision Agriculture is an emerging area where sensor-based technologies play an important role.RHEA project [42], which is funded by the European Union’s Seventh Framework Programme (FP7/2007-2013) under Grant Agreement NO.245986, which has been the platform for the two international conferences on Robotics and associated High-technologies and Equipment mentioned above.Peer Reviewe

Significance of Soil Lightness Versus Physicochemical Soil Properties in Semiarid Areas

This is an author's accepted manuscript of an article published in " Arid Land Research and Management"; Volume 28, Issue 4, 2014; copyright Taylor & Francis; available online at: http://www.tandfonline.com/doi/abs/10.1080/15324982.2014.882871Modern agriculture aims to encompass all soil attributes to optimize soil use and minimize environmental impacts. One of those attributes is soil color, which allows determining important soil variables for crop management and soil conservation. In this study, the relationships between lightness and several pedologic, topographic, and soil management variables were determined. One hundred and ten topsoil points were sampled in an area where the Mediterranean climate is the only homogeneous soil forming factor. Soil samples were air dried, crushed, and sieved, and lightness measurements were made using a trichromatic colorimeter. The relationships between lightness and soil-related parameters were carried out by means of bivariate linear correlation, and Mann-Witney and Kruskal-Wallis tests. Soil textural fractions (sand and silt), electrical conductivity and carbonates were statistically significant (p<0.001) and exhibited moderate correlation coefficients (0.32 0.45). Topographic variables (slope and aspect), soil organic carbon, iron, nitrogen, pH, and parent material (marls) exhibited lower effect on lightness. The response of lightness to clay content was highly conditioned by iron content. Stoniness, phosphorous, elevation, and soil management variables (irrigation and land use) were not statistically significant. The results obtained with calcareous samples from semiarid areas showed that soil lightness behavior agree with findings in developed soils, despite of the large differences in soil composition and the heterogeneity of the study area.Moreno-Ramón, H.; Marqués-Mateu, Á.; Ibañez Asensio, S. (2014). Significance of Soil Lightness Versus Physicochemical Soil Properties in Semiarid Areas. Arid Land Research and Management. 28(4):371-382. doi:10.1080/15324982.2014.882871S371382284Al-Mahawili , S. M. H. , M. F. Baumgardner , R. A. Weismiller , and W. N. Melhorn . 1983 . Satellite image interpretation and laboratory spectral reflectance measurements of saline and gypsiferous soils of West Baghdad, Iraq.LARS Technical Reports. Paper 104.Barrett, L. R. (2002). Spectrophotometric color measurement in situ in well drained sandy soils. Geoderma, 108(1-2), 49-77. doi:10.1016/s0016-7061(02)00121-0Bogrekci, I., & Lee, W. S. (2005). Spectral Phosphorus Mapping using Diffuse Reflectance of Soils and Grass. Biosystems Engineering, 91(3), 305-312. doi:10.1016/j.biosystemseng.2005.04.015Buol, S. W., Southard, R. J., Graham, R. C., & McDaniel, P. A. (2011). Soil Genesis and Classification. doi:10.1002/9780470960622Christensen, L. K., Bennedsen, B. S., Jørgensen, R. N., & Nielsen, H. (2004). Modelling Nitrogen and Phosphorus Content at Early Growth Stages in Spring Barley using Hyperspectral Line Scanning. Biosystems Engineering, 88(1), 19-24. doi:10.1016/j.biosystemseng.2004.02.006Doi, R., Wachrinrat, C., Teejuntuk, S., Sakurai, K., & Sahunalu, P. (2009). Semiquantitative color profiling of soils over a land degradation gradient in Sakaerat, Thailand. Environmental Monitoring and Assessment, 170(1-4), 301-309. doi:10.1007/s10661-009-1233-xDuiker, S. W., Rhoton, F. E., Torrent, J., Smeck, N. E., & Lal, R. (2003). Iron (Hydr)Oxide Crystallinity Effects on Soil Aggregation. Soil Science Society of America Journal, 67(2), 606. doi:10.2136/sssaj2003.0606Ehsani, M. R., Upadhyaya, S. K., Slaughter, D., Shafii, S., & Pelletier, M. (1999). Precision Agriculture, 1(2), 219-236. doi:10.1023/a:1009916108990Gunal, H., Ersahin, S., Yetgin, B., & Kutlu, T. (2008). Use of Chromameter‐Measured Color Parameters in Estimating Color‐Related Soil Variables. Communications in Soil Science and Plant Analysis, 39(5-6), 726-740. doi:10.1080/00103620701879422Ibarra-F., F. A., Martin-R., M. H., Cox, J. R., Crowl, T. A., Post, D. F., Miller, R. W., & Rasmussen, G. A. (1995). Relationship between Buffelgrass Survival, Organic Carbon, and Soil Color in Mexico. Soil Science Society of America Journal, 59(4), 1120. doi:10.2136/sssaj1995.03615995005900040025xKonen, M. E., Burras, C. L., & Sandor, J. A. (2003). Organic Carbon, Texture, and Quantitative Color Measurement Relationships for Cultivated Soils in North Central Iowa. Soil Science Society of America Journal, 67(6), 1823. doi:10.2136/sssaj2003.1823Mouazen, A. M., Maleki, M. R., De Baerdemaeker, J., & Ramon, H. (2007). On-line measurement of some selected soil properties using a VIS–NIR sensor. Soil and Tillage Research, 93(1), 13-27. doi:10.1016/j.still.2006.03.009Pan, G., Xu, X., Smith, P., Pan, W., & Lal, R. (2010). An increase in topsoil SOC stock of China’s croplands between 1985 and 2006 revealed by soil monitoring. Agriculture, Ecosystems & Environment, 136(1-2), 133-138. doi:10.1016/j.agee.2009.12.011Sánchez-Marañón, M., Martín-García, J. M., & Delgado, R. (2011). Effects of the fabric on the relationship between aggregate stability and color in a Regosol–Umbrisol soilscape. Geoderma, 162(1-2), 86-95. doi:10.1016/j.geoderma.2011.01.008Sánchez-Marañón, M., Ortega, R., Miralles, I., & Soriano, M. (2007). Estimating the mass wetness of Spanish arid soils from lightness measurements. Geoderma, 141(3-4), 397-406. doi:10.1016/j.geoderma.2007.07.005Sánchez-Marañón, M., Delgado, G., Melgosa, M., Hita, E., & Delgado, R. (1997). CIELAB COLOR PARAMETERS AND THEIR RELATIONSHIP TO SOIL CHARACTERISTICS IN MEDITERRANEAN RED SOILS. Soil Science, 162(11), 833-842. doi:10.1097/00010694-199711000-00007Singleton, P. (1991). Water tables and soil colour as an indicator of saturation in some soils of the Waikato, New Zealand. Soil Research, 29(4), 467. doi:10.1071/sr9910467Spielvogel, S., Knicker, H., & Kögel-Knabner, I. (2004). Soil organic matter composition and soil lightness. Journal of Plant Nutrition and Soil Science, 167(5), 545-555. doi:10.1002/jpln.200421424Viscarra Rossel, R. A., Minasny, B., Roudier, P., & McBratney, A. B. (2006). Colour space models for soil science. Geoderma, 133(3-4), 320-337. doi:10.1016/j.geoderma.2005.07.017Webster, R., & Oliver, M. A. (2007). Geostatistics for Environmental Scientists. Statistics in Practice. doi:10.1002/978047051727

Analyses of the Impact of Soil Conditions and Soil Degradation on Vegetation Vitality and Crop Productivity Based on Airborne Hyperspectral VNIR–SWIR–TIR Data in a Semi-Arid Rainfed Agricultural Area (Camarena, Central Spain)

Soils are an essential factor contributing to the agricultural production of rainfed crops such as barley and triticale cereals. Changing environmental conditions and inadequate land management are endangering soil quality and productivity and, in turn, crop quality and productivity are affected. Advances in hyperspectral remote sensing are of great use for the spatial characterization and monitoring of the soil degradation status, as well as its impact on crop growth and agricultural productivity. In this study, hyperspectral airborne data covering the visible, near-infrared, short-wave infrared, and thermal infrared (VNIR–SWIR–TIR, 0.4–12 µm) were acquired in a Mediterranean agricultural area of central Spain and used to analyze the spatial differences in vegetation vitality and grain yield in relation to the soil degradation status. Specifically, leaf area index (LAI), crop water stress index (CWSI), and the biomass of the crop yield are derived from the remote sensing data and discussed regarding their spatial differences and relationship to a classification of erosion and accumulation stages (SEAS) based on previous remote sensing analyses during bare soil conditions. LAI and harvested crop biomass yield could be well estimated by PLS regression based on the hyperspectral and in situ reference data (R2 of 0.83, r of 0.91, and an RMSE of 0.2 m2 m−2 for LAI and an R2 of 0.85, r of 0.92, and an RMSE of 0.48 t ha−1 for grain yield). In addition, the soil erosion and accumulation stages (SEAS) were successfully predicted based on the canopy spectral signal of vegetated crop fields using a random forest machine learning approach. Overall accuracy was achieved above 71% by combining the VNIR–SWIR–TIR canopy reflectance and emissivity of the growing season with topographic information after reducing the redundancy in the spectral dataset. The results show that the estimated crop traits are spatially related to the soil’s degradation status, with shallow and highly eroded soils, as well as sandy accumulation zones being associated with areas of low LAI, crop yield, and high crop water stress. Overall, the results of this study illustrate the enormous potential of imaging spectroscopy for a combined analysis of the plant-soil system in the frame of land and soil degradation monitoring

Vegetative growth and fruit set of olive (Olea europaea L. cv. ‘Zard’) in response to some soil and plant factors

This experiment was conducted to explore the reasons of difference between ‘Zard’ olive orchard with the poor vegetative growth rate and fruit set and orchard with suitable vegetative growth rate and fruit set in relation to some soil and plant factors during two seasons. Note that assumptions were based on the overall canopy greenness of the olive trees, so experimental orchards in which the planted trees showed optimum leaf greenness were considered good situations for optimum vegetative growth and productivity. Remote sensing technologies based on normalized difference vegetation index (NDVI) were employed on olive orchards, and two orchards meeting the criteria of highest amount of greenness and lowest amount of greenness were selected. Length of current-year shoot (LCYS) and fruit set were considered indicators of tree vegetative growth and productivity, respectively. Results clearly indicated a significant difference between the two selected orchards in terms of canopy volume (CV), leaf nitrogen content (N), leaf potassium content (K), silt, sand, Sodium adsorption rate (SAR), available phosphorous (Pavi), total neutralizing value (TNV), electrical conductivity (EC), chloride (Cl), and Fe variables. A stepwise regression method was used to evaluate the effects of soil and plant variables on fruit set and LCYS. According to the obtained results, the main reasons for differences between two orchards in fruit set and vegetative growth was N and K deficiencies, soil salinity, and a high percentage of silt in the soil