12 research outputs found
SciKit-GStat 1.0: a SciPy-flavored geostatistical variogram estimation toolbox written in Python
Geostatistical methods are widely used in almost all geoscientific disciplines, i.e., for interpolation, rescaling, data assimilation or modeling. At its core, geostatistics aims to detect, quantify, describe, analyze and model spatial covariance of observations. The variogram, a tool to describe this spatial covariance in a formalized way, is at the heart of every such method. Unfortunately, many applications of geostatistics focus on the interpolation method or the result rather than the quality of the estimated variogram. Not least because estimating a variogram is commonly left as a task for computers, and some software implementations do not even show a variogram to the user. This is a miss, because the quality of the variogram largely determines whether the application of geostatistics makes sense at all. Furthermore, the Python programming language was missing a mature, well-established and tested package for variogram estimation a couple of years ago.
Here I present SciKit-GStat, an open-source Python package for variogram estimation that fits well into established frameworks for scientific computing and puts the focus on the variogram before more sophisticated methods are about to be applied. SciKit-GStat is written in a mutable, object-oriented way that mimics the typical geostatistical analysis workflow. Its main strength is the ease of use and interactivity, and it is therefore usable with only a little or even no knowledge of Python. During the last few years, other libraries covering geostatistics for Python developed along with SciKit-GStat. Today, the most important ones can be interfaced by SciKit-GStat. Additionally, established data structures for scientific computing are reused internally, to keep the user from learning complex data models, just for using SciKit-GStat. Common data structures along with powerful interfaces enable the user to use SciKit-GStat along with other packages in established workflows rather than forcing the user to stick to the author\u27s programming paradigms.
SciKit-GStat ships with a large number of predefined procedures, algorithms and models, such as variogram estimators, theoretical spatial models or binning algorithms. Common approaches to estimate variograms are covered and can be used out of the box. At the same time, the base class is very flexible and can be adjusted to less common problems, as well. Last but not least, it was made sure that a user is aided in implementing new procedures or even extending the core functionality as much as possible, to extend SciKit-GStat to uncovered use cases. With broad documentation, a user guide, tutorials and good unit-test coverage, SciKit-GStat enables the user to focus on variogram estimation rather than implementation details
SciKit-GStat Uncertainty: A software extension to cope with uncertain geostatistical estimates
This study is focused on an extension of a well established geostatistical software to enable one to effectively and interactively cope with uncertainty in geostatistical applications. The extension includes a rich component library, pre-built interfaces and an online application. We discuss the concept of replacing the empirical variogram with its uncertainty bound. This enables one to acknowledge uncertainties characterizing the underlying geostatistical datasets and typical methodological approaches. This allows for a probabilistic description of the variogram and its parameters at the same time. Our approach enables (1) multiple interpretations of a sample and (2) a multi-model context for geostatistical applications. We focus the sample application on propagating observation uncertainties into manual variogram parametrization and analyze its effects. Using two different datasets, we show how insights on uncertainty can be used to reject variogram models, thus constraining the space of formally equally probable models to tackle the issue of parameter equifinality
Soil moisture: variable in space but redundant in time
Soil moisture at the catchment scale exhibits a huge spatial variability. This suggests that even a large amount of observation points would not be able to capture soil moisture variability.
We present a measure to capture the spatial dissimilarity and its change over time. Statistical dispersion among observation points is related to their distance to describe spatial patterns. We analyzed the temporal evolution and emergence of these patterns and used the mean shift clustering algorithm to identify and analyze clusters. We found that soil moisture observations from the 19.4âkm2 Colpach catchment in Luxembourg cluster in two fundamentally different states. On the one hand, we found rainfall-driven data clusters, usually characterized by strong relationships between dispersion and distance. Their spatial extent roughly matches the average hillslope length in the study area of about 500âm. On the other hand, we found clusters covering the vegetation period. In drying and then dry soil conditions there is no particular spatial dependence in soil moisture patterns, and the values are highly similar beyond hillslope scale.
By combining uncertainty propagation with information theory, we were able to calculate the information content of spatial similarity with respect to measurement uncertainty (when are patterns different outside of uncertainty margins?). We were able to prove that the spatial information contained in soil moisture observations is highly redundant (differences in spatial patterns over time are within the error margins). Thus, they can be compressed (all cluster members can be substituted by one representative member) to only a fragment of the original data volume without significant information loss.
Our most interesting finding is that even a few soil moisture time series bear a considerable amount of information about dynamic changes in soil moisture. We argue that distributed soil moisture sampling reflects an organized catchment state, where soil moisture variability is not random. Thus, only a small amount of observation points is necessary to capture soil moisture dynamics
Histogram via entropy reduction (HER): an information-theoretic alternative for geostatistics
Interpolation of spatial data has been regarded in many different forms, varying from deterministic to stochastic, parametric to nonparametric, and purely data-driven to geostatistical methods. In this study, we propose a nonparametric interpolator, which combines information theory with probability aggregation methods in a geostatistical framework for the stochastic estimation of unsampled points. Histogram via entropy reduction (HER) predicts conditional distributions based on empirical probabilities, relaxing parameterizations and, therefore, avoiding the risk of adding information not present in data. By construction, it provides a proper framework for uncertainty estimation since it accounts for both spatial configuration and data values, while allowing one to introduce or infer properties of the field through the aggregation method. We investigate the framework using synthetically generated data sets and demonstrate its efficacy in ascertaining the underlying field with varying sample densities and data properties. HER shows a comparable performance to popular benchmark models, with the additional advantage of higher generality. The novel method brings a new perspective of spatial interpolation and uncertainty analysis to geostatistics and statistical learning, using the lens of information theory
V-FOR-WaTer - a virtual research environment for environmental research
Extent and diversity of environmental data are continuously increasing due to more sensor networks with higher spatial and temporal resolution. To find appropriate data for analyses and especially for large scale models and simulations in this data explosion can take up to several months. The preprocessing of these heterogeneous datasets from different research disciplines to acquire a coherent dataset, can be done with a wide range of algorithms and tools. The outcome is a base dataset that is not reproducible and in consequence, neither are the resulting analyses [3, 9]. The datasets therefore do not obey the FAIR principles [13]. The V-FOR-WaTer web portal [11] aims to improve this situation by collecting data and metadata from a wide variety of sources and by offering preprocessed data
V-FOR-WaTer â the virtual research environment to discover and analyse environmental data
The extent and diversity of environmental data continuously increase due to more and new sensors with higher spatial and temporal resolution and due to the growth and automation of observational networks. The observed data form the basis for a better understanding of ecological systems either by data driven methods or by comparisons of the data with model predictions. However, a considerable amount of these data are difficult to access or even still stored on local data storage devices making it difficult or even impossible to find, access and re-use the data. In addition the data lack a proper metadata description required for an interoperable analysis, hence they are barely useful for science. This results in very time consuming preparation and pre-processing of data, especially when datasets from different sources are combined.
The main objectives of V-FOR-WaTer are to simplify data access for environmental sciences, foster data publications, and facilitate preparations of data and their analyses with a comprehensive toolbox. Also, bringing data and tools together in a single environment maximises the reproducibility of analyses and models.
We will present the status of the V-FOR-WaTer system and describe its components and features. The database contains point and time series data, with an extensive metadata model customized for hydrological and environmental data that fulfils international standards (INSPIRE, ISO19115). The incorporated datasets originate from university projects and state offices. The V-FOR-WaTer web portal provides user-friendly access to the datasets by way of a comprehensive search and filter menu. The toolbox includes tools for pre-processing, common hydrologic applications and geostatistics, and is easily extendable due to the modular design of the system (e.g. various tools for data scaling are planned). In cooperation with the users and data providers, the further development of V-FOR-WaTer is expected to create a significant benefit for the scientific work with environmental data
Agroforestry : an appropriate and sustainable response to a changing climate in Southern Africa?
CITATION: Sheppard, Jonathan P. et al. 2020. Agroforestry : an appropriate and sustainable response to a changing climate in Southern Africa? Sustainability 12(17):6796, doi:10.3390/su12176796.The original publication is available at: https://www.mdpi.comENGLISH ABSTRACT: Agroforestry is often discussed as a strategy that can be used both for the adaptation to and
the mitigation of climate change e ects. The climate of southern Africa is predicted to be severely
a ected by such changes. With agriculture noted as the continentâs largest economic sector, issues
such as food security and land degradation are in the forefront. In the light of such concerns we
review the current literature to investigate if agroforestry systems (AFS) are a suitable response to the
challenges besetting traditional agricultural caused by a changing climate. The benefits bestowed by
AFS are multiple, o ering ecosystem services, influence over crop production and positive impacts on
rural livelihoods through provisioning and income generation. Nevertheless, knowledge gaps remain.
We identify outstanding questions requiring further investigation such as the interplay between trees
and crops and their combination, with a discussion of potential benefits. Furthermore, we identify
deficiencies in the institutional and policy frameworks that underlie the adoption and stimulus of
AFS in the southern African region. We uphold the concept that AFS remains an appropriate and
sustainable response for an increased resilience against a changing climate in southern Africa for the
benefit of livelihoods and multiple environmental values.Publisher's versio
mmaelicke/zonal-variograms: Version 0.2
<h1>Zonal variograms</h1>
<p>This small piece of code can generate variograms using a zonal statistics approach. For any given geoTiff, it will calculate a variogram for any Polygon supplied to tool.
It accepts shapefiles and geopackages, and can handle multiple layers as input. The output is always a geopackage with the same inputs. Zonal variogram parameters are
added as columns to the properties table.
By default, the tool will calculate a variogram for all input cells, but can be configured to rather use a random subsample. This can be helpful in case the zones are big.</p>
<p>The variograms will share the same hyper-parameters, thus differences are only due to different variogram parameters fitted to the data. Multiple parameters are not possible right not.
The tool does not support multi-band input right now.</p>
<p>The tool can be used as a Python library or a command line tool</p>
<p>Install like:</p>
<pre><code>pip install zonal_variograms
</code></pre>
<p>Example call</p>
<pre><code class="language-bash">zonal_variograms ./in/my_raster.tif ./in/Features.gpkg --model=exponential --maxlag=median --use-nugget --n-lags=25 --sample=400 --add-json --add-data-uri
</code></pre>
Eine virtuelle Forschungsumgebung fĂŒr die Wasser- und terrestrische Umweltforschung
Die Datenaufbereitung fĂŒr (umwelt-)wissenschaftliche Analysen stellt oftmals eine groĂe Herausforderung dar, da Daten verschiedenster Quellen erst aufwĂ€ndig umformatiert und prĂ€prozessiert werden mĂŒssen, um einen kohĂ€renten Datensatz zu erhalten. Ziel der Forschungsumgebung von V-FOR-WaTer ist es, den Zugang zu Daten aus den terrestrischen Umweltwissenschaften zu vereinfachen, die Publikation von Daten zu unterstĂŒtzen und die Datenaufbereitung sowie die Analyse von Daten mithilfe einer umfangreichen Auswahl von Werkzeugen zu erleichtern. Durch diesen einfachen Zugriff auf Daten und Werkzeuge, und deren VerknĂŒpfung in âWorkflowsâ fĂŒr Wissenschaftler aus UniversitĂ€ten und LandesĂ€mtern, wird die wissenschaftliche Arbeit beschleunigt, und die Reproduzierbarkeit von Analysen wird gefördert. Das RĂŒckgrat des Prototyps der Forschungsumgebung bildet eine Datenbank mit einer detaillierten Metadatenbeschreibung, die auf die Anforderungen von Wasser- und terrestrischen Umweltdaten zugeschnitten ist. Die bisher integrierten Daten stammen aus UniversitĂ€tsprojekten und von LandesĂ€mtern. Weiter wird an einer Verbindung zu den âGFZ Data Servicesâ, dem etablierten Repositorium fĂŒr geowissenschaftliche Daten des Geoforschungszentrums Potsdam, gearbeitet. Dadurch wird zum einen die Publikation von Daten aus der Forschungsumgebung heraus vereinfacht und zum anderen der Zugriff auf externe Daten im Portal des GFZ ermöglicht. Der Grundlage, um mit den GFZ Data Services und anderen Systemen kompatibel zu sein, ist die KonformitĂ€t unseres Metadatenschemas mit internationalen Standards (INSPIRE, ISO19115). Durch die BerĂŒcksichtigung der gĂ€ngigen Standards kann das Portal - nach entsprechenden Anpassungen - auch von anderen Geo- und Umweltwissenschaftlichen Disziplinen genutzt werden. Das Design der Forschungsumgebung ist an typischen ArbeitsablĂ€ufen in den Umweltwissenschaften ausgerichtet. Ăber eine Karte und einen Filter können Daten einfach ausgewĂ€hlt werden, wĂ€hrend ein eigener Arbeitsbereich Werkzeuge fĂŒr die PrĂ€prozessierung, Skalierung und hĂ€ufige hydrologische Anwendungen bereithĂ€lt. DarĂŒber hinaus sind auch spezifischere Werkzeuge wie z.B. fĂŒr die Geostatistik, und demnĂ€chst auch fĂŒr Berechnungen zur Evapotranspiration verfĂŒgbar. Die Auswahl an Werkzeugen kann flexibel erweitert werden und wird letztendlich auch Werkzeuge enthalten, die von Nutzern entwickelt wurden, wodurch die aktuellen Forschungsthemen und âmethoden der hydrologischen Gemeinschaft widergespiegelt werden. Die Werkzeuge sind als âWeb Processing Servicesâ (WPS) angebunden, die als âWorkflowsâ verknĂŒpft und gespeichert werden können. Dies ermöglicht auch komplexere Analysen und erhöht die Reproduzierbarkeit der Forschung