Decentralized Infrastructure for Reproducible and Replicable Geographical Science

The I-GUIDE cyberinfrastructure project for convergence science is a leading example of the possibilities the geospatial data revolution holds for scientific discovery. However, rapidly expanding access to increasingly complex data sources and methods of computational analysis also presents a challenge to the research community. With more data and more potential analyses, researchers face the possibility of jeopardizing the inferential power of convergence research with selection bias. Well-designed infrastructure that can flexibly guide researchers as they record and track decisions in their research designs opens a path to mitigating this problem, while also expanding the reproducibility and replicability of research. Much of the infrastructure needed for convergence research can be borrowed and adapted from other disciplines, but geographic convergence research confronts at least five novel challenges. These are the need for geographically-explicit project metadata, managing diverse and complex data inputs, handling restricted data, specifying and reproducing computational environments, and disclosing researcher decisions and threats to validity that are unique to geographic research. We introduce a template research compendium and analysis plan for study preregistration to address these novel challenges

Publications

Collection of pre-prints and published peer-reviewed article

Decentralized Infrastructure for Reproducible and Replicable Geographical Science

Conference Paper in I-GUIDE Forum 2023 - Harnessing the Geospatial Data Revolution for Sustainability Solution

Template for Reproducible and Replicable Research in Human-Environment and Geographical Sciences

This template Git repository contains a folder structure, template documents, and best practice suggestions for conducting reproducible geographic research

Controlling for spatial confounding and spatial interference in causal inference: modelling insights from a computational experiment

ABSTRACTCausal inference is a rapidly growing field of statistics that applies logical reasoning to statistical inference to estimate causal relationships. Spatial data poses several problems in causal inference – namely, spatial confounding and interference – that require different strategies when designing causal models. In order to obtain valid inferences, existing nonspatial causal models must adjust for such spatial problems. Given the blossoming literature on spatial causal inference, this research analyzes the usage of spatial causal models under a priori knowledge and a priori ignorance of the spatial structure of data. We synthesize existing research directions in noncausal spatial modelling and causal nonspatial modelling by assessing the performance of 28 spatial causal models across 16 spatial data scenarios. We used ordinary least squares (OLS) models, conditional autoregressive (CAR) models, and jointly CAR models for outcome and treatment variables as the basis for the tested models, equipping them with a variety of spatial causal adjustments. We compare our results to principles of noncausal spatial modelling and investigate their implications for spatial causal modelling. Specifically, we show that noncausal spatial modelling guidance holds in causal spatial modelling workflows and demonstrate how researchers can leverage noncausal theory to great effect. In parallel, we introduce the spycause Python package of spatial causal models and data simulators to facilitate the widespread use of these models and to enable reproduction and extension of our work

Operationalizing Spatial Causal Inference

Most spatial inquiries seek to investigate causal questions about spatial processes, but many quantitative spatial methods are designed to identify correlations and spatial patterns. Studying the structure of associations that make up a spatial pattern can provide information about what the process that generated that pattern is likely to be, but it does not provide a means of testing any one explanation against alternative explanations. Causal inference provides a set of approaches to formally make comparisons between explanations. An opportunity exists to incorporate these techniques and spatialize the theory of cause in GIScience by building on recent advances in computer science and statistics. However, implementing causal inference in geography may require a shift in the design of geographic information systems

How to Improve the Reproducibility, Replicability, and Extensibility of Remote Sensing Research

The field of remote sensing has undergone a remarkable shift where vast amounts of imagery are now readily available to researchers. New technologies, such as uncrewed aircraft systems, make it possible for anyone with a moderate budget to gather their own remotely sensed data, and methodological innovations have added flexibility for processing and analyzing data. These changes create both the opportunity and need to reproduce, replicate, and compare remote sensing methods and results across spatial contexts, measurement systems, and computational infrastructures. Reproducing and replicating research is key to understanding the credibility of studies and extending recent advances into new discoveries. However, reproducibility and replicability (R&R) remain issues in remote sensing because many studies cannot be independently recreated and validated. Enhancing the R&R of remote sensing research will require significant time and effort by the research community. However, making remote sensing research reproducible and replicable does not need to be a burden. In this paper, we discuss R&R in the context of remote sensing and link the recent changes in the field to key barriers hindering R&R while discussing how researchers can overcome those barriers. We argue for the development of two research streams in the field: (1) the coordinated execution of organized sequences of forward-looking replications, and (2) the introduction of benchmark datasets that can be used to test the replicability of results and methods

Manuscript

Manuscript preprin

Reproducible Research Practices and Barriers to Reproducible Research in Geography: Insights from a Survey

The number of reproduction and replication studies undertaken across the sciences continues to rise, however such studies have not yet become commonplace in geography. Existing attempts to reproduce geographic research suggest that many studies cannot be fully reproduced, or are simply missing components needed to attempt a reproduction. Despite this suggestive evidence, we have not yet systematically assessed geographers’ perceptions of reproducibility or the use of reproducible research practices across the discipline’s diverse research traditions; nor have we identified the factors that keep geographers from conducting more reproduction studies. We address each of these needs by surveying active geographic researchers selected using probability sampling techniques from a rigorously constructed sampling frame. We identify a clear division in perceptions of reproducibility among geographic subfields. We also find varying levels of familiarity with reproducible research practices and a perceived lack of incentives to attempt and publish reproduction studies. Despite many barriers to reproducibility and divisions between subfields, we also find common foundations for examining and expanding reproducibility in the field. These include interest in publishing transparent and reproducible methods, and in reproducing other researchers’ studies for a variety of motivations including learning, assessing the internal validity of a study, or extending prior work