We present quantitative probing as a model-agnostic framework for validating
causal models in the presence of quantitative domain knowledge. The method is
constructed as an analogue of the train/test split in correlation-based machine
learning and as an enhancement of current causal validation strategies that are
consistent with the logic of scientific discovery. The effectiveness of the
method is illustrated using Pearl's sprinkler example, before a thorough
simulation-based investigation is conducted. Limits of the technique are
identified by studying exemplary failing scenarios, which are furthermore used
to propose a list of topics for future research and improvements of the
presented version of quantitative probing. The code for integrating
quantitative probing into causal analysis, as well as the code for the
presented simulation-based studies of the effectiveness of quantitative probing
is provided in two separate open-source Python packages.Comment: submitted to the Journal of Causal Inferenc