Conformal Prediction: a Unified Review of Theory and New Challenges

In this work we provide a review of basic ideas and novel developments about Conformal Prediction -- an innovative distribution-free, non-parametric forecasting method, based on minimal assumptions -- that is able to yield in a very straightforward way predictions sets that are valid in a statistical sense also in in the finite sample case. The in-depth discussion provided in the paper covers the theoretical underpinnings of Conformal Prediction, and then proceeds to list the more advanced developments and adaptations of the original idea.Comment: arXiv admin note: text overlap with arXiv:0706.3188, arXiv:1604.04173, arXiv:1709.06233, arXiv:1203.5422 by other author

Strong validity, consonance, and conformal prediction

Valid prediction of future observations is an important and challenging problem. The two general approaches for quantifying uncertainty about the future value employ prediction regions and predictive distribution, respectively, with the latter usually considered to be more informative because it performs other prediction-related tasks. Standard notions of validity focus on the former, i.e., coverage probability bounds for prediction regions, but a notion of validity relevant to the other prediction-related tasks performed by the latter is lacking. In this paper, we present a new notion---strong prediction validity---relevant to these more general prediction tasks. We show that strong validity is connected to more familiar notions of coherence, and argue that imprecise probability considerations are required in order to achieve it. We go on to show that strong prediction validity can be achieved by interpreting the conformal prediction output as the contour function of a consonant plausibility function. We also offer an alternative characterization, based on a new nonparametric inferential model construction, wherein the appearance of consonance is more natural, and prove strong prediction validity.Comment: 34 pages, 3 figures, 2 tables. Comments welcome at https://www.researchers.one/article/2020-01-1

Inductive conformal predictors in the batch mode

Conformal predictors are set predictors that are automatically valid in the sense of having coverage probability equal to or exceeding a given confidence level. Inductive conformal predictors are a computationally efficient version of conformal predictors satisfying the same property of validity. However, inductive conformal predictors have been only known to control unconditional coverage probability. This paper explores various versions of conditional validity and various ways to achieve them using inductive conformal predictors and their modifications