Non‐parametric predictive inference for the validation of credit rating systems

Credit rating or credit scoring systems are important tools for estimating the obligor's creditworthiness and for providing an indication of the obligor's future status. The discriminatory power of a credit rating or credit scoring system refers to its ex ante ability to distinguish between two or more classes of borrowers. One of the most popular tools for the validation of the power of credit rating or credit scoring models to distinguish between two (or more) classes of borrowers is the receiver operating characteristic (ROC) curve (hypersurface) and its widely used overall summary, the area (hypervolume) under the curve (hypersurface). As the end goal of building such models is to predict and quantify uncertainty about future loans, prediction methods are especially valuable in this context. For this, non‐parametric predictive inference is a promising candidate for such inference as it is a frequentist statistical method that is explicitly aimed at using few modelling assumptions, enabled through the use of lower and upper probabilities to quantify uncertainty. The aim of the paper is to introduce non‐parametric predictive inference for ROC analysis within a banking context, for which novel results on ROC hypersurfaces for more than three groups are presented. Examples are provided to illustrate the method

The survival signature for quantifying system reliability: an introductory overview from practical perspective

The structure function describes the functioning of a system dependent on the states of its components, and is central to theory of system reliability. The survival signature is a summary of the structure function which is sufficient to derive the system’s reliability function. Since its introduction in 2012, the survival signature has received much attention in the literature, with developments on theory, computation and generalizations. This paper presents an introductory overview of the survival signature, including some recent developments. We discuss challenges for practical use of survival signatures for large systems

Predictive inference for system reliability after common-cause component failures

This paper presents nonparametric predictive inference for system reliability following common-cause failures of components. It is assumed that a single failure event may lead to simultaneous failure of multiple components. Data consist of frequencies of such events involving particular numbers of components. These data are used to predict the number of components that will fail at the next failure event. The effect of failure of one or more components on the system reliability is taken into account through the system׳s survival signature. The predictive performance of the approach, in which uncertainty is quantified using lower and upper probabilities, is analysed with the use of ROC curves. While this approach is presented for a basic scenario of a system consisting of only a single type of components and without consideration of failure behaviour over time, it provides many opportunities for more general modelling and inference, these are briefly discussed together with the related research challenges

Three-group ROC predictive analysis for ordinal outcomes

Measuring the accuracy of diagnostic tests is crucial in many application areas including medicine, machine learning and credit scoring. The receiver operating characteristic (ROC) surface is a useful tool to assess the ability of a diagnostic test to discriminate among three ordered classes or groups. In this paper, nonparametric predictive inference (NPI) for three-group ROC analysis for ordinal outcomes is presented. NPI is a frequentist statistical method that is explicitly aimed at using few modelling assumptions, enabled through the use of lower and upper probabilities to quantify uncertainty. This paper also includes results on the volumes under the ROC surfaces and consideration of the choice of decision thresholds for the diagnosis. Two examples are provided to illustrate our method

Nonparametric predictive pairwise comparison with competing risks

In reliability, failure data often correspond to competing risks, where several failure modes can cause a unit to fail. This paper presents nonparametric predictive inference (NPI) for pairwise comparison with competing risks data, assuming that the failure modes are independent. These failure modes could be the same or different among the two groups, and these can be both observed and unobserved failure modes. NPI is a statistical approach based on few assumptions, with inferences strongly based on data and with uncertainty quantified via lower and upper probabilities. The focus is on the lower and upper probabilities for the event that the lifetime of a future unit from one group, say Y, is greater than the lifetime of a future unit from the second group, say X. The paper also shows how the two groups can be compared based on particular failure mode(s), and the comparison of the two groups when some of the competing risks are combined is discussed

Pricing exotic options in the incomplete market: an imprecise probability method

This paper considers a novel exotic option pricing method for incomplete markets. Nonparametric Predictive Inference (NPI) is applied to the option pricing procedure based on the binomial tree model allowing the method to evaluate exotic options with limited information and few assumptions. As the implementation of the NPI method is greatly simplified by the monotonicity of the option payoff in the tree, we categorize exotic options by their payoff monotonicity and study a typical type of exotic option in each category, the barrier option and the look-back option. By comparison with the classic binomial tree model, we investigate the performance of our method either with different moneyness or varying maturity. All outcomes show that our model offers a feasible approach to price the exotic options with limited information, which makes it can be utilized for both complete and incomplete markets

Nonparametric predictive inference for diagnostic test thresholds

Measuring the accuracy of diagnostic tests is crucial in many application areas including medicine, machine learning and credit scoring. The receiver operating characteristic (ROC) curve and surface are useful tools to assess the ability of diagnostic tests to discriminate between ordered classes or groups. To define these diagnostic tests, selecting the optimal thresholds that maximize the accuracy of these tests is required. One procedure that is commonly used to find the optimal thresholds is by maximizing what is known as Youden’s index. This article presents nonparametric predictive inference (NPI) for selecting the optimal thresholds of a diagnostic test. NPI is a frequentist statistical method that is explicitly aimed at using few modeling assumptions, enabled through the use of lower and upper probabilities to quantify uncertainty. Based on multiple future observations, the NPI approach is presented for selecting the optimal thresholds for two-group and three-group scenarios. In addition, a pairwise approach has also been presented for the three-group scenario. The article ends with an example to illustrate the proposed methods and a simulation study of the predictive performance of the proposed methods along with some classical methods such as Youden index. The NPI-based methods show some interesting results that overcome some of the issues concerning the predictive performance of Youden’s index