On the suitability of resampling techniques for the class imbalance problem in credit scoring

In real-life credit scoring applications, the case in which the class of defaulters is under-represented in comparison with the class of non-defaulters is a very common situation, but it has still received little attention. The present paper investigates the suitability and performance of several resampling techniques when applied in conjunction with statistical and artificial intelligence prediction models over five real-world credit data sets, which have artificially been modified to derive different imbalance ratios (proportion of defaulters and non-defaulters examples). Experimental results demonstrate that the use of resampling methods consistently improves the performance given by the original imbalanced data. Besides, it is also important to note that in general, over-sampling techniques perform better than any under-sampling approach.This work has partially been supported by the Spanish Ministry of Education and Science under grant TIN2009– 14205 and the Generalitat Valenciana under grant PROMETEO/2010/ 028

Explainable credit scoring through generative adversarial networks

Credit scoring has been playing a vital role in mitigating financial risk that could affect the sustainability of financial institutions. An accurate and automated credit scoring allows to control the financial risk by using the state-of-the-art and data-driven analytics. The primary rationale of this thesis is to understand and improve financial credit scoring models. The key issues that occur in the process of developing credit scoring model using the state-of-the-art machine learning(ML) techniques, are identified and investigated. Through the proposed models using ML approaches in this thesis, the challenges in credit scoring can be resolved. Therefore, the existing credit scoring models can be improved by novel computer science techniques in realistic problem of the areas as follows. First, an interpretability aspect of credit scoring as eXplainable Artificial Intelligence (XAI) is examined by non-parametric tree-based ML models combining with SHapley Additive exPlanations (SHAP). In this experiment, the suitability of tree-based ensemble models is also assessed in imbalanced credit scoring dataset, comparing the performance of different class imbalance. In order to achieve explainability as well as high predictive performance in credit scoring, we propose a model named as NATE which is Non-pArameTric approach for Explainable credit scoring. This explainable and comprehensible NATE allows us to analyse the key factors of credit scoring by SHAP values both locally and globally in addition to robust predictive power for creditworthiness. Second, the issue of class imbalance is investigated. Class imbalance in datasets occurs when there are a huge number of differences of observations between the classes in the dataset. The imbalanced class in real-world credit scoring datasets results in the biased classification performance for credit worthiness. As an approach to overcome the limitation of traditional resampling methods for class imbalance, we propose a model named as NOTE which is Non-parametric Oversampling Techniques for Explainable credit scoring. By using conditional Wasserstein Generative Adversarial Networks (cWGAN)-based oversampling technique paired with Non-parametric Stacked Autoen-coder (NSA), NOTE as a generative model allows to oversample minority class with reflecting the complex and non-linear patterns in the dataset. Therefore, NOTE predicts the classification and explains the credit scoring model with unbiased performance on a balanced credit scoring dataset. Third, incomplete data is also a common issue in credit scoring datasets. This missingness normally distorts the analysis and prediction for credit scoring, and results in the misclassification for creditworthiness. To address the issue of missing values in the dataset and overcome the limitation of conventional imputation methods, we propose a model named as DITE which is Denoising Imputation TEchniques for missingness in credit scoring. By using the extended Generative Adversarial Imputation Networks (GAIN) paired with randomised Singular Value Decomposition (rSVD), DITE is capable of replacing missing values with plausible estimation through reducing the noise and capturing complex missing patterns in dataset. To evaluate the robustness and effectiveness of the proposed models for key issues, namely, model explainability, class imbalance, and missing-ness in the dataset, the performances of models using ML are compared against the benchmarks of literature on publicly available real-world financial credit scoring datasets, respectively. Our experimental results successfully demonstrated the robustness and effectiveness of the novel concepts used in the models by outperforming the benchmarks. Furthermore, the pro-posed NATE, NOTE and DITE also lead to a better model explainability, suitability, stability, and superiority on complex and non-linear credit scoring datasets. Finally, this thesis demonstrated that the existing credit scoring models can be improved by novel computer science techniques in real-world problem of credit scoring domain

Improving Risk Predictions by Preprocessing Imbalanced Credit Data

Imbalanced credit data sets refer to databases in which the class of defaulters is heavily under-represented in comparison to the class of non-defaulters. This is a very common situation in real-life credit scoring applications, but it has still received little attention. This paper investigates whether data resampling can be used to improve the performance of learners built from imbalanced credit data sets, and whether the effectiveness of resampling is related to the type of classifier. Experimental results demonstrate that learning with the resampled sets consistently outperforms the use of the original imbalanced credit data, independently of the classifier used

Investigating the Performance of Smote for Class Imbalanced Learning: A Case Study of Credit Scoring Datasets

Classification of datasets is one of the major issues encountered by the data mining community. This problem heightens when the real world datasets is also imbalanced in nature. A dataset happens to be imbalanced when the numbers of observations belonging to rare class are greatly outnumbered by the observations of another class. Class with greater number of observation is called the majority or the negative class, while the other with rare observations is referred to as the minority or the positive class. Literature represents number of resampling techniques that address the problem of class imbalance. One of the most important strategies is to resample the datasets that aim to balance the number of minority or majority observations by over-sampling or under-sampling respectively. This paper aims to investigates and analyze the performance of most widely used oversampling procedure Synthetic Minority Oversampling Technique (SMOTE) for different thresholds of oversampling using four classifiers for three credit scoring datasets

An insight into the experimental design for credit risk and corporate bankruptcy prediction systems

Over the last years, it has been observed an increasing interest of the finance and business communities in any application tool related to the prediction of credit and bankruptcy risk, probably due to the need of more robust decision-making systems capable of managing and analyzing complex data. As a result, plentiful techniques have been developed with the aim of producing accurate prediction models that are able to tackle these issues. However, the design of experiments to assess and compare these models has attracted little attention so far, even though it plays an important role in validating and supporting the theoretical evidence of performance. The experimental design should be done carefully for the results to hold significance; otherwise, it might be a potential source of misleading and contradictory conclusions about the benefits of using a particular prediction system. In this work, we review more than 140 papers published in refereed journals within the period 2000–2013, putting the emphasis on the bases of the experimental design in credit scoring and bankruptcy prediction applications. We provide some caveats and guidelines for the usage of databases, data splitting methods, performance evaluation metrics and hypothesis testing procedures in order to converge on a systematic, consistent validation standard.This work has partially been supported by the Mexican Science and Technology Council (CONACYT-Mexico) through a Postdoctoral Fellowship [223351], the Spanish Ministry of Economy under grant TIN2013-46522-P and the Generalitat Valenciana under grant PROMETEOII/2014/062

Credit Scoring Using Machine Learning

For financial institutions and the economy at large, the role of credit scoring in lending decisions cannot be overemphasised. An accurate and well-performing credit scorecard allows lenders to control their risk exposure through the selective allocation of credit based on the statistical analysis of historical customer data. This thesis identifies and investigates a number of specific challenges that occur during the development of credit scorecards. Four main contributions are made in this thesis. First, we examine the performance of a number supervised classification techniques on a collection of imbalanced credit scoring datasets. Class imbalance occurs when there are significantly fewer examples in one or more classes in a dataset compared to the remaining classes. We demonstrate that oversampling the minority class leads to no overall improvement to the best performing classifiers. We find that, in contrast, adjusting the threshold on classifier output yields, in many cases, an improvement in classification performance. Our second contribution investigates a particularly severe form of class imbalance, which, in credit scoring, is referred to as the low-default portfolio problem. To address this issue, we compare the performance of a number of semi-supervised classification algorithms with that of logistic regression. Based on the detailed comparison of classifier performance, we conclude that both approaches merit consideration when dealing with low-default portfolios. Third, we quantify the differences in classifier performance arising from various implementations of a real-world behavioural scoring dataset. Due to commercial sensitivities surrounding the use of behavioural scoring data, very few empirical studies which directly address this topic are published. This thesis describes the quantitative comparison of a range of dataset parameters impacting classification performance, including: (i) varying durations of historical customer behaviour for model training; (ii) different lengths of time from which a borrower’s class label is defined; and (iii) using alternative approaches to define a customer’s default status in behavioural scoring. Finally, this thesis demonstrates how artificial data may be used to overcome the difficulties associated with obtaining and using real-world data. The limitations of artificial data, in terms of its usefulness in evaluating classification performance, are also highlighted. In this work, we are interested in generating artificial data, for credit scoring, in the absence of any available real-world data

A New Model Averaging Approach in Predicting Credit Risk Default

none2siThe paper introduces a novel approach to ensemble modeling as a weighted model average technique. The proposed idea is prudent, simple to understand, and easy to implement compared to the Bayesian and frequentist approach. The paper provides both theoretical and empirical contributions for assessing credit risk (probability of default) effectively in a new way by creating an ensemble model as a weighted linear combination of machine learning models. The idea can be generalized to any classification problems in other domains where ensemble-type modeling is a subject of interest and is not limited to an unbalanced dataset or credit risk assessment. The results suggest a better forecasting performance compared to the single best well-known machine learning of parametric, non-parametric, and other ensemble models. The scope of our approach can be extended to any further improvement in estimating weights differently that may be beneficial to enhance the performance of the model average as a future research direction.openParitosh Navinchandra Jha; Cucculelli MarcoJha, Paritosh Navinchandra; Cucculelli, Marc