Performance evaluation of machine learning models for credit risk prediction

The purpose of this research paper is to propose an approach for calculating the optimal threshold for predictions generated by binomial classification models for credit risk prediction. Our approach is considering the cost matrix and cumulative profit chart for setting the threshold value. In the paper we examine the performance of several models trained with homogeneous (Random Forest, XGBoost, etc.) and heterogeneous (Stacked Ensemble) ensemble classifiers. Models are trained on data extracted from Lending Club website. Different evaluation measures are derived to compare and rank the fitted models. Further analysis reveals that application of trained models with the set according to the proposed approach threshold leads to significantly reduced default loans ratio and at the same time improves the credit portfolio structure of the Peer-to-Peer lending platform. We evaluate the models performance and demonstrate that with machine learning models Peer-to-Peer lending platform can decrease the default loan ratio by 8% and generate profit lift of 16%

Ranked List Fusion and Re-Ranking With Pre-Trained Transformers for ARQMath Lab

This paper elaborates on our submission to the ARQMath track at CLEF 2021. For our submission this year we use a collection of methods to retrieve and re-rank the answers in Math Stack Exchange in addition to our two-stage model which was comparable to the best model last year in terms of NDCG’. We also provide a detailed analysis of what the transformers are learning and why is it hard to train a math language model using transformers. This year’s submission to Task-1 includes summarizing long question-answer pairs to augment and index documents, using byte-pair encoding to tokenize formula and then re-rank them, and finally important keywords extraction from posts. Using an ensemble of these methods our approach shows a 20% improvement than our ARQMath’2020 Task-1 submission

Pessimistic Rescaling and Distribution Shift of Boosting Models for Impression-Aware Online Advertising Recommendation

In this paper, we provide an overview of the approach we used as team Gabibboost for the ACM RecSys Challenge 2023, organized by ShareChat and Moj. The challenge focused on predicting user activity in the online advertising setting based on impression data, in particular, predicting whether a user would install an advertised application using a high-dimensional anonymized feature vector. Our proposed solution is based on an ensemble model that combines the strengths of several machine learning sub-models, including CatBoost, LightGBM, HistGradientBoosting, and two hybrid models. Our proposal is able to harness the strengths of our models through a distribution shift postprocessing and fine-Tune the final prediction via a custom build pessimistic rescaling function. The final ensemble model allowed us to rank 1st on the academic leaderboard and 9th overall

Feature-Enhanced Network with Hybrid Debiasing Strategies for Unbiased Learning to Rank

Unbiased learning to rank (ULTR) aims to mitigate various biases existing in user clicks, such as position bias, trust bias, presentation bias, and learn an effective ranker. In this paper, we introduce our winning approach for the "Unbiased Learning to Rank" task in WSDM Cup 2023. We find that the provided data is severely biased so neural models trained directly with the top 10 results with click information are unsatisfactory. So we extract multiple heuristic-based features for multi-fields of the results, adjust the click labels, add true negatives, and re-weight the samples during model training. Since the propensities learned by existing ULTR methods are not decreasing w.r.t. positions, we also calibrate the propensities according to the click ratios and ensemble the models trained in two different ways. Our method won the 3rd prize with a DCG@10 score of 9.80, which is 1.1% worse than the 2nd and 25.3% higher than the 4th.Comment: 5 pages, 1 figure, WSDM Cup 202

Hyperparameter Optimization and Boosting for Classifying Facial Expressions: How good can a “Null” Model be?

One of the goals of the ICML workshop on representation and learning is to establish benchmark scores for a new data set of labeled facial expressions. This paper presents the performance of a "Null" model consisting of convolutions with random weights, PCA, pooling, normalization, and a linear readout. Our approach focused on hyperparameter optimization rather than novel model components. On the Facial Expression Recognition Challenge held by the Kaggle website, our hyperparameter optimization approach achieved a score of 60% accuracy on the test data. This paper also introduces a new ensemble construction variant that combines hyperparameter optimization with the construction of ensembles. This algorithm constructed an ensemble of four models that scored 65.5% accuracy. These scores rank 12th and 5th respectively among the 56 challenge participants. It is worth noting that our approach was developed prior to the release of the data set, and applied without modification; our strong competition performance suggests that the TPE hyperparameter optimization algorithm and domain expertise encoded in our Null model can generalize to new image classification data sets.Engineering and Applied Science