Enhancing logistic regression model through AHP-initialized weight optimization using regularization and gradient descent adaptation: A comparative study

Abstract

This study explores an approach to improving the performance of logistic regression model (LR) integrated with Analytic Hierarchy Process (AHP) for weight initialization model with regularization and adaptation of gradient descent (GD). Traditional LR model relies on random weight initialization leading to suboptimal performances. By employing AHP, a hybrid model that deployed priority vector as initial weights is obtained, reflecting the relative importance of input features. Previous works reported subpar performances of AHP-LR hybrid model due to the lack of optimizing for the initialized weights. In this study, the weights are proposed to be optimized with L1 and L2 regularization approach, penalizing deviations from the AHP-initialized weights through modified log-likelihood function with modified GD optimization. This comparative analysis involves four models: LR with L2 regularization, AHP weights as LR weights, and AHP-weights optimized with L1 and L2 regularization. A prediction experiment is conducted using synthetic dataset to assess the models' performance in terms of accuracy, recall, precision, F1-score, and ROC-AUC. The results indicate that optimizing weights with L1 or L2 regularization significantly enhances model performance, compared to direct application of AHP weights without optimization yields near-random guesses. Additionally, incorporating true expert-derived weights, evaluating their impact on model performance and experimenting with authentic dataset and different weight derivation methods would offer valuable insights