Investigation into the Predictive Power of Artificial Neural Networks and Logistic Regression for Predicting Default in Chit Funds

This study evaluated the performance of an artificial neural network (ANN) multi-layer perceptron model and a logistic regression logitboost (LR) model to predict default in chit funds. The two types of default investigated were late payment of 30 days and late payment of 90 days. The dataset was broken up into training and validation datasets using random sampling and K folds cross validation was used on the training dataset to assess performance of the tuning parameters. The validation dataset was used to compare performance of both algorithms. Principle component analysis (PCA) was used to reduce the feature set while still explaining 95% of the variance in the data. The classes were highly imbalanced and Synthetic Minority Oversampling Technique (SMOTE) and down sampling were used to overcome the class imbalance. 16 experiments were ran, 8 for each of the two defaults. The three key metrics that were measured for these experiments were balanced accuracy, Area under the ROC curve (AUC) and F1 score. After making Bonferroni’s adjustment to the original p value statistical significance was set to 0.003 when comparing multiple experiments. In these experiments the ANN model had the best results for balanced accuracy, AUC and F1score. Statistical analysis using a paired t test showed that there was a statistically significant difference in the results between ANN and LR. The results of these experiments also showed that there was very little difference in the contribution of the top 20 features to the first 30 principal components, which were used to predict default. These features included family id, income and address. Features that had little or no contribution to the principle components included Commission, Auction Amount, and type of relation the nominee is to the chit fund member. These findings are context specific and in this case the context is chit funds from a digital chit fund operator in Indi

A review of machine learning applications in wildfire science and management

Artificial intelligence has been applied in wildfire science and management since the 1990s, with early applications including neural networks and expert systems. Since then the field has rapidly progressed congruently with the wide adoption of machine learning (ML) in the environmental sciences. Here, we present a scoping review of ML in wildfire science and management. Our objective is to improve awareness of ML among wildfire scientists and managers, as well as illustrate the challenging range of problems in wildfire science available to data scientists. We first present an overview of popular ML approaches used in wildfire science to date, and then review their use in wildfire science within six problem domains: 1) fuels characterization, fire detection, and mapping; 2) fire weather and climate change; 3) fire occurrence, susceptibility, and risk; 4) fire behavior prediction; 5) fire effects; and 6) fire management. We also discuss the advantages and limitations of various ML approaches and identify opportunities for future advances in wildfire science and management within a data science context. We identified 298 relevant publications, where the most frequently used ML methods included random forests, MaxEnt, artificial neural networks, decision trees, support vector machines, and genetic algorithms. There exists opportunities to apply more current ML methods (e.g., deep learning and agent based learning) in wildfire science. However, despite the ability of ML models to learn on their own, expertise in wildfire science is necessary to ensure realistic modelling of fire processes across multiple scales, while the complexity of some ML methods requires sophisticated knowledge for their application. Finally, we stress that the wildfire research and management community plays an active role in providing relevant, high quality data for use by practitioners of ML methods.Comment: 83 pages, 4 figures, 3 table

Methods to Improve the Prediction Accuracy and Performance of Ensemble Models

The application of ensemble predictive models has been an important research area in predicting medical diagnostics, engineering diagnostics, and other related smart devices and related technologies. Most of the current predictive models are complex and not reliable despite numerous efforts in the past by the research community. The performance accuracy of the predictive models have not always been realised due to many factors such as complexity and class imbalance. Therefore there is a need to improve the predictive accuracy of current ensemble models and to enhance their applications and reliability and non-visual predictive tools. The research work presented in this thesis has adopted a pragmatic phased approach to propose and develop new ensemble models using multiple methods and validated the methods through rigorous testing and implementation in different phases. The first phase comprises of empirical investigations on standalone and ensemble algorithms that were carried out to ascertain their performance effects on complexity and simplicity of the classifiers. The second phase comprises of an improved ensemble model based on the integration of Extended Kalman Filter (EKF), Radial Basis Function Network (RBFN) and AdaBoost algorithms. The third phase comprises of an extended model based on early stop concepts, AdaBoost algorithm, and statistical performance of the training samples to minimize overfitting performance of the proposed model. The fourth phase comprises of an enhanced analytical multivariate logistic regression predictive model developed to minimize the complexity and improve prediction accuracy of logistic regression model. To facilitate the practical application of the proposed models; an ensemble non-invasive analytical tool is proposed and developed. The tool links the gap between theoretical concepts and practical application of theories to predict breast cancer survivability. The empirical findings suggested that: (1) increasing the complexity and topology of algorithms does not necessarily lead to a better algorithmic performance, (2) boosting by resampling performs slightly better than boosting by reweighting, (3) the prediction accuracy of the proposed ensemble EKF-RBFN-AdaBoost model performed better than several established ensemble models, (4) the proposed early stopped model converges faster and minimizes overfitting better compare with other models, (5) the proposed multivariate logistic regression concept minimizes the complexity models (6) the performance of the proposed analytical non-invasive tool performed comparatively better than many of the benchmark analytical tools used in predicting breast cancers and diabetics ailments. The research contributions to ensemble practice are: (1) the integration and development of EKF, RBFN and AdaBoost algorithms as an ensemble model, (2) the development and validation of ensemble model based on early stop concepts, AdaBoost, and statistical concepts of the training samples, (3) the development and validation of predictive logistic regression model based on breast cancer, and (4) the development and validation of a non-invasive breast cancer analytic tools based on the proposed and developed predictive models in this thesis. To validate prediction accuracy of ensemble models, in this thesis the proposed models were applied in modelling breast cancer survivability and diabetics’ diagnostic tasks. In comparison with other established models the simulation results of the models showed improved predictive accuracy. The research outlines the benefits of the proposed models, whilst proposes new directions for future work that could further extend and improve the proposed models discussed in this thesis

Ensemble learning in the presence of noise

Tesis Doctoral inédita leída en la Universidad Autónoma de Madrid, Escuela Politécnica Superior, Departamento de Ingenieria Informática. Fecha de lectura: 14-02-2019La disponibilidad de grandes cantidades de datos provenientes de diversas fuentes ampl a enormemente las posibilidades para una explotaci on inteligente de la informaci on. No obstante, la extracci on de conocimiento a partir de datos en bruto es una tarea compleja que requiere el desarrollo de m etodos de aprendizaje e cientes y robustos. Una de las principales di cultades en el aprendizaje autom atico es la presencia de ruido en los datos. En esta tesis, abordamos el problema del aprendizaje autom atico en presencia de ruido. Para este prop osito, nos centraremos en el uso de conjuntos de clasi cadores. Nuestro objetivo es crear colecciones de aprendices base cuyos resultados, al ser combinados, mejoren no solo la precisi on sino tambi en la robustez de las predicciones. Una primera contribuci on de esta tesis es aprovechar el ratio de submuestreo para construir conjuntos de clasi cadores basados en bootstrap (como bagging o random forests) precisos y robustos. La idea de utilizar el submuestreo como mecanismo de regularizaci on tambi en se explota para la detecci on de ejemplos ruidosos. En concreto, los ejemplos que est an mal clasi cados por una fracci on de los miembros del conjunto se marcan como ruido. El valor optimo de este umbral se determina mediante validaci on cruzada. Las instancias ruidosas se eliminan ( ltrado) o se corrigen sus etiquetas de su clase (limpieza). Finalmente, se construye un conjunto de clasi cadores utilizando los datos de entrenamiento limpios ( ltrados o limpiados). Otra contribuci on de esta tesis es vote-boosting, un m etodo de conjuntos secuencial especialmente dise~nado para ser robusto al ruido en las etiquetas de clase. Vote-boosting reduce la excesiva sensibilidad a este tipo de ruido de los algoritmos basados en boosting, como adaboost. En general, los algoritmos basados en booting modi can la distribuci on de pesos en los datos de entrenamiento progresivamente para enfatizar instancias mal clasi cadas. Este enfoque codicioso puede terminar dando un peso excesivamente alto a instancias cuya etiqueta de clase sea incorrecta. Por el contrario, en vote-boosting, el enfasis se basa en el nivel de incertidumbre (acuerdo o desacuerdo) de la predicci on del conjunto, independientemente de la etiqueta de clase. Al igual que en boosting, voteboosting se puede analizar como una optimizaci on de descenso por gradiente en espacio funcional. Uno de los problemas abiertos en el aprendizaje de conjuntos es c omo construir combinaciones de clasi cadores fuertes. La principal di cultad es lograr diversidad entre los clasi cadores base sin un deterioro signi cativo de su rendimiento y sin aumentar en exceso el coste computacional. En esta tesis, proponemos construir conjuntos de SVM con la ayuda de mecanismos de aleatorizaci on y optimizaci on. Gracias a esta combinaci on de estrategias complementarias, es posible crear conjuntos de SVM que son mucho m as r apidos de entrenar y son potencialmente m as precisos que un SVM individual optimizado. Por ultimo, hemos desarrollado un procedimiento para construir conjuntos heterog eneos que interpolan sus decisiones a partir de conjuntos homog eneos compuestos por diferentes tipos de clasi cadores. La composici on optima del conjunto se determina mediante validaci on cruzada. v

Essays on Tree-based Methods for Prediction and Causal Inference

The first chapter of this thesis contains an application of causal forests to a residential electricity smart meter trial dataset. Household specific estimates are obtained for the effect of a Time-of-Use pricing scheme on peak demand. The most and least responsive households differ across education, age, employment status, and past electricity consumption. The results suggest that past consumption information is more useful than pre-trial survey information, which includes building characteristics, household characteristics, and responses to appliance usage questions. The second chapter explores new variations of Bayesian tree-based machine learning algorithms. Bayesian Additive Regression Trees (BART) (Chipman et al. 2010) and Bayesian Causal Forests (BCF) (Hahn et al. 2020) are state-of-the-art machine learning methods for prediction and causal inference. A number of existing implementations of BART make use of Markov Chain Monte Carlo algorithms, which can be computationally expensive when applied to high-dimensional datasets, do not always perform well in terms of mixing of chains, and have limited parallelizability. The second chapter introduces four variations of BART that do not rely on MCMC: 1. An improved implementation of the existing method BART-BMA (Hernandez et al. 2018), which averages over sum-of-tree models found by a model search algorithm, performs well on high-dimensional datasets, and produces more interpretable output than other BART implementations because the output includes a comparatively small number of sum-of-tree models. %, each of which contains (under the default settings) 5 trees. Improvements are made to the model search algorithm, calculation of predictions, and credible intervals.% The algorithm is entirely deterministic. 2. A treatment effect estimation algorithm that combines the model structure of BCF with the implementation of BART-BMA (BCF-BMA). This method successfully accounts for confounding on observables using the BCF parameterization, while retaining the parsimonious model selection approach of BART-BMA. 3. A simple alternative BART implementation algorithm that uses importance sampling of models (BART-IS). This approach contrasts with existing MCMC and model-search based approaches in that BART-IS makes fast data-independent draws of many sum-of-tree models. The advantages of this approach are that it is straightforward to implement, fast, and trivially parallelizable. 4. Bayesian Causal Forests using Importance Sampling (BCF-IS). This is a combination of the BCF model framework with the BART-IS implementation. BART-IS and BCF-IS exhibit comparable performance to BART-MCMC and BCF across a large number of simulated datasets. The second chapter also includes some illustrative applications. The methods are extendable to multiple treatments, multivariate outcomes, and panel data methods. The third chapter of this thesis describes how the methods introduced in the second chapter can be generalized from regression and treatment effect estimation for continuous outcomes, to a range of models with various link functions and outcome variables. As examples of how to apply the general approach, Logit-BART-BMA and Logit-BART-IS are introduced with illustrative applications