Artificial neural network technique for improving prediction of credit card default: A stacked sparse autoencoder approach

Presently, the use of a credit card has become an integral part of contemporary banking and financial system. Predicting potential credit card defaulters or debtors is a crucial business opportunity for financial institutions. For now, some machine learning methods have been applied to achieve this task. However, with the dynamic and imbalanced nature of credit card default data, it is challenging for classical machine learning algorithms to proffer robust models with optimal performance. Research has shown that the performance of machine learning algorithms can be significantly improved when provided with optimal features. In this paper, we propose an unsupervised feature learning method to improve the performance of various classifiers using a stacked sparse autoencoder (SSAE). The SSAE was optimized to achieve improved performance. The proposed SSAE learned excellent feature representations that were used to train the classifiers. The performance of the proposed approach is compared with an instance where the classifiers were trained using the raw data. Also, a comparison is made with previous scholarly works, and the proposed approach showed superior performance over other methods

Power disturbance monitoring through techniques for novelty detection on wind power and photovoltaic generation

Novelty detection is a statistical method that verifies new or unknown data, determines whether these data are inliers (within the norm) or outliers (outside the norm), and can be used, for example, in developing classification strategies in machine learning systems for industrial applications. To this end, two types of energy that have evolved over time are solar photovoltaic and wind power generation. Some organizations around the world have developed energy quality standards to avoid known electric disturbances; however, their detection is still a challenge. In this work, several techniques for novelty detection are implemented to detect different electric anomalies (disturbances), which are k-nearest neighbors, Gaussian mixture models, one-class support vector machines, self-organizing maps, stacked autoencoders, and isolation forests. These techniques are applied to signals from real power quality environments of renewable energy systems such as solar photovoltaic and wind power generation. The power disturbances that will be analyzed are considered in the standard IEEE-1159, such as sag, oscillatory transient, flicker, and a condition outside the standard attributed to meteorological conditions. The contribution of the work consists of the development of a methodology based on six techniques for novelty detection of power disturbances, under known and unknown conditions, over real signals in the power quality assessment. The merit of the methodology is a set of techniques that allow to obtain the best performance of each one under different conditions, which constitutes an important contribution to the renewable energy systems.Postprint (published version

Deep Learning for predictive maintenance

Recently, with the appearance of Industry 4.0 (I4.0), machine learning (ML) within artificial intelligence (AI), industrial Internet of things (IIoT) and cyber-physical system (CPS) have accelerated the development of a data-orientated applications such as predictive maintenance (PdM). PdM applied to asset-dependent industries has led to operational cost savings, productivity improvements and enhanced safety management capabilities. In addition, predictive maintenance strategies provide useful information concerning the source of the failure or malfunction, reducing unnecessary maintenance operations. The concept of prognostics and health management (PHM) has appeared as a predictive maintenance process. PHM has become an unavoidable tendency in smart manufacturing to offer a reliable solution for handling industrial equipment’s health status. This later requires efficient and effective system health monitoring methods, including processing and analysing massive machinery data to detect anomalies and perform diagnosis and prognosis. Prognostics is considered a key PHM process with capabilities for predicting future states, mainly based on predicting the residual lifetime during which a machine can perform its intended function, i.e., estimating the remaining useful life (RUL) of a system. The prognostic research domain is far from being mature, which is still new and explains the various challenges that must be addressed. Therefore, the work presented in this thesis will mainly focus on the prognostic of monitored machinery from an RUL estimation point of view using Deep Learning (DL) algorithms. Capitalising on the recent success of the DL, this dissertation introduces methods and algorithms dedicated to predictive maintenance. We focused on improving the performance of aero-engine prognostic, particularly in estimating an accurate RUL using ensemble learning and deep learning. To this end, two contributions have been proposed, and the results obtained were validated by an extensive comparative analysis using public C-MAPSS turbofan engine benchmark datasets. The first contribution, for RUL predictions, we proposed two-hybrid methods based on the promising DL architectures to leverage the power of the multimodal and hybrid deep neural network in order to capture various information at different time intervals and ultimately achieve more accurate RUL predictions. The proposed end-to-end deep architectures jointly optimise the feature reduction and RUL prediction steps in a hierarchical manner, intending to achieve data representation in low dimensionality and minimal variable redundancy while preserving critical asset degradation information with minimal preprocessing effort. The second contribution, in a practical situation, RUL is usually affected by uncertainty. Therefore, we proposed an innovative RUL estimation strategy that assesses degrading machinery’s health status (provides the probabilities of system failure in different time windows) and provides the prediction of RUL window. Keywords: Prognostics and Health Management (PHM), Remaining useful life (RUL), Predictive Maintenance (PdM), C-MAPSS dataset, Ensemble learning, Deep learnin

Artificial Intelligence-based Technique for Fault Detection and Diagnosis of EV Motors: A Review

The motor drive system plays a significant role in the safety of electric vehicles as a bridge for power transmission. Meanwhile, to enhance the efficiency and stability of the drive system, more and more studies based on AI technology are devoted to the fault detection and diagnosis of the motor drive system. This paper reviews the application of AI techniques in motor fault detection and diagnosis in recent years. AI-based FDD is divided into two main steps: feature extraction and fault classification. The application of different signal processing methods in feature extraction is discussed. In particular, the application of traditional machine learning and deep learning algorithms for fault classification is presented in detail. In addition, the characteristics of all techniques reviewed are summarized. Finally, the latest developments, research gaps and future challenges in fault monitoring and diagnosis of motor faults are discussed

Foundations and Recent Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions

Multimodal machine learning is a vibrant multi-disciplinary research field that aims to design computer agents with intelligent capabilities such as understanding, reasoning, and learning through integrating multiple communicative modalities, including linguistic, acoustic, visual, tactile, and physiological messages. With the recent interest in video understanding, embodied autonomous agents, text-to-image generation, and multisensor fusion in application domains such as healthcare and robotics, multimodal machine learning has brought unique computational and theoretical challenges to the machine learning community given the heterogeneity of data sources and the interconnections often found between modalities. However, the breadth of progress in multimodal research has made it difficult to identify the common themes and open questions in the field. By synthesizing a broad range of application domains and theoretical frameworks from both historical and recent perspectives, this paper is designed to provide an overview of the computational and theoretical foundations of multimodal machine learning. We start by defining two key principles of modality heterogeneity and interconnections that have driven subsequent innovations, and propose a taxonomy of 6 core technical challenges: representation, alignment, reasoning, generation, transference, and quantification covering historical and recent trends. Recent technical achievements will be presented through the lens of this taxonomy, allowing researchers to understand the similarities and differences across new approaches. We end by motivating several open problems for future research as identified by our taxonomy

DEEP-AD: The deep learning model for diagnostic classification and prognostic prediction of alzheimer's disease

In terms of context, the aim of this dissertation is to aid neuroradiologists in their clinical judgment regarding the early detection of AD by using DL. To that aim, the system design research methodology is suggested in this dissertation for achieving three goals. The first goal is to investigate the DL models that have performed well at identifying patterns associated with AD, as well as the accuracy so far attained, limitations, and gaps. A systematic review of the literature (SLR) revealed a shortage of empirical studies on the early identification of AD through DL. In this regard, thirteen empirical studies were identified and examined. We concluded that three-dimensional (3D) DL models have been generated far less often and that their performance is also inadequate to qualify them for clinical trials. The second goal is to provide the neuroradiologist with the computer-interpretable information they need to analyze neuroimaging biomarkers. Given this context, the next step in this dissertation is to find the optimum DL model to analyze neuroimaging biomarkers. It has been achieved in two steps. In the first step, eight state-of-the-art DL models have been implemented by training from scratch using end-to-end learning (E2EL) for two binary classification tasks (AD vs. CN and AD vs. stable MCI) and compared by utilizing MRI scans from the publicly accessible datasets of neuroimaging biomarkers. Comparative analysis is carried out by utilizing efficiency-effects graphs, comprehensive indicators, and ranking mechanisms. For the training of the AD vs. sMCI task, the EfficientNet-B0 model gets the highest value for the comprehensive indicator and has the fewest parameters. DenseNet264 performed better than the others in terms of evaluation matrices, but since it has the most parameters, it costs more to train. For the AD vs. CN task by DenseNet264, we achieved 100% accuracy for training and 99.56% accuracy for testing. However, the classification accuracy was still only 82.5% for the AD vs. sMCI task. In the second step, fusion of transfer learning (TL) with E2EL is applied to train the EfficientNet-B0 for the AD vs. sMCI task, which achieved 95.29% accuracy for training and 93.10% accuracy for testing. Additionally, we have also implemented EfficientNet-B0 for the multiclass AD vs. CN vs. sMCI classification task with E2EL to be used in ensemble of models and achieved 85.66% training accuracy and 87.38% testing accuracy. To evaluate the model’s robustness, neuroradiologists must validate the implemented model. As a result, the third goal of this dissertation is to create a tool that neuroradiologists may use at their convenience. To achieve this objective, this dissertation proposes a web-based application (DEEP-AD) that has been created by making an ensemble of Efficient-Net B0 and DenseNet 264 (based on the contribution of goal 2). The accuracy of a DEEP-AD prototype has undergone repeated evaluation and improvement. First, we validated 41 subjects of Spanish MRI datasets (acquired from HT Medica, Madrid, Spain), achieving an accuracy of 82.90%, which was later verified by neuroradiologists. The results of these evaluation studies showed the accomplishment of such goals and relevant directions for future research in applied DL for the early detection of AD in clinical settings.En términos de contexto, el objetivo de esta tesis es ayudar a los neurorradiólogos en su juicio clínico sobre la detección precoz de la AD mediante el uso de DL. Para ello, en esta tesis se propone la metodología de investigación de diseño de sistemas para lograr tres objetivos. El segundo objetivo es proporcionar al neurorradiólogo la información interpretable por ordenador que necesita para analizar los biomarcadores de neuroimagen. Dado este contexto, el siguiente paso en esta tesis es encontrar el modelo DL óptimo para analizar biomarcadores de neuroimagen. Esto se ha logrado en dos pasos. En el primer paso, se han implementado ocho modelos DL de última generación mediante entrenamiento desde cero utilizando aprendizaje de extremo a extremo (E2EL) para dos tareas de clasificación binarias (AD vs. CN y AD vs. MCI estable) y se han comparado utilizando escaneos MRI de los conjuntos de datos de biomarcadores de neuroimagen de acceso público. El análisis comparativo se lleva a cabo utilizando gráficos de efecto-eficacia, indicadores exhaustivos y mecanismos de clasificación. Para el entrenamiento de la tarea AD vs. sMCI, el modelo EfficientNet-B0 obtiene el valor más alto para el indicador exhaustivo y tiene el menor número de parámetros. DenseNet264 obtuvo mejores resultados que los demás en términos de matrices de evaluación, pero al ser el que tiene más parámetros, su entrenamiento es más costoso. Para la tarea AD vs. CN de DenseNet264, conseguimos una accuracy del 100% en el entrenamiento y del 99,56% en las pruebas. Sin embargo, la accuracy de la clasificación fue sólo del 82,5% para la tarea AD vs. sMCI. En el segundo paso, se aplica la fusión del aprendizaje por transferencia (TL) con E2EL para entrenar la EfficientNet-B0 para la tarea AD vs. sMCI, que alcanzó una accuracy del 95,29% en el entrenamiento y del 93,10% en las pruebas. Además, también hemos implementado EfficientNet-B0 para la tarea de clasificación multiclase AD vs. CN vs. sMCI con E2EL para su uso en conjuntos de modelos y hemos obtenido una accuracy de entrenamiento del 85,66% y una precisión de prueba del 87,38%. Para evaluar la solidez del modelo, los neurorradiólogos deben validar el modelo implementado. Como resultado, el tercer objetivo de esta disertación es crear una herramienta que los neurorradiólogos puedan utilizar a su conveniencia. Para lograr este objetivo, esta disertación propone una aplicación basada en web (DEEP-AD) que ha sido creada haciendo un ensemble de Efficient-Net B0 y DenseNet 264 (basado en la contribución del objetivo 2). La accuracy del prototipo DEEP-AD ha sido sometida a repetidas evaluaciones y mejoras. En primer lugar, validamos 41 sujetos de conjuntos de datos de MRI españoles (adquiridos de HT Medica, Madrid, España), logrando una accuracy del 82,90%, que posteriormente fue verificada por neurorradiólogos. Los resultados de estos estudios de evaluación mostraron el cumplimiento de dichos objetivos y las direcciones relevantes para futuras investigaciones en DL, aplicada en la detección precoz de la AD en entornos clínicos.Escuela de DoctoradoDoctorado en Tecnologías de la Información y las Telecomunicacione

Online Condition Monitoring of Electric Powertrains using Machine Learning and Data Fusion

Safe and reliable operations of industrial machines are highly prioritized in industry. Typical industrial machines are complex systems, including electric motors, gearboxes and loads. A fault in critical industrial machines may lead to catastrophic failures, service interruptions and productivity losses, thus condition monitoring systems are necessary in such machines. The conventional condition monitoring or fault diagnosis systems using signal processing, time and frequency domain analysis of vibration or current signals are widely used in industry, requiring expensive and professional fault analysis team. Further, the traditional diagnosis methods mainly focus on single components in steady-state operations. Under dynamic operating conditions, the measured quantities are non-stationary, thus those methods cannot provide reliable diagnosis results for complex gearbox based powertrains, especially in multiple fault contexts. In this dissertation, four main research topics or problems in condition monitoring of gearboxes and powertrains have been identified, and novel solutions are provided based on data-driven approach. The first research problem focuses on bearing fault diagnosis at early stages and dynamic working conditions. The second problem is to increase the robustness of gearbox mixed fault diagnosis under noise conditions. Mixed fault diagnosis in variable speeds and loads has been considered as third problem. Finally, the limitation of labelled training or historical failure data in industry is identified as the main challenge for implementing data-driven algorithms. To address mentioned problems, this study aims to propose data-driven fault diagnosis schemes based on order tracking, unsupervised and supervised machine learning, and data fusion. All the proposed fault diagnosis schemes are tested with experimental data, and key features of the proposed solutions are highlighted with comparative studies.publishedVersio