Fast Genetic Algorithm for feature selection -- A qualitative approximation approach

Evolutionary Algorithms (EAs) are often challenging to apply in real-world settings since evolutionary computations involve a large number of evaluations of a typically expensive fitness function. For example, an evaluation could involve training a new machine learning model. An approximation (also known as meta-model or a surrogate) of the true function can be used in such applications to alleviate the computation cost. In this paper, we propose a two-stage surrogate-assisted evolutionary approach to address the computational issues arising from using Genetic Algorithm (GA) for feature selection in a wrapper setting for large datasets. We define 'Approximation Usefulness' to capture the necessary conditions to ensure correctness of the EA computations when an approximation is used. Based on this definition, we propose a procedure to construct a lightweight qualitative meta-model by the active selection of data instances. We then use a meta-model to carry out the feature selection task. We apply this procedure to the GA-based algorithm CHC (Cross generational elitist selection, Heterogeneous recombination and Cataclysmic mutation) to create a Qualitative approXimations variant, CHCQX. We show that CHCQX converges faster to feature subset solutions of significantly higher accuracy (as compared to CHC), particularly for large datasets with over 100K instances. We also demonstrate the applicability of the thinking behind our approach more broadly to Swarm Intelligence (SI), another branch of the Evolutionary Computation (EC) paradigm with results of PSOQX, a qualitative approximation adaptation of the Particle Swarm Optimization (PSO) method. A GitHub repository with the complete implementation is available

Rolling the dice for better deep learning performance: A study of randomness techniques in deep neural networks

This paper investigates how various randomization techniques impact Deep Neural Networks (DNNs). Randomization, like weight noise and dropout, aids in reducing overfitting and enhancing generalization, but their interactions are poorly understood. The study categorizes randomness techniques into four types and proposes new methods: adding noise to the loss function and random masking of gradient updates. Using Particle Swarm Optimizer (PSO) for hyperparameter optimization, it explores optimal configurations across MNIST, FASHION-MNIST, CIFAR10, and CIFAR100 datasets. Over 30,000 configurations are evaluated, revealing data augmentation and weight initialization randomness as main performance contributors. Correlation analysis shows different optimizers prefer distinct randomization types. The complete implementation and dataset are available on GitHub

Stacked Ensemble of Recurrent Neural Networks for Predicting Turbocharger Remaining Useful Life

Predictive Maintenance (PM) is a proactive maintenance strategy that tries to minimize a system’s downtime by predicting failures before they happen. It uses data from sensors to measure the component’s state of health and make forecasts about its future degradation. However, existing PM methods typically focus on individual measurements. While it is natural to assume that a history of measurements carries more information than a single one. This paper aims at incorporating such information into PM models. In practice, especially in the automotive domain, diagnostic models have low performance, due to a large amount of noise in the data and limited sensing capability. To address this issue, this paper proposes to use a specific type of ensemble learning known as Stacked Ensemble. The idea is to aggregate predictions of multiple models—consisting of Long Short-Term Memory (LSTM) and Convolutional-LSTM—via a meta model, in order to boost performance. Stacked Ensemble model performs well when its base models are as diverse as possible. To this end, each such model is trained using a specific combination of the following three aspects: feature subsets, past dependency horizon, and model architectures. Experimental results demonstrate benefits of the proposed approach on a case study of heavy-duty truck turbochargers. © 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/). HEALTH-VINNOV

Bridging the Gap: A Comparative Analysis of Regressive Remaining Useful Life Prediction and Survival Analysis Methods for Predictive Maintenance

Regressive Remaining Useful Life Prediction and Survival Analysis are two lines of research with similar goals but different origins; one from engineering and the other from survival study in clinical research. Although the two research paths share a common objective of predicting the time to an event, researchers from each path typically do not compare their methods with methods from the other direction. Given the mentioned gap, we propose a framework to compare methods from the two lines of research using run-to-failure datasets. Then by utilizing the proposed framework, we compare six models incorporating three widely recognized degradation models along with two learning algorithms. The first dataset used in this study is C-MAPSS which includes simulation data from aircraft turbofan engines. The second dataset is real-world data from streamed condition monitoring of turbocharger devices installed on a fleet of Volvo trucks

Domain Adaptation in Predicting Turbocharger Failures Using Vehicle's Sensor Measurements

The discrepancy in the distribution of source and target domains is usually referred to as a domain shift. It is one of the reasons for the inferior performance of machine learning solutions at deployment. We illustrate that the domain shift issue is pertinent to the readings of the vehicles’ operational sensors. This is due to the fact that these measurements are collected over a period of time and are susceptible to various changes that happen in the meantime. Examples of these changes are usage pattern variations, aging of the vehicles, seasonal shifts, and driver changes. However, domain adversarial neural networks (DANN) have shown promising results to reduce the negative impact of the domain shift. The present study investigates domain adaptation (DA) in the predictive maintenance field by estimating the remaining useful life (RUL) of turbochargers. The devices are operating on a fleet of VOLVO trucks, and the information about their services is collected over four years between 2016 and 2019. The input features to the model are a set of bi-weekly collected measurements called logged vehicle data (LVD). The contributions of this paper are two-fold. First, we propose a new approach for detecting domain (covariate) shift using an autoencoder. Second, we adapt domain adversarial neural networks to the specific application of predicting turbocharger failures. Finally, we deploy a recurrent feature extraction layer in the DANN architecture to incorporate temporal aspect of the data. The experimental results demonstrate the superiority of the proposed method over the traditional approach

AID4HAI : Automatic Idea Detection for Healthcare-Associated Infections from Twitter, A Framework based on Active Learning and Transfer Learning

This research is an interdisciplinary work between data scientists, innovation management researchers, and experts from a Swedish hygiene and health company. Based on this collaboration, we have developed a novel package for automatic idea detection to control and prevent healthcare-associated infections (HAI). The principal idea of this study is to use machine learning methods to extract informative ideas from social media to assist healthcare professionals in reducing the rate of HAI. Therefore, the proposed package offers a corpus of data collected from Twitter, associated expert-created labels, and software implementation of an annotation framework based on the Active Learning paradigm. We employed Transfer Learning and built a two-step deep neural network model that incrementally extracts the semantic representation of the collected text data using the BERTweet language model in the first step and classifies these representations as informative or non-informative using a multi-layer perception (MLP) in the second step. The package is AID4HAI (Automatic Idea Detection for controlling and preventing Healthcare-Associated Infections) and is made fully available (software code and the collected data) through a public GitHub repository (https://github.com/XaraKar/AID4HAI). We believe that sharing our ideas and releasing these ready-to-use tools contributes to the development of the field and inspires future research.Funding: KK-Foundation, Scania CV AB and the Vinnova program for Strategic Vehicle Research and Innovation (FFI).AID projec