A Recommendation System for Meta-modeling: A Meta-learning Based Approach

Various meta-modeling techniques have been developed to replace computationally expensive simulation models. The performance of these meta-modeling techniques on different models is varied which makes existing model selection/recommendation approaches (e.g., trial-and-error, ensemble) problematic. To address these research gaps, we propose a general meta-modeling recommendation system using meta-learning which can automate the meta-modeling recommendation process by intelligently adapting the learning bias to problem characterizations. The proposed intelligent recommendation system includes four modules: (1) problem module, (2) meta-feature module which includes a comprehensive set of meta-features to characterize the geometrical properties of problems, (3) meta-learner module which compares the performance of instance-based and model-based learning approaches for optimal framework design, and (4) performance evaluation module which introduces two criteria, Spearman\u27s ranking correlation coefficient and hit ratio, to evaluate the system on the accuracy of model ranking prediction and the precision of the best model recommendation, respectively. To further improve the performance of meta-learning for meta-modeling recommendation, different types of feature reduction techniques, including singular value decomposition, stepwise regression and ReliefF, are studied. Experiments show that our proposed framework is able to achieve 94% correlation on model rankings, and a 91% hit ratio on best model recommendation. Moreover, the computational cost of meta-modeling recommendation is significantly reduced from an order of minutes to seconds compared to traditional trial-and-error and ensemble process. The proposed framework can significantly advance the research in meta-modeling recommendation, and can be applied for data-driven system modeling

Monitoring and optimization of an autonomous learning system

Dissertação de mestrado em Informatics EngineeringIn the last years, the number of Machine Learning algorithms and their parameters has increased significantly. This allows for more accurate models to be found, but it also increases the complexity of the task of training a model, as the search space expands significantly. As datasets keep growing in size, traditional approaches based on extensive search start to become costly in terms of computational resources and time, especially in data streaming scenarios. With this growth, new challenges in Machine Learning started to appear. The speed at which data arrives and different ways of storing data are forcing organizations to address and explore new ways of adapting fast enough so their ML models don’t become obsolete. This dissertation aims to develop an approach based on meta-learning that tackles two main challenges: predict ing the performance metrics of a future model and recommending the best algorithm/configuration for training a model for a specific Machine Learning problem. Throughout this dissertation, all the study objectives and questions, along with the relevant contextualization will be exposed. The proposed solution, when compared to an AutoML approach is up to 130x faster and only 2% worse in terms of average model quality, showing it is a good solution for scenarios in which models need to be updated regularly, such as in streaming scenarios with Big Data, in which some accuracy can be traded for a much shorter model training time.Nos últimos anos, o número de algoritmos de Machine Learning e seus parâmetros aumentou significativamente. Isso permite que modelos mais precisos sejam encontrados, mas também aumenta a complexidade da tarefa de treinar um modelo, pois o espaço de busca expande-se significativamente. À medida que os conjuntos de dados continuam a crescer em tamanho, abordagens tradicionais baseadas em uma pesquisa extensiva começam a se tornar caras em termos de recursos computacionais e tempo, especialmente em cenários de streaming de dados. Com esse crescimento, novos desafios no Machine Learning começaram a aparecer. A velocidade com que os dados chegam e as diferentes maneiras de armazenar dados estão a forçar as organizações a abordar e explorar novas formas de se adaptar rápido o suficiente para que os seus modelos de ML não se tornem obsoletos. Esta dissertação visa desenvolver uma abordagem baseada em Meta-Learning que aborda dois desafios principais: prever as métricas de desempenho de um modelo futuro e recomendar o melhor algoritmo/configuração para treinar um modelo para um problema específico de Machine Learning. Ao longo desta dissertação, serão expostos todos os objetivos e questões do estudo, juntamente com a contextualização relevante. A solução proposta, quando comparada a uma abordagem AutoML é até 130x mais rápida e apenas 2% pior em termos de qualidade média do modelo, mostrando que é uma boa solução para cenários em que os modelos precisam ser atualizados regularmente, como em cenários de streaming com Big Data, em que alguma precisão pode ser negociada por um tempo de treino de modelo muito menor

Towards developing a reliable medical device for automated epileptic seizure detection in the ICU

Abstract. Epilepsy is a prevalent neurological disorder that affects millions of people globally, and its diagnosis typically involves laborious manual inspection of electroencephalography (EEG) data. Automated detection of epileptic seizures in EEG signals could potentially improve diagnostic accuracy and reduce diagnosis time, but there should be special attention to the number of false alarms to reduce unnecessary treatments and costs. This research presents a study on the use of machine learning techniques for EEG seizure detection with the aim of investigating the effectiveness of different algorithms in terms of high sensitivity and low false alarm rates for feature extraction, selection, pre-processing, classification, and post-processing in designing a medical device for detecting seizure activity in EEG data. The current state-of-the-art methods which are validated clinically using large amounts of data are introduced. The study focuses on finding potential machine learning methods, considering KNN, SVM, decision trees and, Random forests, and compares their performance on the task of seizure detection using features introduced in the literature. Also using ensemble methods namely, bootstrapping and majority voting techniques we achieved a sensitivity of 0.80 and FAR/h of 2.10, accuracy of 97.1% and specificity of 98.2%. Overall, the findings of this study can be useful for developing more accurate and efficient algorithms for EEG seizure detection medical device, which can contribute to the early diagnosis and treatment of epilepsy in the intensive care unit for critically ill patients

A Robust Classifier to Distinguish Noise from fMRI Independent Components

Analyzing Functional Magnetic Resonance Imaging (fMRI) of resting brains to determine the spatial location and activity of intrinsic brain networks–a novel and burgeoning research field–is limited by the lack of ground truth and the tendency of analyses to overfit the data. Independent Component Analysis (ICA) is commonly used to separate the data into signal and Gaussian noise components, and then map these components on to spatial networks. Identifying noise from this data, however, is a tedious process that has proven hard to automate, particularly when data from different institutions, subjects, and scanners is used. Here we present an automated method to delineate noisy independent components in ICA using a data-driven infrastructure that queries a database of 246 spatial and temporal features to discover a computational signature of different types of noise. We evaluated the performance of our method to detect noisy components from healthy control fMRI (sensitivity = 0.91, specificity = 0.82, cross validation accuracy (CVA) = 0.87, area under the curve (AUC) = 0.93), and demonstrate its generalizability by showing equivalent performance on (1) an age- and scanner-matched cohort of schizophrenia patients from the same institution (sensitivity = 0.89, specificity = 0.83, CVA = 0.86), (2) an agematched cohort on an equivalent scanner from a different institution (sensitivity = 0.88, specificity = 0.88, CVA = 0.88), and (3) an age-matched cohort on a different scanner from a different institution (sensitivity = 0.72, specificity = 0.92, CVA = 0.79). We additionally compare our approach with a recently published method [1]. Our results suggest that our method is robust to noise variations due to population as well as scanner differences, thereby making it well suited to the goal of automatically distinguishing noise from functional networks to enable investigation of human brain function

A Robust Classifier to Distinguish Noise from fMRI Independent Components

Analyzing Functional Magnetic Resonance Imaging (fMRI) of resting brains to determine the spatial location and activity of intrinsic brain networks–a novel and burgeoning research field–is limited by the lack of ground truth and the tendency of analyses to overfit the data. Independent Component Analysis (ICA) is commonly used to separate the data into signal and Gaussian noise components, and then map these components on to spatial networks. Identifying noise from this data, however, is a tedious process that has proven hard to automate, particularly when data from different institutions, subjects, and scanners is used. Here we present an automated method to delineate noisy independent components in ICA using a data-driven infrastructure that queries a database of 246 spatial and temporal features to discover a computational signature of different types of noise. We evaluated the performance of our method to detect noisy components from healthy control fMRI (sensitivity = 0.91, specificity = 0.82, cross validation accuracy (CVA) = 0.87, area under the curve (AUC) = 0.93), and demonstrate its generalizability by showing equivalent performance on (1) an age- and scanner-matched cohort of schizophrenia patients from the same institution (sensitivity = 0.89, specificity = 0.83, CVA = 0.86), (2) an agematched cohort on an equivalent scanner from a different institution (sensitivity = 0.88, specificity = 0.88, CVA = 0.88), and (3) an age-matched cohort on a different scanner from a different institution (sensitivity = 0.72, specificity = 0.92, CVA = 0.79). We additionally compare our approach with a recently published method [1]. Our results suggest that our method is robust to noise variations due to population as well as scanner differences, thereby making it well suited to the goal of automatically distinguishing noise from functional networks to enable investigation of human brain function

Computational Intelligence in Healthcare

This book is a printed edition of the Special Issue Computational Intelligence in Healthcare that was published in Electronic

Brain Microstructure: Impact of the Permeability on Diffusion MRI

Diffusion Magnetic Resonance Imaging (dMRI) enables a non invasive in-vivo characterization of the brain tissue. The disentanglement of each microstructural property reflected on the total dMRI signal is one of the hottest topics in the field. The dMRI reconstruction techniques ground on assumptions on the signal model and consider the neurons axons as impermeable cylinders. Nevertheless, interactions with the environment is characteristic of the biological life and diffusional water exchange takes place through cell membranes. Myelin wraps axons with multiple layers constitute a barrier modulating exchange between the axon and the extracellular tissue. Due to the short transverse relaxation time (T2) of water trapped between sheets, myelin contribution to the diffusion signal is often neglected. This thesis aims to explore how the exchange influences the dMRI signal and how this can be informative on myelin structure. We also aimed to explore how recent dMRI signal reconstruction techniques could be applied in clinics proposing a strategy for investigating the potential as biomarkers of the derived tissue descriptors. The first goal of the thesis was addressed performing Monte Carlo simulations of a system with three compartments: intra-axonal, spiraling myelin and extra-axonal. The experiments showed that the exchange time between intra- and extra-axonal compartments was on the sub-second level (and thus possibly observable) for geometries with small axon diameter and low number of wraps such as in the infant brain and in demyelinating diseases. The second goal of the thesis was reached by assessing the indices derived from three dimensional simple harmonics oscillator-based reconstruction and estimation (3D-SHORE) in stroke disease. The tract-based analysis involving motor networks and the region-based analysis in grey matter (GM) were performed. 3D-SHORE indices proved to be sensitive to plasticity in both white matter (WM) and GM, highlighting their viability as biomarkers in ischemic stroke. The overall study could be considered the starting point for a future investigation of the interdependence of different phenomena like exchange and relaxation related to the established dMRI indices. This is valuable for the accurate dMRI data interpretation in heterogeneous tissues and different physiological conditions

Computational Intelligence in Healthcare

The number of patient health data has been estimated to have reached 2314 exabytes by 2020. Traditional data analysis techniques are unsuitable to extract useful information from such a vast quantity of data. Thus, intelligent data analysis methods combining human expertise and computational models for accurate and in-depth data analysis are necessary. The technological revolution and medical advances made by combining vast quantities of available data, cloud computing services, and AI-based solutions can provide expert insight and analysis on a mass scale and at a relatively low cost. Computational intelligence (CI) methods, such as fuzzy models, artificial neural networks, evolutionary algorithms, and probabilistic methods, have recently emerged as promising tools for the development and application of intelligent systems in healthcare practice. CI-based systems can learn from data and evolve according to changes in the environments by taking into account the uncertainty characterizing health data, including omics data, clinical data, sensor, and imaging data. The use of CI in healthcare can improve the processing of such data to develop intelligent solutions for prevention, diagnosis, treatment, and follow-up, as well as for the analysis of administrative processes. The present Special Issue on computational intelligence for healthcare is intended to show the potential and the practical impacts of CI techniques in challenging healthcare applications

The impact of macroeconomic leading indicators on inventory management

Forecasting tactical sales is important for long term decisions such as procurement and informing lower level inventory management decisions. Macroeconomic indicators have been shown to improve the forecast accuracy at tactical level, as these indicators can provide early warnings of changing markets while at the same time tactical sales are sufficiently aggregated to facilitate the identification of useful leading indicators. Past research has shown that we can achieve significant gains by incorporating such information. However, at lower levels, that inventory decisions are taken, this is often not feasible due to the level of noise in the data. To take advantage of macroeconomic leading indicators at this level we need to translate the tactical forecasts into operational level ones. In this research we investigate how to best assimilate top level forecasts that incorporate such exogenous information with bottom level (at Stock Keeping Unit level) extrapolative forecasts. The aim is to demonstrate whether incorporating these variables has a positive impact on bottom level planning and eventually inventory levels. We construct appropriate hierarchies of sales and use that structure to reconcile the forecasts, and in turn the different available information, across levels. We are interested both at the point forecast and the prediction intervals, as the latter inform safety stock decisions. Therefore the contribution of this research is twofold. We investigate the usefulness of macroeconomic leading indicators for SKU level forecasts and alternative ways to estimate the variance of hierarchically reconciled forecasts. We provide evidence using a real case study