Facing-up Challenges of Multiobjective Clustering Based on Evolutionary Algorithms: Representations, Scalability and Retrieval Solutions

Aquesta tesi es centra en algorismes de clustering multiobjectiu, que estan basats en optimitzar varis objectius simultàniament obtenint una col•lecció de solucions potencials amb diferents compromisos entre objectius. El propòsit d'aquesta tesi consisteix en dissenyar i implementar un nou algorisme de clustering multiobjectiu basat en algorismes evolutius per afrontar tres reptes actuals relacionats amb aquest tipus de tècniques. El primer repte es centra en definir adequadament l'àrea de possibles solucions que s'explora per obtenir la millor solució i que depèn de la representació del coneixement. El segon repte consisteix en escalar el sistema dividint el conjunt de dades original en varis subconjunts per treballar amb menys dades en el procés de clustering. El tercer repte es basa en recuperar la solució més adequada tenint en compte la qualitat i la forma dels clusters a partir de la regió més interessant de la col•lecció de solucions ofertes per l’algorisme.Esta tesis se centra en los algoritmos de clustering multiobjetivo, que están basados en optimizar varios objetivos simultáneamente obteniendo una colección de soluciones potenciales con diferentes compromisos entre objetivos. El propósito de esta tesis consiste en diseñar e implementar un nuevo algoritmo de clustering multiobjetivo basado en algoritmos evolutivos para afrontar tres retos actuales relacionados con este tipo de técnicas. El primer reto se centra en definir adecuadamente el área de posibles soluciones explorada para obtener la mejor solución y que depende de la representación del conocimiento. El segundo reto consiste en escalar el sistema dividiendo el conjunto de datos original en varios subconjuntos para trabajar con menos datos en el proceso de clustering El tercer reto se basa en recuperar la solución más adecuada según la calidad y la forma de los clusters a partir de la región más interesante de la colección de soluciones ofrecidas por el algoritmo.This thesis is focused on multiobjective clustering algorithms, which are based on optimizing several objectives simultaneously obtaining a collection of potential solutions with different trade¬offs among objectives. The goal of the thesis is to design and implement a new multiobjective clustering technique based on evolutionary algorithms for facing up three current challenges related to these techniques. The first challenge is focused on successfully defining the area of possible solutions that is explored in order to find the best solution, and this depends on the knowledge representation. The second challenge tries to scale-up the system splitting the original data set into several data subsets in order to work with less data in the clustering process. The third challenge is addressed to the retrieval of the most suitable solution according to the quality and shape of the clusters from the most interesting region of the collection of solutions returned by the algorithm

Unsupervised Algorithms for Microarray Sample Stratification

The amount of data made available by microarrays gives researchers the opportunity to delve into the complexity of biological systems. However, the noisy and extremely high-dimensional nature of this kind of data poses significant challenges. Microarrays allow for the parallel measurement of thousands of molecular objects spanning different layers of interactions. In order to be able to discover hidden patterns, the most disparate analytical techniques have been proposed. Here, we describe the basic methodologies to approach the analysis of microarray datasets that focus on the task of (sub)group discovery.Peer reviewe

MOCF: A Multi-Objective Clustering Framework using an Improved Particle Swarm Optimization Algorithm

Traditional clustering algorithms, such as K-Means, perform clustering with a single goal in mind. However, in many real-world applications, multiple objective functions must be considered at the same time. Furthermore, traditional clustering algorithms have drawbacks such as centroid selection, local optimal, and convergence. Particle Swarm Optimization (PSO)-based clustering approaches were developed to address these shortcomings. Animals and their social Behaviour, particularly bird flocking and fish schooling, inspire PSO. This paper proposes the Multi-Objective Clustering Framework (MOCF), an improved PSO-based framework. As an algorithm, a Particle Swarm Optimization (PSO) based Multi-Objective Clustering (PSO-MOC) is proposed. It significantly improves clustering efficiency. The proposed framework's performance is evaluated using a variety of real-world datasets. To test the performance of the proposed algorithm, a prototype application was built using the Python data science platform. The empirical results showed that multi-objective clustering outperformed its single-objective counterparts

Online suicide prevention through optimised text classification

Online communication platforms are increasingly used to express suicidal thoughts. There is considerable interest in monitoring such messages, both for population-wide and individual prevention purposes, and to inform suicide research and policy. Online information overload prohibits manual detection, which is why keyword search methods are typically used. However, these are imprecise and unable to handle implicit references or linguistic noise. As an alternative, this study investigates supervised text classification to model and detect suicidality in Dutch-language forum posts. Genetic algorithms were used to optimise models through feature selection and hyperparameter optimisation. A variety of features was found to be informative, including token and character ngram bags-of-words, presence of salient suicide-related terms and features based on LSA topic models and polarity lexicons. The results indicate that text classification is a viable and promising strategy for detecting suicide-related and alarming messages, with F-scores comparable to human annotators (93% for relevant messages, 70% for severe messages). Both types of messages can be detected with high precision and minimal noise, even on large high-skew corpora. This suggests that they would be fit for use in a real-world prevention setting

Learning predictive models from temporal three-way data using triclustering: applications in clinical data analysis

Tese de mestrado, Ciência de Dados, Universidade de Lisboa, Faculdade de Ciências, 2020O conceito de triclustering estende o conceito de biclustering para um espaço tridimensional, cujo o objetivo é encontrar subespaços coerentes em dados tridimensionais. Considerando dados com dimensão temporal, a necessidade de aprender padrões temporais interessantes e usá-los para aprender modelos preditivos efetivos e interpretáveis, despoleta necessidade em investigar novas metodologias para análise de dados tridimensionais. Neste trabalho, propomos duas metodologias para esse efeito. Na primeira metodologia, encontramos os melhores parâmetros a serem usados em triclustering para descobrir os melhores triclusters (conjuntos de objetos com um padrão coerente ao longo de um dado conjunto de pontos temporais) para que depois estes padrões sejam usados como features por um dos mais apropriados classificadores encontrados na literatura. Neste caso, propomos juntar o classificador com uma abordagem de triclustering temporal. Para isso, idealizámos um algoritmo de triclustering com uma restrição temporal, denominado TCtriCluster para desvendar triclusters temporalmente contínuos (constituídos por pontos temporais contínuos). Na segunda metodologia, adicionámos uma fase de biclustering para descobrir padrões nos dados estáticos (dados que não mudam ao longo do tempo) e juntá-los aos triclusters para melhorar o desempenho e a interpretabilidade dos modelos. Estas metodologias foram usadas para prever a necessidade de administração de ventilação não invasiva (VNI) em pacientes com Esclerose Lateral Amiotrófica (ELA). Neste caso de estudo, aprendemos modelos de prognóstico geral, para os dados de todos os pacientes, e modelos especializados, depois de feita uma estratificação dos pacientes em 3 grupos de progressão: Lentos, Neutros e Rápidos. Os resultados demonstram que, além de serem bastante equiparáveis e por vezes superiores quando comparados com os resultados obtidos por um classificador de alto desempenho (Random Forests), os nossos classificadores são capazes de refinar as previsões através das potencialidades da interpretabilidade do modelo. De facto, quando usados os triclusters (e biclusters) como previsores, estamos a promover o uso de padrões de progressão da doença altamente interpretáveis. Para além disso, quando usados para previsão de prognóstico em doentes com ELA, os nossos modelos preditivos interpretáveis desvendaram padrões clinicamente relevantes para um grupo específico de padrões de progressão da doença, ajudando os médicos a entender a elevada heterogeneidade da progressão da ELA. Os resultados mostram ainda que a restrição temporal tem impacto na melhoria da efetividade e preditividade dos modelos.Triclustering extends biclustering to the three-dimensional space, aiming to find coherent subspaces in three-way data (sets of objects described by subsets of features in a subset of contexts). When the context is time, the need to learn interesting temporal patterns and use them to learn effective and interpretable predictive models triggers the need for new research methodologies to be used in three-way data analysis. In this work, we propose two approaches to learn predictive models from three-way data: 1) a triclustering-based classifier (considering just temporal data) and 2) a mixture of biclustering (with static data) and triclustering (with temporal data). In the first approach, we find the best triclustering parameters to uncover the best triclusters (sets of objects with a coherent pattern along a set of time-points) and then use these patterns as features in a state-of-the-art classifier. In the case of temporal data, we propose to couple the classifier with a temporal triclustering approach. With this aim, we devised a temporally constrained triclustering algorithm, termed TCtriCluster algorithm to mine time-contiguous triclusters. In the second approach, we extended the triclustering-based classifier with a biclustering task, where biclusters are discovered in static data (not changed over the time) and integrated with triclusters to improve performance and model explainability. The proposed methodologies were used to predict the need for non-invasive ventilation (NIV) in patients with Amyotrophic Lateral Sclerosis (ALS). In this case study, we learnt a general prognostic model from all patients data and specialized models after patient stratification into Slow, Neutral and Fast progressors. Our results show that besides comparable and sometimes outperforming results, when compared to a high performing random forest classifier, our predictive models enhance prediction with the potentialities of model interpretability. Indeed, when using triclusters (and biclusters) as predictors, we promoting the use of highly interpretable disease progression patterns. Furthermore, when used for prognostic prediction in ALS, our interpretable predictive models unravelled clinically relevant and group-specific disease progression patterns, helping clinicians to understand the high heterogeneity of ALS disease progression. Results further show that the temporal restriction is effective in improving the effectiveness of the predictive models

Novel sampling techniques for reservoir history matching optimisation and uncertainty quantification in flow prediction

Modern reservoir management has an increasing focus on accurately predicting the likely range of field recoveries. A variety of assisted history matching techniques has been developed across the research community concerned with this topic. These techniques are based on obtaining multiple models that closely reproduce the historical flow behaviour of a reservoir. The set of resulted history matched models is then used to quantify uncertainty in predicting the future performance of the reservoir and providing economic evaluations for different field development strategies. The key step in this workflow is to employ algorithms that sample the parameter space in an efficient but appropriate manner. The algorithm choice has an impact on how fast a model is obtained and how well the model fits the production data. The sampling techniques that have been developed to date include, among others, gradient based methods, evolutionary algorithms, and ensemble Kalman filter (EnKF). This thesis has investigated and further developed the following sampling and inference techniques: Particle Swarm Optimisation (PSO), Hamiltonian Monte Carlo, and Population Markov Chain Monte Carlo. The inspected techniques have the capability of navigating the parameter space and producing history matched models that can be used to quantify the uncertainty in the forecasts in a faster and more reliable way. The analysis of these techniques, compared with Neighbourhood Algorithm (NA), has shown how the different techniques affect the predicted recovery from petroleum systems and the benefits of the developed methods over the NA. The history matching problem is multi-objective in nature, with the production data possibly consisting of multiple types, coming from different wells, and collected at different times. Multiple objectives can be constructed from these data and explicitly be optimised in the multi-objective scheme. The thesis has extended the PSO to handle multi-objective history matching problems in which a number of possible conflicting objectives must be satisfied simultaneously. The benefits and efficiency of innovative multi-objective particle swarm scheme (MOPSO) are demonstrated for synthetic reservoirs. It is demonstrated that the MOPSO procedure can provide a substantial improvement in finding a diverse set of good fitting models with a fewer number of very costly forward simulations runs than the standard single objective case, depending on how the objectives are constructed. The thesis has also shown how to tackle a large number of unknown parameters through the coupling of high performance global optimisation algorithms, such as PSO, with model reduction techniques such as kernel principal component analysis (PCA), for parameterising spatially correlated random fields. The results of the PSO-PCA coupling applied to a recent SPE benchmark history matching problem have demonstrated that the approach is indeed applicable for practical problems. A comparison of PSO with the EnKF data assimilation method has been carried out and has concluded that both methods have obtained comparable results on the example case. This point reinforces the need for using a range of assisted history matching algorithms for more confidence in predictions

Contributions on evolutionary computation for statistical inference

Evolutionary Computation (EC) techniques have been introduced in the 1960s for dealing with complex situations. One possible example is an optimization problems not having an analytical solution or being computationally intractable; in many cases such methods, named Evolutionary Algorithms (EAs), have been successfully implemented. In statistics there are many situations where complex problems arise, in particular concerning optimization. A general example is when the statistician needs to select, inside a prohibitively large discrete set, just one element, which could be a model, a partition, an experiment, or such: this would be the case of model selection, cluster analysis or design of experiment. In other situations there could be an intractable function of data, such as a likelihood, which needs to be maximized, as it happens in model parameter estimation. These kind of problems are naturally well suited for EAs, and in the last 20 years a large number of papers has been concerned with applications of EAs in tackling statistical issues. The present dissertation is set in this part of literature, as it reports several implementations of EAs in statistics, although being mainly focused on statistical inference problems. Original results are proposed, as well as overviews and surveys on several topics. EAs are employed and analyzed considering various statistical points of view, showing and confirming their efficiency and flexibility. The first proposal is devoted to parametric estimation problems. When EAs are employed in such analysis a novel form of variability related to their stochastic elements is introduced. We shall analyze both variability due to sampling, associated with selected estimator, and variability due to the EA. This analysis is set in a framework of statistical and computational tradeoff question, crucial in nowadays problems, by introducing cost functions related to both data acquisition and EA iterations. The proposed method will be illustrated by means of model building problem examples. Subsequent chapter is concerned with EAs employed in Markov Chain Monte Carlo (MCMC) sampling. When sampling from multimodal or highly correlated distribution is concerned, in fact, a possible strategy suggests to run several chains in parallel, in order to improve their mixing. If these chains are allowed to interact with each other then many analogies with EC techniques can be observed, and this has led to research in many fields. The chapter aims at reviewing various methods found in literature which conjugates EC techniques and MCMC sampling, in order to identify specific and common procedures, and unifying them in a framework of EC. In the last proposal we present a complex time series model and an identification procedure based on Genetic Algorithms (GAs). The model is capable of dealing with seasonality, by Periodic AutoRegressive (PAR) modelling, and structural changes in time, leading to a nonstationary structure. As far as a very large number of parameters and possibilites of change points are concerned, GAs are appropriate for identifying such model. Effectiveness of procedure is shown on both simulated data and real examples, these latter referred to river flow data in hydrology. The thesis concludes with some final remarks, concerning also future work

Contributions on evolutionary computation for statistical inference

Evolutionary Computation (EC) techniques have been introduced in the 1960s for dealing with complex situations. One possible example is an optimization problems not having an analytical solution or being computationally intractable; in many cases such methods, named Evolutionary Algorithms (EAs), have been successfully implemented. In statistics there are many situations where complex problems arise, in particular concerning optimization. A general example is when the statistician needs to select, inside a prohibitively large discrete set, just one element, which could be a model, a partition, an experiment, or such: this would be the case of model selection, cluster analysis or design of experiment. In other situations there could be an intractable function of data, such as a likelihood, which needs to be maximized, as it happens in model parameter estimation. These kind of problems are naturally well suited for EAs, and in the last 20 years a large number of papers has been concerned with applications of EAs in tackling statistical issues. The present dissertation is set in this part of literature, as it reports several implementations of EAs in statistics, although being mainly focused on statistical inference problems. Original results are proposed, as well as overviews and surveys on several topics. EAs are employed and analyzed considering various statistical points of view, showing and confirming their efficiency and flexibility. The first proposal is devoted to parametric estimation problems. When EAs are employed in such analysis a novel form of variability related to their stochastic elements is introduced. We shall analyze both variability due to sampling, associated with selected estimator, and variability due to the EA. This analysis is set in a framework of statistical and computational tradeoff question, crucial in nowadays problems, by introducing cost functions related to both data acquisition and EA iterations. The proposed method will be illustrated by means of model building problem examples. Subsequent chapter is concerned with EAs employed in Markov Chain Monte Carlo (MCMC) sampling. When sampling from multimodal or highly correlated distribution is concerned, in fact, a possible strategy suggests to run several chains in parallel, in order to improve their mixing. If these chains are allowed to interact with each other then many analogies with EC techniques can be observed, and this has led to research in many fields. The chapter aims at reviewing various methods found in literature which conjugates EC techniques and MCMC sampling, in order to identify specific and common procedures, and unifying them in a framework of EC. In the last proposal we present a complex time series model and an identification procedure based on Genetic Algorithms (GAs). The model is capable of dealing with seasonality, by Periodic AutoRegressive (PAR) modelling, and structural changes in time, leading to a nonstationary structure. As far as a very large number of parameters and possibilites of change points are concerned, GAs are appropriate for identifying such model. Effectiveness of procedure is shown on both simulated data and real examples, these latter referred to river flow data in hydrology. The thesis concludes with some final remarks, concerning also future work