Time series data mining: preprocessing, analysis, segmentation and prediction. Applications

Currently, the amount of data which is produced for any information system is increasing exponentially. This motivates the development of automatic techniques to process and mine these data correctly. Specifically, in this Thesis, we tackled these problems for time series data, that is, temporal data which is collected chronologically. This kind of data can be found in many fields of science, such as palaeoclimatology, hydrology, financial problems, etc. TSDM consists of several tasks which try to achieve different objectives, such as, classification, segmentation, clustering, prediction, analysis, etc. However, in this Thesis, we focus on time series preprocessing, segmentation and prediction. Time series preprocessing is a prerequisite for other posterior tasks: for example, the reconstruction of missing values in incomplete parts of time series can be essential for clustering them. In this Thesis, we tackled the problem of massive missing data reconstruction in SWH time series from the Gulf of Alaska. It is very common that buoys stop working for different periods, what it is usually related to malfunctioning or bad weather conditions. The relation of the time series of each buoy is analysed and exploited to reconstruct the whole missing time series. In this context, EANNs with PUs are trained, showing that the resulting models are simple and able to recover these values with high precision. In the case of time series segmentation, the procedure consists in dividing the time series into different subsequences to achieve different purposes. This segmentation can be done trying to find useful patterns in the time series. In this Thesis, we have developed novel bioinspired algorithms in this context. For instance, for paleoclimate data, an initial genetic algorithm was proposed to discover early warning signals of TPs, whose detection was supported by expert opinions. However, given that the expert had to individually evaluate every solution given by the algorithm, the evaluation of the results was very tedious. This led to an improvement in the body of the GA to evaluate the procedure automatically. For significant wave height time series, the objective was the detection of groups which contains extreme waves, i.e. those which are relatively large with respect other waves close in time. The main motivation is to design alert systems. This was done using an HA, where an LS process was included by using a likelihood-based segmentation, assuming that the points follow a beta distribution. Finally, the analysis of similarities in different periods of European stock markets was also tackled with the aim of evaluating the influence of different markets in Europe. When segmenting time series with the aim of reducing the number of points, different techniques have been proposed. However, it is an open challenge given the difficulty to operate with large amounts of data in different applications. In this work, we propose a novel statistically-driven CRO algorithm (SCRO), which automatically adapts its parameters during the evolution, taking into account the statistical distribution of the population fitness. This algorithm improves the state-of-the-art with respect to accuracy and robustness. Also, this problem has been tackled using an improvement of the BBPSO algorithm, which includes a dynamical update of the cognitive and social components in the evolution, combined with mathematical tricks to obtain the fitness of the solutions, which significantly reduces the computational cost of previously proposed coral reef methods. Also, the optimisation of both objectives (clustering quality and approximation quality), which are in conflict, could be an interesting open challenge, which will be tackled in this Thesis. For that, an MOEA for time series segmentation is developed, improving the clustering quality of the solutions and their approximation. The prediction in time series is the estimation of future values by observing and studying the previous ones. In this context, we solve this task by applying prediction over high-order representations of the elements of the time series, i.e. the segments obtained by time series segmentation. This is applied to two challenging problems, i.e. the prediction of extreme wave height and fog prediction. On the one hand, the number of extreme values in SWH time series is less with respect to the number of standard values. In this way, the prediction of these values cannot be done using standard algorithms without taking into account the imbalanced ratio of the dataset. For that, an algorithm that automatically finds the set of segments and then applies EANNs is developed, showing the high ability of the algorithm to detect and predict these special events. On the other hand, fog prediction is affected by the same problem, that is, the number of fog events is much lower tan that of non-fog events, requiring a special treatment too. A preprocessing of different data coming from sensors situated in different parts of the Valladolid airport are used for making a simple ANN model, which is physically corroborated and discussed. The last challenge which opens new horizons is the estimation of the statistical distribution of time series to guide different methodologies. For this, the estimation of a mixed distribution for SWH time series is then used for fixing the threshold of POT approaches. Also, the determination of the fittest distribution for the time series is used for discretising it and making a prediction which treats the problem as ordinal classification. The work developed in this Thesis is supported by twelve papers in international journals, seven papers in international conferences, and four papers in national conferences

Particle Swarm Optimization

Particle swarm optimization (PSO) is a population based stochastic optimization technique influenced by the social behavior of bird flocking or fish schooling.PSO shares many similarities with evolutionary computation techniques such as Genetic Algorithms (GA). The system is initialized with a population of random solutions and searches for optima by updating generations. However, unlike GA, PSO has no evolution operators such as crossover and mutation. In PSO, the potential solutions, called particles, fly through the problem space by following the current optimum particles. This book represents the contributions of the top researchers in this field and will serve as a valuable tool for professionals in this interdisciplinary field

Effective image clustering based on human mental search

Image segmentation is one of the fundamental techniques in image analysis. One group of segmentation techniques is based on clustering principles, where association of image pixels is based on a similarity criterion. Conventional clustering algorithms, such as k-means, can be used for this purpose but have several drawbacks including dependence on initialisation conditions and a higher likelihood of converging to local rather than global optima. In this paper, we propose a clustering-based image segmentation method that is based on the human mental search (HMS) algorithm. HMS is a recent metaheuristic algorithm based on the manner of searching in the space of online auctions. In HMS, each candidate solution is called a bid, and the algorithm comprises three major stages: mental search, which explores the vicinity of a solution using Levy flight to find better solutions; grouping which places a set of candidate solutions into a group using a clustering algorithm; and moving bids toward promising solution areas. In our image clustering application, bids encode the cluster centres and we evaluate three different objective functions. In an extensive set of experiments, we compare the efficacy of our proposed approach with several state-of-the-art metaheuristic algorithms including a genetic algorithm, differential evolution, particle swarm optimisation, artificial bee colony algorithm, and harmony search. We assess the techniques based on a variety of metrics including the objective functions, a cluster validity index, as well as unsupervised and supervised image segmentation criteria. Moreover, we perform some tests in higher dimensions, and conduct a statistical analysis to compare our proposed method to its competitors. The obtained results clearly show that the proposed algorithm represents a highly effective approach to image clustering that outperforms other state-of-the-art techniques

Intelligent Leukaemia Diagnosis with Bare-Bones PSO based Feature Optimization

In this research, we propose an intelligent decision support system for acute lymphoblastic leukaemia (ALL) diagnosis using microscopic images. Two Bare-bones Particle Swarm Optimization (BBPSO) algorithms are proposed to identify the most significant discriminative characteristics of healthy and blast cells to enable efficient ALL classification. The first BBPSO variant incorporates accelerated chaotic search mechanisms of food chasing and enemy avoidance to diversify the search and mitigate the premature convergence of the original BBPSO algorithm. The second BBPSO variant exhibits both of the abovementioned new search mechanisms in a subswarm-based search. Evaluated with the ALL-IDB2 database, both proposed algorithms achieve superior geometric mean performances of 94.94% and 96.25%, respectively, and outperform other metaheuristic search and related methods significantly for ALL classification

Novel particle swarm optimization algorithms with applications in power systems

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonOptimization problems are vital in physical sciences, commercial and finance matters. In a nutshell, almost everyone is the stake-holder in certain optimization problems aiming at minimizing the cost of production and losses of system, and also maximizing the profit. In control systems, the optimal configuration problems are essential that have been solved by various newly developed methods. The literature is exhaustively explored for an appropriate optimization method to solve such kind of problems. Particle Swarm Optimization is found to be one of the best among several optimization methods by analysing the experimental results. Two novel PSO variants are introduced in this thesis. The first one is named as N State Markov Jumping Particle Swarm Optimization, which is based on the stochastic technique and Markov chain in updating the particle velocity. We have named the second variant as N State Switching Particle Swarm Optimization, which is based on the evolutionary factor information for updating the velocity. The proposed algorithms are then applied to some widely used mathematical benchmark functions. The statistical results of 30 independent trails illustrate the robustness and accuracy of the proposed algorithms for most of the benchmark functions. The better results in terms of mean minimum evaluation errors and the shortest computation time are illustrated. In order to verify the satisfactory performance and robustness of the proposed algorithms, we have further formulated some basic applications in power system operations. The first application is about the static Economic Load Dispatch and the second application is on the Dynamic Economic Load Dispatch. These are highly complex and non-linear problems of power system operations consisting of various systems and generator constraints. Basically, in the static Economic Load Dispatch, a single load is considered for calculating the cost function. In contrast, the Dynamic Economic Load Dispatch changes the load demand for the cost function dynamically with time. In such a challenging and complex environment the proposed algorithms can be applied. The empirical results obtained by applying both of the proposed methods have substantiated their adaptability and robustness into the real-world environment. It is shown in the numerical results that the proposed algorithms are robust and accurate as compared to the other algorithms. The proposed algorithms have produced consistent best values for their objectives, where satisfying all constraints with zero penalty

Development of emergency response systems by intelligent and integrated approaches for marine oil spill accidents

Oil products play a pervasive role in modern society as one of the dominant energy fuel sources. Marine activities related to oil extraction and transportation play a vital role in resource supply. However, marine oil spills occur due to such human activities or harsh environmental factors. The emergency accidents of spills cause negative impacts on the marine environment, human health, and economic loss. The responses to marine oil spills, especially large-scale spills, are relatively challenging and inefficient due to changing environmental conditions, limited response resources, various unknown or uncertain factors and complex resource allocation processes. The development of previous research mainly focused on single process simulation, prediction, or optimization (e.g., oil trajectory, weathering, or cleanup optimization). There is still a lack of research on comprehensive and integrated emergency responses considering multiple types of simulations, types of resource allocations, stages of accident occurrence to response, and criteria for system optimizations. Optimization algorithms are an important part of system optimization and decision-making. Their performance directly affacts the quality of emergency response systems and operations. Thus, how to improve efficiency of emergency response systems becomes urgent and essential for marine oil spill management. The power and potential of integrating intelligent-based modeling of dynamic processes and system optimization have been recognized to better support oil spill responders with more efficient response decisions and planning tools. Meanwhile, response decision-making combined with human factor analysis can help quantitatively evaluate the impacts of multiple causal factors on the overall processes and operational performance after an accident. To address the challenges and gaps, this dissertation research focused on the development and improvement of new emergency response systems and their applications for marine oil spill response in the following aspects: 1) Realization of coupling dynamic simulation and system optimization for marine oil spill responses - The developed Simulation-Based Multi-Agent Particle Swarm Optimization (SA-PSO) modeling investigated the capacity of agent-based modeling on dynamic simulation of spill fate and response, particle swarm optimization on response allocation with minimal time and multi-agent system on information sharing. 2) Investigation of multi-type resource allocation under a complex simulation condition and improvement of optimization performance - The improved emergency response system was achieved by dynamic resource transportation, oil weathering and response simulations and resource allocation optimization. The enhanced particle swarm optimization (ME-PSO) algorithm performed outstanding convergence performance and low computation cost characteristics integrating multi-agent theory (MA) and evolutionary population dynamics (EPD). 3) Analysis and evaluation of influencing factors of multiple stages of spill accidents based on human factors/errors and multi-criteria decision making - The developed human factors analysis and classification system for marine oil spill accidents (HFACS-OS) framework qualitatively evaluated the influence of various factors and errors associated with the multiple operational stages considered for oil spill preparedness and response (e.g., oil spill occurrence, spill monitoring, decision making/contingency planning, and spill response). The framework was further coupled with quantitative data analysis by Fuzzy-based Technique for Order Preference by Similarity to Idea Solution (Fuzzy-TOPSIS) to enhance decision-making during response operations under multiple criteria. 4) Development of a multi-criteria emergency response system with the enhanced optimization algorithm, multi-mode resource transportation and allocation and a more complex and realistic simulation modelling - The developed multi-criteria emergency response system (MC-ERS) system integrated dynamic process simulations and weighted multi-criteria system optimization. Total response time, response cost and environmental impacts were regarded as multiple optimization goals. An improved weighted sum optimization function was developed to unify the scaling and proportion of different goals. A comparative PSO was also developed with various algorithm-improving methods and the best-performing inertia weight function. The proposed emergency response approaches in studies were examined by oil spill case studies related to the North Atlantic Ocean and Canada circumstances to analyze the modelling performance and evaluate their practicality and applicability. The developed optimization algorithms were tested by benchmarked functions, other optimization algorithms, and an oil spill case. The developed emergency response systems and the contained simulation and optimization algorithms showed the strong capability for decision-making and emergency responses by recommending optimal resource management or evaluations of essential factors. This research was expected to provide time-efficient, and cost-saving emergency response management approaches for handling and managing marine oil spills. The research also improved our knowledge of the significance of human factors/errors to oil spill accidents and response operations and provided improved support tools for decision making. The dissertation research helped fill some important gaps in emergency response research and management practice, especially in marine oil spill response, through an innovative integration of dynamic simulation, resource optimization, human factor analysis, and artificial intelligence methods. The research outcomes can also provide methodological support and valuable references for other fields that require timely and effective decisions, system optimizations, process controls, planning and designs under complicated conditions, uncertainties, and interactions

Applied Metaheuristic Computing

For decades, Applied Metaheuristic Computing (AMC) has been a prevailing optimization technique for tackling perplexing engineering and business problems, such as scheduling, routing, ordering, bin packing, assignment, facility layout planning, among others. This is partly because the classic exact methods are constrained with prior assumptions, and partly due to the heuristics being problem-dependent and lacking generalization. AMC, on the contrary, guides the course of low-level heuristics to search beyond the local optimality, which impairs the capability of traditional computation methods. This topic series has collected quality papers proposing cutting-edge methodology and innovative applications which drive the advances of AMC

Applied Methuerstic computing

For decades, Applied Metaheuristic Computing (AMC) has been a prevailing optimization technique for tackling perplexing engineering and business problems, such as scheduling, routing, ordering, bin packing, assignment, facility layout planning, among others. This is partly because the classic exact methods are constrained with prior assumptions, and partly due to the heuristics being problem-dependent and lacking generalization. AMC, on the contrary, guides the course of low-level heuristics to search beyond the local optimality, which impairs the capability of traditional computation methods. This topic series has collected quality papers proposing cutting-edge methodology and innovative applications which drive the advances of AMC

Development of registration methods for cardiovascular anatomy and function using advanced 3T MRI, 320-slice CT and PET imaging

Different medical imaging modalities provide complementary anatomical and functional information. One increasingly important use of such information is in the clinical management of cardiovascular disease. Multi-modality data is helping improve diagnosis accuracy, and individualize treatment. The Clinical Research Imaging Centre at the University of Edinburgh, has been involved in a number of cardiovascular clinical trials using longitudinal computed tomography (CT) and multi-parametric magnetic resonance (MR) imaging. The critical image processing technique that combines the information from all these different datasets is known as image registration, which is the topic of this thesis. Image registration, especially multi-modality and multi-parametric registration, remains a challenging field in medical image analysis. The new registration methods described in this work were all developed in response to genuine challenges in on-going clinical studies. These methods have been evaluated using data from these studies. In order to gain an insight into the building blocks of image registration methods, the thesis begins with a comprehensive literature review of state-of-the-art algorithms. This is followed by a description of the first registration method I developed to help track inflammation in aortic abdominal aneurysms. It registers multi-modality and multi-parametric images, with new contrast agents. The registration framework uses a semi-automatically generated region of interest around the aorta. The aorta is aligned based on a combination of the centres of the regions of interest and intensity matching. The method achieved sub-voxel accuracy. The second clinical study involved cardiac data. The first framework failed to register many of these datasets, because the cardiac data suffers from a common artefact of magnetic resonance images, namely intensity inhomogeneity. Thus I developed a new preprocessing technique that is able to correct the artefacts in the functional data using data from the anatomical scans. The registration framework, with this preprocessing step and new particle swarm optimizer, achieved significantly improved registration results on the cardiac data, and was validated quantitatively using neuro images from a clinical study of neonates. Although on average the new framework achieved accurate results, when processing data corrupted by severe artefacts and noise, premature convergence of the optimizer is still a common problem. To overcome this, I invented a new optimization method, that achieves more robust convergence by encoding prior knowledge of registration. The registration results from this new registration-oriented optimizer are more accurate than other general-purpose particle swarm optimization methods commonly applied to registration problems. In summary, this thesis describes a series of novel developments to an image registration framework, aimed to improve accuracy, robustness and speed. The resulting registration framework was applied to, and validated by, different types of images taken from several ongoing clinical trials. In the future, this framework could be extended to include more diverse transformation models, aided by new machine learning techniques. It may also be applied to the registration of other types and modalities of imaging data