Evolutionary Dynamic Optimization and Machine Learning

Evolutionary Computation (EC) has emerged as a powerful field of Artificial Intelligence, inspired by nature's mechanisms of gradual development. However, EC approaches often face challenges such as stagnation, diversity loss, computational complexity, population initialization, and premature convergence. To overcome these limitations, researchers have integrated learning algorithms with evolutionary techniques. This integration harnesses the valuable data generated by EC algorithms during iterative searches, providing insights into the search space and population dynamics. Similarly, the relationship between evolutionary algorithms and Machine Learning (ML) is reciprocal, as EC methods offer exceptional opportunities for optimizing complex ML tasks characterized by noisy, inaccurate, and dynamic objective functions. These hybrid techniques, known as Evolutionary Machine Learning (EML), have been applied at various stages of the ML process. EC techniques play a vital role in tasks such as data balancing, feature selection, and model training optimization. Moreover, ML tasks often require dynamic optimization, for which Evolutionary Dynamic Optimization (EDO) is valuable. This paper presents the first comprehensive exploration of reciprocal integration between EDO and ML. The study aims to stimulate interest in the evolutionary learning community and inspire innovative contributions in this domain

Repurposing existing skeletal spatial structure (SkS) system designs using the Field Information Modeling (FIM) framework for generative decision-support in future construction projects

Skeletal spatial structure (SkS) systems are modular systems which have shown promise to support mass customization, and sustainability in construction. SkS have been used extensively in the reconstruction efforts since World War II, particularly to build geometrically flexible and free-form structures. By employing advanced digital engineering and construction practices, the existing SkS designs may be repurposed to generate new optimal designs that satisfy current construction demands of contemporary societies. To this end, this study investigated the application of point cloud processing using the Field Information Modeling (FIM) framework for the digital documentation and generative redesign of existing SkS systems. Three new algorithms were proposed to (i) expand FIM to include generative decision-support; (ii) generate as-built building information modeling (BIM) for SkS; and (iii) modularize SkS designs with repeating patterns for optimal production and supply chain management. These algorithms incorporated a host of new AI-inspired methods, including support vector machine (SVM) for decision support; Bayesian optimization for neighborhood definition; Bayesian Gaussian mixture clustering for modularization; and Monte Carlo stochastic multi-criteria decision making (MCDM) for selection of the top Pareto front solutions obtained by the non-dominant sorting Genetic Algorithm (NSGA II). The algorithms were tested and validated on four real-world point cloud datasets to solve two generative modeling problems, namely, engineering design optimization and facility location optimization. It was observed that the proposed Bayesian neighborhood definition outperformed particle swarm and uniform sampling by 34% and 27%, respectively. The proposed SVM-based linear feature detection outperformed k-means and spectral clustering by 56% and 9%, respectively. Finally, the NSGA II algorithm combined with the stochastic MCDM produced diverse “top four” solutions based on project-specific criteria. The results indicate promise for future utilization of the framework to produce training datasets for generative adversarial networks that generate new designs based only on stakeholder requirements

Innovative hybrid MOEA/AD variants for solving multi-objective combinatorial optimization problems

Orientador : Aurora Trinidad Ramirez PozoCoorientador : Roberto SantanaTese (doutorado) - Universidade Federal do Paraná, Setor de Ciências Exatas, Programa de Pós-Graduação em Informática. Defesa: Curitiba, 16/12/2016Inclui referências : f. 103-116Resumo: Muitos problemas do mundo real podem ser representados como um problema de otimização combinatória. Muitas vezes, estes problemas são caracterizados pelo grande número de variáveis e pela presença de múltiplos objetivos a serem otimizados ao mesmo tempo. Muitas vezes estes problemas são difíceis de serem resolvidos de forma ótima. Suas resoluções tem sido considerada um desafio nas últimas décadas. Os algoritimos metaheurísticos visam encontrar uma aproximação aceitável do ótimo em um tempo computacional razoável. Os algoritmos metaheurísticos continuam sendo um foco de pesquisa científica, recebendo uma atenção crescente pela comunidade. Uma das têndencias neste cenário é a arbordagem híbrida, na qual diferentes métodos e conceitos são combinados objetivando propor metaheurísticas mais eficientes. Nesta tese, nós propomos algoritmos metaheurísticos híbridos para a solução de problemas combinatoriais multiobjetivo. Os principais ingredientes das nossas propostas são: (i) o algoritmo evolutivo multiobjetivo baseado em decomposição (MOEA/D framework), (ii) a otimização por colônias de formigas e (iii) e os algoritmos de estimação de distribuição. Em nossos frameworks, além dos operadores genéticos tradicionais, podemos instanciar diferentes modelos como mecanismo de reprodução dos algoritmos. Além disso, nós introduzimos alguns componentes nos frameworks objetivando balancear a convergência e a diversidade durante a busca. Nossos esforços foram direcionados para a resolução de problemas considerados difíceis na literatura. São eles: a programação quadrática binária sem restrições multiobjetivo, o problema de programação flow-shop permutacional multiobjetivo, e também os problemas caracterizados como deceptivos. Por meio de estudos experimentais, mostramos que as abordagens propostas são capazes de superar os resultados do estado-da-arte em grande parte dos casos considerados. Mostramos que as diretrizes do MOEA/D hibridizadas com outras metaheurísticas é uma estratégia promissora para a solução de problemas combinatoriais multiobjetivo. Palavras-chave: metaheuristicas, otimização multiobjetivo, problemas combinatoriais, MOEA/D, otimização por colônia de formigas, algoritmos de estimação de distribuição, programação quadrática binária sem restrições multiobjetivo, problema de programação flow-shop permutacional multiobjetivo, abordagens híbridas.Abstract: Several real-world problems can be stated as a combinatorial optimization problem. Very often, they are characterized by the large number of variables and the presence of multiple conflicting objectives to be optimized at the same time. These kind of problems are, usually, hard to be solved optimally, and their solutions have been considered a challenge for a long time. Metaheuristic algorithms aim at finding an acceptable approximation to the optimal solution in a reasonable computational time. The research on metaheuristics remains an attractive area and receives growing attention. One of the trends in this scenario are the hybrid approaches, in which different methods and concepts are combined aiming to propose more efficient approaches. In this thesis, we have proposed hybrid metaheuristic algorithms for solving multi-objective combinatorial optimization problems. Our proposals are based on (i) the multi-objective evolutionary algorithm based on decomposition (MOEA/D framework), (ii) the bio-inspired metaheuristic ant colony optimization, and (iii) the probabilistic models from the estimation of distribution algorithms. Our algorithms are considered MOEA/D variants. In our MOEA/D variants, besides the traditional genetic operators, we can instantiate different models as the variation step (reproduction). Moreover, we include some design modifications into the frameworks to control the convergence and the diversity during their search (evolution). We have addressed some important problems from the literature, e.g., the multi-objective unconstrained binary quadratic programming, the multiobjective permutation flowshop scheduling problem, and the problems characterized by deception. As a result, we show that our proposed frameworks are able to solve these problems efficiently by outperforming the state-of-the-art approaches in most of the cases considered. We show that the MOEA/D guidelines hybridized to other metaheuristic components and concepts is a powerful strategy for solving multi-objective combinatorial optimization problems. Keywords: meta-heuristics, multi-objective optimization, combinatorial problems, MOEA/D, ant colony optimization, estimation of distribution algorithms, unconstrained binary quadratic programming, permutation flowshop scheduling problem, hybrid approaches

Preference-based evolutionary algorithm for airport surface operations

In addition to time efficiency, minimisation of fuel consumption and related emissions has started to be considered by research on optimisation of airport surface operations as more airports face severe congestion and tightening environmental regulations. Objectives are related to economic cost which can be used as preferences to search for a region of cost efficient and Pareto optimal solutions. A multi-objective evolutionary optimisation framework with preferences is proposed in this paper to solve a complex optimisation problem integrating runway scheduling and airport ground movement problem. The evolutionary search algorithm uses modified crowding distance in the replacement procedure to take into account cost of delay and fuel price. Furthermore, uncertainty inherent in prices is reflected by expressing preferences as an interval. Preference information is used to control the extent of region of interest, which has a beneficial effect on algorithm performance. As a result, the search algorithm can achieve faster convergence and potentially better solutions. A filtering procedure is further proposed to select an evenly distributed subset of Pareto optimal solutions in order to reduce its size and help the decision maker. The computational results with data from major international hub airports show the efficiency of the proposed approach

Learning-based generative representations for automotive design optimization

In automotive design optimizations, engineers intuitively look for suitable representations of CAE models that can be used across different optimization problems. Determining a suitable compact representation of 3D CAE models facilitates faster search and optimization of 3D designs. Therefore, to support novice designers in the automotive design process, we envision a cooperative design system (CDS) which learns the experience embedded in past optimization data and is able to provide assistance to the designer while performing an engineering design optimization task. The research in this thesis addresses different aspects that can be combined to form a CDS framework. First, based on the survey of deep learning techniques, a point cloud variational autoencoder (PC-VAE) is adapted from the literature, extended and evaluated as a shape generative model in design optimizations. The performance of the PC-VAE is verified with respect to state-of-the-art architectures. The PC-VAE is capable of generating a continuous low-dimensional search space for 3D designs, which further supports the generation of novel realistic 3D designs through interpolation and sampling in the latent space. In general, while designing a 3D car design, engineers need to consider multiple structural or functional performance criteria of a 3D design. Hence, in the second step, the latent representations of the PC-VAE are evaluated for generating novel designs satisfying multiple criteria and user preferences. A seeding method is proposed to provide a warm start to the optimization process and improve convergence time. Further, to replace expensive simulations for performance estimation in an optimization task, surrogate models are trained to map each latent representation of an input 3D design to their respective geometric and functional performance measures. However, the performance of the PC-VAE is less consistent due to additional regularization of the latent space. Thirdly, to better understand which distinct region of the input 3D design is learned by a particular latent variable of the PC-VAE, a new deep generative model is proposed (Split-AE), which is an extension of the existing autoencoder architecture. The Split-AE learns input 3D point cloud representations and generates two sets of latent variables for each 3D design. The first set of latent variables, referred to as content, which helps to represent an overall underlying structure of the 3D shape to discriminate across other semantic shape categories. The second set of latent variables refers to the style, which represents the unique shape part of the input 3D shape and this allows grouping of shapes into shape classes. The reconstruction and latent variables disentanglement properties of the Split-AE are compared with other state-of-the-art architectures. In a series of experiments, it is shown that for given input shapes, the Split-AE is capable of generating the content and style variables which gives the flexibility to transfer and combine style features between different shapes. Thus, the Split-AE is able to disentangle features with minimum supervision and helps in generating novel shapes that are modified versions of the existing designs. Lastly, to demonstrate the application of our initial envisioned CDS, two interactive systems were developed to assist designers in exploring design ideas. In the first CDS framework, the latent variables of the PC-VAE are integrated with a graphical user interface. This framework enables the designer to explore designs taking into account the data-driven knowledge and different performance measures of 3D designs. The second interactive system aims to guide the designers to achieve their design targets, for which past human experiences of performing 3D design modifications are captured and learned using a machine learning model. The trained model is then used to guide the (novice) engineers and designers by predicting the next step of design modification based on the current applied changes

On partitioning multivariate self-affine time series

Given a multivariate time series, possibly of high dimension, with unknown and time-varying joint distribution, it is of interest to be able to completely partition the time series into disjoint, contiguous subseries, each of which has different distributional or pattern attributes from the preceding and succeeding subseries. An additional feature of many time series is that they display self-affinity, so that subseries at one time scale are similar to subseries at another after application of an affine transformation. Such qualities are observed in time series from many disciplines, including biology, medicine, economics, finance, and computer science. This paper defines the relevant multiobjective combinatorial optimization problem with limited assumptions as a biobjective one, and a specialized evolutionary algorithm is presented which finds optimal self-affine time series partitionings with a minimum of choice parameters. The algorithm not only finds partitionings for all possible numbers of partitions given data constraints, but also for self-affinities between these partitionings and some fine-grained partitioning. The resulting set of Pareto-efficient solution sets provides a rich representation of the self-affine properties of a multivariate time series at different locations and time scales

Preference-based evolutionary algorithm for airport surface operations

In addition to time efficiency, minimisation of fuel consumption and related emissions has started to be considered by research on optimisation of airport surface operations as more airports face severe congestion and tightening environmental regulations. Objectives are related to economic cost which can be used as preferences to search for a region of cost efficient and Pareto optimal solutions. A multi-objective evolutionary optimisation framework with preferences is proposed in this paper to solve a complex optimisation problem integrating runway scheduling and airport ground movement problem. The evolutionary search algorithm uses modified crowding distance in the replacement procedure to take into account cost of delay and fuel price. Furthermore, uncertainty inherent in prices is reflected by expressing preferences as an interval. Preference information is used to control the extent of region of interest, which has a beneficial effect on algorithm performance. As a result, the search algorithm can achieve faster convergence and potentially better solutions. A filtering procedure is further proposed to select an evenly distributed subset of Pareto optimal solutions in order to reduce its size and help the decision maker. The computational results with data from major international hub airports show the efficiency of the proposed approach.This work is supported in part by the Engineering and Physical Sciences Research Council (EPSRC) under Grant EP/H004424/1, EP/N029496/1 and EP/N029496/2

Modeling and optimization of environment in agricultural greenhouses for improving cleaner and sustainable crop production

Resource-use efficiency and crop yield are significant factors in the management of agricultural greenhouse. Appropriate modeling methods effectively improve the control performance and efficiency of the greenhouse system and are conducive to the design of water and energy-saving strategies. Meanwhile, the extreme environment could be forecasted in advance, which reduces pests and diseases as well as provides high-quality food. Accordingly, the interest of the scientific community in greenhouse modeling and optimizing has grown considerably. The objective of this work is to provide guidance and insight into the topic by reviewing 73 representative articles and to further support cleaner and sustainable crop production. Compared to the existing literature review, this work details the approaches to improve the greenhouse model in the aspects of parameter identification, structure and process optimization, and multi-model integration to better model complex greenhouse system. Furthermore, a statistical study has been carried out to summarize popular technology and future trends. It was found that dynamic and neural network techniques are most commonly used to establish the greenhouse model and the heuristic algorithm is popular to improve the accuracy and generalization ability of the model. Notably, deep learning, the combination of “knowledge” and “data”, and coupling between the greenhouse system elements have been considered as future valuable development

Efficient Learning Machines

Computer scienc