Treed Gaussian Process Regression for Solving Offline Data-Driven Continuous Multiobjective Optimization Problems

This is the final version. Available from MIT Press via the DOI in this recordFor offline data-driven multiobjective optimization problems (MOPs), no new data is available during the optimization process. Approximation models (or surrogates) are first built using the provided offline data and an optimizer, e.g. a multiobjective evolutionary algorithm, can then be utilized to find Pareto optimal solutions to the problem with surrogates as objective functions. In contrast to online data-driven MOPs, these surrogates cannot be updated with new data and, hence, the approximation accuracy cannot be improved by considering new data during the optimization process. Gaussian process regression (GPR) models are widely used as surrogates because of their ability to provide uncertainty information. However, building GPRs becomes computationally expensive when the size of the dataset is large. Using sparse GPRs reduces the computational cost of building the surrogates. However, sparse GPRs are not tailored to solve offline data-driven MOPs, where good accuracy of the surrogates is needed near Pareto optimal solutions. Treed GPR (TGPR-MO) surrogates for offline data-driven MOPs with continuous decision variables are proposed in this paper. The proposed surrogates first split the decision space into subregions using regression trees and build GPRs sequentially in regions close to Pareto optimal solutions in the decision space to accurately approximate tradeoffs between the objective functions. TGPR-MO surrogates are computationally inexpensive because GPRs are built only in a smaller region of the decision space utilizing a subset of the data. The TGPR-MO surrogates were tested on distance-based visualizable problems with various data sizes, sampling strategies, numbers of objective functions, and decision variables. Experimental results showed that the TGPR-MO surrogates are computationally cheaper and can handle datasets of large size. Furthermore, TGPR-MO surrogates produced solutions closer to Pareto optimal solutions compared to full GPRs and sparse GPRs.Academy of Finlan

Tools for Landscape Analysis of Optimisation Problems in Procedural Content Generation for Games

The term Procedural Content Generation (PCG) refers to the (semi-)automatic generation of game content by algorithmic means, and its methods are becoming increasingly popular in game-oriented research and industry. A special class of these methods, which is commonly known as search-based PCG, treats the given task as an optimisation problem. Such problems are predominantly tackled by evolutionary algorithms. We will demonstrate in this paper that obtaining more information about the defined optimisation problem can substantially improve our understanding of how to approach the generation of content. To do so, we present and discuss three efficient analysis tools, namely diagonal walks, the estimation of high-level properties, as well as problem similarity measures. We discuss the purpose of each of the considered methods in the context of PCG and provide guidelines for the interpretation of the results received. This way we aim to provide methods for the comparison of PCG approaches and eventually, increase the quality and practicality of generated content in industry.Comment: 30 pages, 8 figures, accepted for publication in Applied Soft Computin

Visualising the Landscape of Multi-Objective Problems using Local Optima Networks

This is the author accepted manuscript. The final version is available from ACM via the DOI in this recordThe codebase for this paper is available at https://github.com/fieldsend/mo_lonsLocal optima networks (LONs) represent the landscape of optimisation problems. In a LON, graph vertices represent local optima in the search domain, their radii the basin sizes, and directed edges between vertices the ability to transit from one basin to another (with the edge width denoting how easy this is). Recently, a network construction approach inspired by LONs has been proposed for multi-objective problems which uses an undirected graph, representing mutually non-dominating solutions and neighbouring links, but not basin sizes. In contrast, here we introduce two formulations for multi/many-objective problems which are analogous to the traditional LON, using dominance-based hill-climbing to characterise the search domain. Each vertex represents a set of locally optimal solutions, with basins and ease of transition between them shown. These LONs vary depending on whether a point-based (dominance neutral optima) or set-based (Pareto local optima) representation is used to define mode construction. We illustrate these alternative formulations on some illustrative problems.We discuss some of the underlying computational issues in constructing LONs in a multiobjective as opposed to uni-objective problem domain, along with the inherent issue of neutrality — as each a vertex in these graphs almost invariably represents a set in our proposed constructs.Engineering and Physical Sciences Research Council (EPSRC

Probabilistic Selection Approaches in Decomposition-based Evolutionary Algorithms for Offline Data-Driven Multiobjective Optimization

This is the author accepted manuscript. The final version is available from IEEE via the DOI in this recordIn offline data-driven multiobjective optimization, no new data is available during the optimization process. Approximation models, also known as surrogates, are built using the provided offline data. A multiobjective evolutionary algorithm can be utilized to find solutions by using these surrogates. The accuracy of the approximated solutions depends on the surrogates and approximations typically involve uncertainties. In this paper, we propose probabilistic selection approaches that utilize the uncertainty information of the Kriging models (as surrogates) to improve the solution process in offline data-driven multiobjective optimization. These approaches are designed for decomposition-based multiobjective evolutionary algorithms and can, thus, handle a large number of objectives. The proposed approaches were tested on distance-based visualizable test problems and the DTLZ suite. The proposed approaches produced solutions with a greater hypervolume, and a lower root mean squared error compared to generic approaches and a transfer learning approach that do not use uncertainty information

Strain Elevation Tension Spring embedding and Cascading failures on the power-grid

Understanding the dynamics and properties of networks is of great importance in our highly connected data-driven society. When the networks relate to infrastructure, such understanding can have a substantial impact on public welfare. As such, there is a need for algorithms that can provide insights into the observable and latent properties of these structures. This thesis presents a novel embedding algorithm: the Strain Elevation Tension Spring embedding (SETSe), as a method of understanding complex networks. The algorithm is a deterministic physics model that incorporates both node and edge features into the final embedding. SETSe distinguishes itself from most embeddings methods by not having a loss function in the conventional sense and by not trying to place similar nodes close together. Instead, SETSe acts as a smoothing function for node features across the network topology. This approach produces embeddings that are intuitive and interpretable. In this thesis, I demonstrate how SETSe outperforms alternative embedding methods on node level and graph level tasks using networks made from stochastic block models and social networks with over 40,000 nodes and over 1 million edges. I also highlight a weakness of traditional methods to analysing cascading failures on power grids and demonstrate that SETSe is not susceptible to such issues. I then show how SETSe can be used as a measure of robustness in addition to providing a means to create interpretable maps in the geographical space given its smoothing embedding method. The framework has been made widely available through two open source R packages contributions, 1) the implementation of SETSe ("rsetse" on CRAN), and 2) a package for analysing cascading failures on power grids

Generative neural data synthesis for autonomous systems

A significant number of Machine Learning methods for automation currently rely on data-hungry training techniques. The lack of accessible training data often represents an insurmountable obstacle, especially in the fields of robotics and automation, where acquiring new data can be far from trivial. Additional data acquisition is not only often expensive and time-consuming, but occasionally is not even an option. Furthermore, the real world applications sometimes have commercial sensitivity issues associated with the distribution of the raw data. This doctoral thesis explores bypassing the aforementioned difficulties by synthesising new realistic and diverse datasets using the Generative Adversarial Network (GAN). The success of this approach is demonstrated empirically through solving a variety of case-specific data-hungry problems, via application of novel GAN-based techniques and architectures. Specifically, it starts with exploring the use of GANs for the realistic simulation of the extremely high-dimensional underwater acoustic imagery for the purpose of training both teleoperators and autonomous target recognition systems. We have developed a method capable of generating realistic sonar data of any chosen dimension by image-translation GANs with Markov principle. Following this, we apply GAN-based models to robot behavioural repertoire generation, that enables a robot manipulator to successfully overcome unforeseen impedances, such as unknown sets of obstacles and random broken joints scenarios. Finally, we consider dynamical system identification for articulated robot arms. We show how using diversity-driven GAN models to generate exploratory trajectories can allow dynamic parameters to be identified more efficiently and accurately than with conventional optimisation approaches. Together, these results show that GANs have the potential to benefit a variety of robotics learning problems where training data is currently a bottleneck

A Feature Rich Distance-Based Many-Objective Visualisable Test Problem Generator

This is the author accepted manuscript. The final version is available from ACM via the DOI in this recordThe codebase for this paper is available at https://github.com/fieldsend/DBMOPP_generatorIn optimiser analysis and design it is informative to visualise how a search point/population moves through the design space over time. Visualisable distance-based many-objective optimisation problems have been developed whose design space is in two-dimensions with arbitrarily many objective dimensions. Previous work has shown how disconnected Pareto sets may be formed, how problems can be projected to and from arbitrarily many design dimensions, and how dominance resistant regions of design space may be defined. Most recently, a test suite has been proposed using distances to lines rather than points. However, active use of visualisable problems has been limited. This may be because the type of problem characteristics available has been relatively limited compared to many practical problems (and non-visualisable problem suites). Here we introduce the mechanisms required to embed several widely seen problem characteristics in the existing problem framework. These include variable density of solutions in objective space, landscape discontinuities, varying objective ranges, neutrality, and non-identical disconnected Pareto set regions. Furthermore, we provide an automatic problem generator (as opposed to hand-tuned problem definitions). The flexibility of the problem generator is demonstrated by analysing the performance of popular optimisers on a range of sampled instances.Engineering and Physical Sciences Research Council (EPSRC)Natural Environment Research Council (NERC

Colour constancy in simple and complex scenes

PhD ThesisColour constancy is defined as the ability to perceive the surface colours of objects within scenes as approximately constant through changes in scene illumination. Colour constancy in real life functions so seamlessly that most people do not realise that the colour of the light emanating from an object can change markedly throughout the day. Constancy measurements made in simple scenes constructed from flat coloured patches do not produce constancy of this high degree. The question that must be asked is: what are the features of everyday scenes that improve constancy? A novel technique is presented for testing colour constancy. Results are presented showing measurements of constancy in simple and complex scenes. More specifically, matching experiments are performed for patches against uniform and multi-patch backgrounds, the latter of which provide colour contrast. Objects created by the addition of shape and 3-D shading information are also matched against backgrounds consisting of matte reflecting patches. In the final set of experiments observers match detailed depictions of objects - rich in chromatic contrast, shading, mutual illumination and other real life features - within depictions of real life scenes. The results show similar performance across the conditions that contain chromatic contrast, although some uncertainty still remains as to whether the results are indicative of human colour constancy performance or to sensory match capabilities. An interesting division exists between patch matches performed against uniform and multi-patch backgrounds that is manifested as a shift in CIE xy space. A simple model of early chromatic processes is proposed and examined in the context of the results

Higher dimensional theories in physics, following the Kaluza model of unification

This thesis traces the origins and evolution of higher dimensional models in physics, with particular reference to the five-dimensional Kaluza-Klein unification. It includes the motivation needed, and the increasing status and significance of the multidimensional description of reality for the 1990's. The differing conceptualisations are analysed, from the mathematical, via Kasner's embedding dimensions and Schrodinger's waves, to the high status of Kaluza-Klein dimensions in physics today. This includes the use of models, and the metaphysical interpretations needed to translate the mathematics. The main area of original research is the unpublished manuscripts and letters of Theodor Kaiuza, some Einstein letters, further memoirs from his son Theodor Kaiuza Junior and from some of his original students. Unpublished material from Helsinki concerns the Finnish physicist Nordstrom, the real originator of the idea that 'forces' in 4-dimensional spacetime might arise from gravity in higher dimensions. The work of the Swedish physicist Oskar Klein and the reactions of de Broglie and Einstein initiated the Kaluza-Klein connection which is traced through fifty years of neglect to its re-entry into mainstream physics. The cosmological significance and conceptualisation through analogue models is charted by personal correspondence with key scientists across a range of theoretical physics, involving the use of aesthetic criteria where there is no direct physical verification. Qualitative models implicitly indicating multidimensions are identified in the paradoxes and enigmas of existing physics, in Quantum Mechanics and the singularities in General Relativity. The Kaluza-Klein philosophy brings this wide range of models together in the late 1980's via supergravity, superstrings and supermanifolds. This new multidimensional paradigm wave is seen to produce a coherent and consistent metaphysics, a new perspective on reality. It may also have immense potential significance for philosophy and theology. The thesis concludes with the reality question, "Are we a four-dimensional projection of a deeper reality of many, even infinite, dimensions?