A novel preference articulation operator for the Evolutionary Multi-Objective Optimisation of classifiers in concealed weapons detection

Abstract The incorporation of decision maker preferences is often neglected in the Evolutionary Multi-Objective Optimisation (EMO) literature. The majority of the research in the field and the development of EMO algorithms is primarily focussed on converging to a Pareto optimal approximation close to or along the true Pareto front of synthetic test problems. However, when EMO is applied to real-world optimisation problems there is often a decision maker who is only interested in a portion of the Pareto front (the Region of Interest) which is defined by their expressed preferences for the problem objectives. In this paper a novel preference articulation operator for EMO algorithms is introduced (named the Weighted Z-score Preference Articulation Operator) with the flexibility of being incorporated a priori, a posteriori or progressively, and as either a primary or auxiliary fitness operator. The Weighted Z-score Preference Articulation Operator is incorporated into an implementation of the Multi-Objective Evolutionary Algorithm Based on Decomposition (named WZ-MOEA/D) and benchmarked against MOEA/D-DRA on a number of bi-objective and five-objective test problems with test cases containing preference information. After promising results are obtained when comparing WZ-MOEA/D to MOEA/D-DRA in the presence of decision maker preferences, WZ-MOEA/D is successfully applied to a real-world optimisation problem to optimise a classifier for concealed weapon detection, producing better results than previously published classifier implementations

A Multi objective Approach to Evolving Artificial Neural Networks for Coronary Heart Disease Classification

The optimisation of the accuracy of classifiers in pattern recognition is a complex problem that is often poorly understood. Whilst numerous techniques exist for the optimisa- tion of weights in artificial neural networks (e.g. the Widrow-Hoff least mean squares algorithm and back propagation techniques), there do not exist any hard and fast rules for choosing the structure of an artificial neural network - in particular for choosing both the number of the hidden layers used in the network and the size (in terms of number of neurons) of those hidden layers. However, this internal structure is one of the key factors in determining the accuracy of the classification. This paper proposes taking a multi-objective approach to the evolutionary design of artificial neural networks using a powerful optimiser based around the state-of-the-art MOEA/D- DRA algorithm and a novel method of incorporating decision maker preferences. In contrast to previous approaches, the novel approach outlined in this paper allows the intuitive consideration of trade-offs between classification objectives that are frequently present in complex classification problems but are often ignored. The effectiveness of the proposed multi-objective approach to evolving artificial neural networks is then shown on a real-world medical classification problem frequently used to benchmark classification method

Preference focussed many-objective evolutionary computation

Solving complex real-world problems often involves the simultaneous optimisation of multiple con icting performance criteria, these real-world problems occur in the elds of engineering, economics, chemistry, manufacturing, physics and many more. The optimisation process usually involves some design challenges in the form of the optimisation of a number of objectives and constraints. There exist many traditional optimisation methods (calculus based, random search, enumerative, etc...), however, these only o er a single solution in either adequate performance in a narrow problem domain or inadequate performance across a broad problem domain. Evolutionary Multi-objective Optimisation (EMO) algorithms are robust optimisers which are suitable for solving complex real-world multi-objective optimisation problems, as they are able to address each of the con icting objectives simultaneously. Typically, these EMO algorithms are run non-interactively with a Decision Maker (DM) setting the initial parameters of the algorithm and then analysing the results at the end of the optimisation process. When EMO is applied to real-world optimisation problems there is often a DM who is only interested in a portion of the Pareto-optimal front, however, incorporation of DM preferences is often neglected in the EMO literature. In this thesis, the incorporation of DM preferences into EMO search methods has been explored. This has been achieved through the review of EMO literature to identify a powerful method of variation, Covariance Matrix Adaptation (CMA), and its computationally infeasible EMO implementation, MO-CMA-ES. A CMA driven EMO algorithm, CMA-PAES, capable of optimisation in the presence of many objectives has been developed, benchmarked, and statistically veri ed to outperform MO-CMA-ES and MOEA/D-DRA on selected test suites. CMA-PAES and MOEA/D-DRA with the incorporation of the novel Weighted Z-score (WZ) preference articulation operator (supporting a priori, a posteriori or progressive incorporation) are then benchmarked on a range of synthetic and real-world problems. WZ-CMA-PAES is then successfully applied to a real-world problem regarding the optimisation of a classi er for concealed weapon detection, outperforming previously published classi er implementations

A Fast Hypervolume Driven Selection Mechanism for Many-Objective Optimisation Problems.

Solutions to real-world problems often require the simultaneous optimisation of multiple conflicting objectives. In the presence of four or more objectives, the problem is referred to as a “many-objective optimisation problem”. A problem of this category introduces many challenges, one of which is the effective and efficient selection of optimal solutions. The hypervolume indicator (or s-metric), i.e. the size of dominated objective space, is an effective selection criterion for many-objective optimisation. The indicator is used to measure the quality of a nondominated set, and can be used to sort solutions for selection as part of the contributing hypervolume indicator. However, hypervolume based selection methods can have a very high, if not infeasible, computational cost. The present study proposes a novel hypervolume driven selection mechanism for many-objective problems, whilst maintaining a feasible computational cost. This approach, named the Hypervolume Adaptive Grid Algorithm (HAGA), uses two-phases (narrow and broad) to prevent population-wide calculation of the contributing hypervolume indicator. Instead, HAGA only calculates the contributing hypervolume indicator for grid populations, i.e. for a few solutions, which are close in proximity (in the objective space) to a candidate solution when in competition for survival. The result is a trade-off between complete accuracy in selecting the fittest individuals in regards to hypervolume quality, and a feasible computational time in many-objective space. The real-world efficiency of the proposed selection mechanism is demonstrated within the optimisation of a classifier for concealed weapon detection

Age Detection Through Keystroke Dynamics From User Authentication Failures

In this paper an incident response approach is proposed for handling detections of authentication failures in systems that employ dynamic biometric authentication and more specifically keystroke user recognition. The main component of the approach is a multi layer perceptron focusing on the age classification of a user. Empirical findings show that the classifier can detect the age of the subject with a probability that is far from the uniform random distribution, making the proposed method suitable for providing supporting yet circumstantial evidence during e-discovery

Efficient Multi-Objective NeuroEvolution in Computer Vision and Applications for Threat Identification

Concealed threat detection is at the heart of critical security systems designed to en- sure public safety. Currently, methods for threat identification and detection are primarily manual, but there is a recent vision to automate the process. Problematically, developing computer vision models capable of operating in a wide range of settings, such as the ones arising in threat detection, is a challenging task involving multiple (and often conflicting) objectives. Automated machine learning (AutoML) is a flourishing field which endeavours to dis- cover and optimise models and hyperparameters autonomously, providing an alternative to classic, effort-intensive hyperparameter search. However, existing approaches typ- ically show significant downsides, like their (1) high computational cost/greediness in resources, (2) limited (or absent) scalability to custom datasets, (3) inability to provide competitive alternatives to expert-designed and heuristic approaches and (4) common consideration of a single objective. Moreover, most existing studies focus on standard classification tasks and thus cannot address a plethora of problems in threat detection and, more broadly, in a wide variety of compelling computer vision scenarios. This thesis leverages state-of-the-art convolutional autoencoders and semantic seg- mentation (Chapter 2) to develop effective multi-objective AutoML strategies for neural architecture search. These strategies are designed for threat detection and provide in- sights into some quintessential computer vision problems. To this end, the thesis first introduces two new models, a practical Multi-Objective Neuroevolutionary approach for Convolutional Autoencoders (MONCAE, Chapter 3) and a Resource-Aware model for Multi-Objective Semantic Segmentation (RAMOSS, Chapter 4). Interestingly, these ap- proaches reached state-of-the-art results using a fraction of computational resources re- quired by competing systems (0.33 GPU days compared to 3150), yet allowing for mul- tiple objectives (e.g., performance and number of parameters) to be simultaneously op- timised. This drastic speed-up was possible through the coalescence of neuroevolution algorithms with a new heuristic technique termed Progressive Stratified Sampling. The presented methods are evaluated on a range of benchmark datasets and then applied to several threat detection problems, outperforming previous attempts in balancing multiple objectives. The final chapter of the thesis focuses on thread detection, exploiting these two mod- els and novel components. It presents first a new modification of specialised proxy scores to be embedded in RAMOSS, enabling us to further accelerate the AutoML process even more drastically while maintaining avant-garde performance (above 85% precision for SIXray). This approach rendered a new automatic evolutionary Multi-objEctive method for cOncealed Weapon detection (MEOW), which outperforms state-of-the-art models for threat detection in key datasets: a gold standard benchmark (SixRay) and a security- critical, proprietary dataset. Finally, the thesis shifts the focus from neural architecture search to identifying the most representative data samples. Specifically, the Multi-objectIve Core-set Discovery through evolutionAry algorithMs in computEr vision approach (MIRA-ME) showcases how the new neural architecture search techniques developed in previous chapters can be adapted to operate on data space. MIRA-ME offers supervised and unsupervised ways to select maximally informative, compact sets of images via dataset compression. This operation can offset the computational cost further (above 90% compression), with a minimal sacrifice in performance (less than 5% for MNIST and less than 13% for SIXray). Overall, this thesis proposes novel model- and data-centred approaches towards a more widespread use of AutoML as an optimal tool for architecture and coreset discov- ery. With the presented and future developments, the work suggests that AutoML can effectively operate in real-time and performance-critical settings such as in threat de- tection, even fostering interpretability by uncovering more parsimonious optimal models. More widely, these approaches have the potential to provide effective solutions to chal- lenging computer vision problems that nowadays are typically considered unfeasible for AutoML settings

Interactive Decomposition Multi-Objective Optimization via Progressively Learned Value Functions

Decomposition has become an increasingly popular technique for evolutionary multi-objective optimization (EMO). A decomposition-based EMO algorithm is usually designed to approximate a whole Pareto-optimal front (PF). However, in practice, the decision maker (DM) might only be interested in her/his region of interest (ROI), i.e., a part of the PF. Solutions outside that might be useless or even noisy to the decision-making procedure. Furthermore, there is no guarantee to find the preferred solutions when tackling many-objective problems. This paper develops an interactive framework for the decomposition-based EMO algorithm to lead a DM to the preferred solutions of her/his choice. It consists of three modules, i.e., consultation, preference elicitation and optimization. Specifically, after every several generations, the DM is asked to score a few candidate solutions in a consultation session. Thereafter, an approximated value function, which models the DM's preference information, is progressively learned from the DM's behavior. In the preference elicitation session, the preference information learned in the consultation module is translated into the form that can be used in a decomposition-based EMO algorithm, i.e., a set of reference points that are biased toward to the ROI. The optimization module, which can be any decomposition-based EMO algorithm in principle, utilizes the biased reference points to direct its search process. Extensive experiments on benchmark problems with three to ten objectives fully demonstrate the effectiveness of our proposed method for finding the DM's preferred solutions.Comment: 25 pages, 18 figures, 3 table

AREA: An adaptive reference-set based evolutionary algorithm for multiobjective optimisation

The file attached to this record is the author's final peer reviewed version. The Publisher's final version can be found by following the DOI link.Population-based evolutionary algorithms have great potential to handle multiobjective optimisation problems. However, the performance of these algorithms depends largely on problem characteristics. There is a need to improve these algorithms for wide applicability. References, often specified by the decision maker’s preference in different forms, are very effective to boost the performance of algorithms. This paper proposes a novel framework for effective use of references to strengthen algorithms. This framework considers references as search targets which can be adjusted based on the information collected during the search. The proposed framework is combined with new strategies, such as reference adaptation and adaptive local mating, to solve different types of problems. The proposed algorithm is compared with state-of-the-arts on a wide range of problems with diverse characteristics. The comparison and extensive sensitivity analysis demonstrate that the proposed algorithm is competitive and robust across different types of problems studied in this paper

Virtual Reality Games for Motor Rehabilitation

This paper presents a fuzzy logic based method to track user satisfaction without the need for devices to monitor users physiological conditions. User satisfaction is the key to any product’s acceptance; computer applications and video games provide a unique opportunity to provide a tailored environment for each user to better suit their needs. We have implemented a non-adaptive fuzzy logic model of emotion, based on the emotional component of the Fuzzy Logic Adaptive Model of Emotion (FLAME) proposed by El-Nasr, to estimate player emotion in UnrealTournament 2004. In this paper we describe the implementation of this system and present the results of one of several play tests. Our research contradicts the current literature that suggests physiological measurements are needed. We show that it is possible to use a software only method to estimate user emotion