Smith, D

The Genetic and Evolutionary Computation Conference

Tokarchuk, L

Wiggins, G

Queen Mary Research Online

MADNS: Multiple Assessment Directed Novelty SearchSmith, D; Wiggins, G; Tokarchuk, L; The Genetic and Evolutionary Computation Conference     c 2016 Copyright held by the owner/author(s)  For additional information about this publication click this link.http://qmro.qmul.ac.uk/xmlui/handle/123456789/13741   Information about this research object was correct at the time of download; we occasionallymake corrections to records, please therefore check the published record when citing. Formore information contact scholarlycommunications@qmul.ac.ukExploring Conflicting Objectives with MADNS:Multiple Assessment Directed Novelty SearchDavy Smith, Laurissa Tokarchuk and Geraint WigginsQueen Mary University of LondonSchool of Electronic Engineering and Computer ScienceLondon, E1 4NS, UK{david.smith,laurissa.tokarchuk,geraint.wiggins}@qmul.ac.ukABSTRACTNovelty search is an evolutionary approach which promotesphenotypic diversity in a population. Novelty search hasbeen successfully applied to a wide range of domains and anumber of variants have been proposed. Here we introduceMultiple Assessment Directed Novelty Search (MADNS),which exploits the notion that a diverse population opti-mised through phenotypic novelty may contain solutions tomultiple conflicting objectives. We show that by utilisingthe MADNS algorithm, an evolutionary trajectory may besimultaneously directed towards conflicting objectives. Weconclude that, through applying MADNS and MC-MADNS,a divergent evolutionary trajectory may be directed to pro-vide simultaneous solutions to multiple conflicting problemsin domains with large potential for exploration.KeywordsNovelty search; algorithm design; phenotypic diversity; neu-roevolution; evolutionary robotics1. INTRODUCTIONNovelty search [?] is an evolutionary procedure which op-timises a population through the promotion of behaviouraldiversity. Unlike traditional fitness based search, which con-verges towards a particular solution, novelty search is a di-vergent procedure, exploring the phenotypic landscape forpotentially useful solutions in a given domain.Here we propose a novel approach — multiple assess-ment directed novelty search (MADNS), which allows thetrajectory of phenotypic exploration to be directed simul-taneously towards multiple conflicting objectives. We pro-pose variants of the MADNS algorithm based on both tra-ditional novelty search and minimal criteria novelty search(MC-MADNS), highlighting the suitability to extend to un-bounded domains. Our results show that, in large pheno-typic landscapes, directing novelty search towards multipleconflicting objectives produces more optimal solutions andPermission to make digital or hard copies of part or all of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for third-party components of this work must be honored.For all other uses, contact the owner/author(s).GECCO ’16 July 20-24, 2016, Denver, CO, USAc© 2016 Copyright held by the owner/author(s).ACM ISBN 978-1-4503-4323-7/16/07.DOI: http://dx.doi.org/10.1145/2908961.2908975with greater reliability than both novelty alone and minimalcriteria novelty search (MCNS).2. MULTIPLE ASSESSMENT DIRECTEDNOVELTY SEARCHMADNS is an extension to the novelty search algorithmthat rewards high performing solutions over a number ofpredefined objectives. Formally, let the set P denote thecurrent population, with an individual solution defined asρ ∈ P . Next, for a given domain with k objectives, definedby the functions a1(ρ), a2(ρ), . . . , ak(ρ), where ak : P 7→ R,let the set A = {a1(ρ), a2(ρ), . . . , ak(ρ)}. Let the subsetQ ⊂ P contain the maximal solutions for each objective,where:Q = argmaxρ∈Pa1(ρ)⋃argmaxρ∈Pa2(ρ)⋃...⋃argmaxρ∈Pak(ρ).Let fnov(p) be the novelty of a solution and the maxi-mal novelty value for the current population be defined asα = maxρ∈Pfnov(ρ). Finally, let fmad(ρ) be the fitness of anindividual solution, calculated as in ??:fmad(p) ={fnov(p) if p /∈ Q,α if p ∈ Q. (1)A minimal criteria variant of this (MC-MADNS) maybe defined through the replacement of novelty search withMCNS [?]:fmc−mad(p) ={fmcns(p) if p /∈ Q,α if p ∈ Q. (2)3. EXPERIMENTOur experimental domain is based upon previous studieswhich have assessed novelty search and variants of the algo-rithm [?]. The task domain is a simulated maze, in which anagent controller must navigate from an initial starting-pointto one of a possible number of exit points within a fixedtime limit. To fully assess both novelty search and MCNS,the tested domains include both bounded and unboundedversions (??).(a) DIVERGE (b) DIVERGE-U (c) SUBSET (d) SUBSET-Ull10001000(e) SCALEDll10001000(f) SCALED-UFigure 1: Mazes tested.4. RESULTS0 500 1000 1500Generations0.00.20.40.60.81.0ProbabilityofSuccess MCNSMC-MADNSMADNSNOVELTYNS-MAXOBJ-LINOBJECTIVE(a) DIVERGE0 500 1000 1500Generations0.00.20.40.60.81.0ProbabilityofSuccess MCNSMC-MADNSMADNSNOVELTYNS-MAXOBJ-LINOBJECTIVE(b) DIVERGE-U0 500 1000 1500Generations0.00.20.40.60.81.0ProbabilityofSuccess MCNSMC-MADNSMADNSNOVELTYNS-MAXOBJ-LINOBJECTIVE(c) SUBSET0 500 1000 1500Generations0.00.20.40.60.81.0ProbabilityofSuccess MCNSMC-MADNSMADNSNOVELTYNS-MAXOBJ-LINOBJECTIVE(d) SUBSET-U0 500 1000 1500Generations0.00.20.40.60.81.0ProbabilityofSuccess MCNSMC-MADNSMADNSNOVELTYNS-MAXOBJ-LINOBJECTIVE(e) SCALED0 500 1000 1500Generations0.00.20.40.60.81.0ProbabilityofSuccess MCNSMC-MADNSMADNSNOVELTYNS-MAXOBJ-LINOBJECTIVE(f) SCALED-UFigure 2: Probability of locating all exitswithin number of generations?? shows the probability of success for each algorithmin each of the mazes over 100 evolutionary runs. In thebounded domains, the probability of success between MADNSand NS and their minimal criteria variants, MC-MADNSand MCNS, is identical due to the impossibility of a solu-tion to fail the minimal criteria. In the DIVERGE domain,NS achieves its maximum value in fewer generations thanMADNS. Conversely, in the SUBSET domain, MADNS achievesthe same probability of success as NS in fewer generations.NS-MAX produces comparable levels of performance to NSand MADNS in all domains. In the SCALED domain, MADNSand MC-MADNS significantly outperform all other algo-rithms (p < 0.01).The probability of success achieved by MCNS and MC-MADNS in the DIVERGE-U domain is of a level compa-rable to the DIVERGE domain (????). In the SUBSET-Udomain, MC-MADNS and MCNS achieve higher probabilityof success than in the SUBSET domain (????), with MC-MADNS outperforming MCNS in this domain (??). Allalgorithms produce results with lower probability of suc-cess (pos) in the SCALED-U domain MC-MADNS (pos =0.80) and MADNS (pos = 0.43) are less effected by the un-bounded SCALED-U domain than MCNS (pos = 0.38) andNS (pos = 0.21), with MC-MADNS significantly outper-forming all other algorithms (p < 0.0001).5. CONCLUSIONIn this paper, we presented a novel approach for hybriddirected-divergent search towards multiple conflicting objec-tives in both bounded and unbounded domains. Our resultshave shown that, in domains which have a low explorationpotential, NS is an effective method for the optimisation ofpopulations towards multiple conflicting objectives. How-ever, our proposed MADNS and MC-MADNS, dependantupon the domain, are significantly more effective. For do-mains which have high exploration potential, the MADNSand MC-MADNS algorithms have a higher probability ofsuccess than both NS and MCNS.5.1 AcknowledgementsThis work was funded by EPSRC through the Media andArts Technology Programme, an RCUK Doctoral TrainingCentre EP/G03723X/1.

MADNS: Multiple Assessment Directed Novelty Search

https://qmro.qmul.ac.uk/xmlui/bitstream/handle/123456789/13741/Tokarchuk%20MADNS%20Multiple%20Assessment%20Directed%202016%20Accepted.pdf?sequence=1

MADNS: Multiple Assessment Directed Novelty Search

Abstract

Similar works

Full text

Available Versions

Queen Mary Research Online