Problem dependent metaheuristic performance in Bayesian network structure learning.

Bayesian network (BN) structure learning from data has been an active research area in the machine learning field in recent decades. Much of the research has considered BN structure learning as an optimization problem. However, the finding of optimal BN from data is NP-hard. This fact has driven the use of heuristic algorithms for solving this kind of problem. Amajor recent focus in BN structure learning is on search and score algorithms. In these algorithms, a scoring function is introduced and a heuristic search algorithm is used to evaluate each network with respect to the training data. The optimal network is produced according to the best score evaluated. This thesis investigates a range of search and score algorithms to understand the relationship between technique performance and structure features of the problems. The main contributions of this thesis include (a) Two novel Ant Colony Optimization based search and score algorithms for BN structure learning; (b) Node juxtaposition distribution for studying the relationship between the best node ordering and the optimal BN structure; (c) Fitness landscape analysis for investigating the di erent performances of both chain score function and the CH score function; (d) A classifier method is constructed by utilizing receiver operating characteristic curve with the results on fitness landscape analysis; and finally (e) a selective o -line hyperheuristic algorithm is built for unseen BN structure learning with search and score algorithms. In this thesis, we also construct a new algorithm for producing BN benchmark structures and apply our novel approaches to a range of benchmark problems and real world problem

Hyper-heuristic decision tree induction

A hyper-heuristic is any algorithm that searches or operates in the space of heuristics as opposed to the space of solutions. Hyper-heuristics are increasingly used in function and combinatorial optimization. Rather than attempt to solve a problem using a fixed heuristic, a hyper-heuristic approach attempts to find a combination of heuristics that solve a problem (and in turn may be directly suitable for a class of problem instances). Hyper-heuristics have been little explored in data mining. This work presents novel hyper-heuristic approaches to data mining, by searching a space of attribute selection criteria for decision tree building algorithm. The search is conducted by a genetic algorithm. The result of the hyper-heuristic search in this case is a strategy for selecting attributes while building decision trees. Most hyper-heuristics work by trying to adapt the heuristic to the state of the problem being solved. Our hyper-heuristic is no different. It employs a strategy for adapting the heuristic used to build decision tree nodes according to some set of features of the training set it is working on. We introduce, explore and evaluate five different ways in which this problem state can be represented for a hyper-heuristic that operates within a decisiontree building algorithm. In each case, the hyper-heuristic is guided by a rule set that tries to map features of the data set to be split by the decision tree building algorithm to a heuristic to be used for splitting the same data set. We also explore and evaluate three different sets of low-level heuristics that could be employed by such a hyper-heuristic. This work also makes a distinction between specialist hyper-heuristics and generalist hyper-heuristics. The main difference between these two hyperheuristcs is the number of training sets used by the hyper-heuristic genetic algorithm. Specialist hyper-heuristics are created using a single data set from a particular domain for evolving the hyper-heurisic rule set. Such algorithms are expected to outperform standard algorithms on the kind of data set used by the hyper-heuristic genetic algorithm. Generalist hyper-heuristics are trained on multiple data sets from different domains and are expected to deliver a robust and competitive performance over these data sets when compared to standard algorithms. We evaluate both approaches for each kind of hyper-heuristic presented in this thesis. We use both real data sets as well as synthetic data sets. Our results suggest that none of the hyper-heuristics presented in this work are suited for specialization – in most cases, the hyper-heuristic’s performance on the data set it was specialized for was not significantly better than that of the best performing standard algorithm. On the other hand, the generalist hyper-heuristics delivered results that were very competitive to the best standard methods. In some cases we even achieved a significantly better overall performance than all of the standard methods

Automated Design of Metaheuristic Algorithms: A Survey

Metaheuristics have gained great success in academia and practice because their search logic can be applied to any problem with available solution representation, solution quality evaluation, and certain notions of locality. Manually designing metaheuristic algorithms for solving a target problem is criticized for being laborious, error-prone, and requiring intensive specialized knowledge. This gives rise to increasing interest in automated design of metaheuristic algorithms. With computing power to fully explore potential design choices, the automated design could reach and even surpass human-level design and could make high-performance algorithms accessible to a much wider range of researchers and practitioners. This paper presents a broad picture of automated design of metaheuristic algorithms, by conducting a survey on the common grounds and representative techniques in terms of design space, design strategies, performance evaluation strategies, and target problems in this field

Novel Hyper-heuristics Applied to the Domain of Bin Packing

Principal to the ideology behind hyper-heuristic research is the desire to increase the level of generality of heuristic procedures so that they can be easily applied to a wide variety of problems to produce solutions of adequate quality within practical timescales.This thesis examines hyper-heuristics within a single problem domain, that of Bin Packing where the benefits to be gained from selecting or generating heuristics for large problem sets with widely differing characteristics is considered. Novel implementations of both selective and generative hyper-heuristics are proposed. The former approach attempts to map the characteristics of a problem to the heuristic that best solves it while the latter uses Genetic Programming techniques to automate the heuristic design process. Results obtained using the selective approach show that solution quality was improved significantly when contrasted to the performance of the best single heuristic when applied to large sets of diverse problem instances. Although enforcing the benefits to be gained by selecting from a range of heuristics the study also highlighted the lack of diversity in human designed algorithms. Using Genetic Programming techniques to automate the heuristic design process allowed both single heuristics and collectives of heuristics to be generated that were shown to perform significantly better than their human designed counterparts. The thesis concludes by combining both selective and generative hyper-heuristic approaches into a novel immune inspired system where heuristics that cover distinct areas of the problem space are generated. The system is shown to have a number of advantages over similar cooperative approaches in terms of its plasticity, efficiency and long term memory. Extensive testing of all of the hyper-heuristics developed on large sets of both benchmark and newly generated problem instances enforces the utility of hyper-heuristics in their goal of producing fast understandable procedures that give good quality solutions for a range of problems with widely varying characteristics

Multiobjective in-core fuel management optimisation for nuclear research reactors

Thesis (PhD)--Stellenbosch University, 2016.ENGLISH SUMMARY : The efficiency and effectiveness of fuel usage in a typical nuclear reactor is influenced by the specific arrangement of available fuel assemblies in the reactor core positions. This arrangement of assemblies is referred to as a fuel reload configuration and usually has to be determined anew for each operational cycle of a reactor. Very often, multiple objectives are pursued simultaneously when designing a reload configuration, especially in the context of nuclear research reactors. In the multiobjective in-core fuel management optimization (MICFMO) problem, the aim is to identify a Pareto optimal set of compromise or trade-off reload configurations. Such a set may then be presented to a decision maker (i.e. a nuclear reactor operator) for consideration so as to select a preferred configuration. In the first part of this dissertation, a secularization-based methodology for MICFMO is pro- posed in order to address several shortcomings associated with the popular weighting method often employed in the literature for solving the MICFMO problem. The proposed methodology has been implemented in a reactor simulation code, called the OSCAR-4 system. In order to demonstrate its practical applicability, the methodology is applied to solve several MICFMO problem instances in the context of two research reactors. In the second part of the dissertation, an extensive investigation is conducted into the suitability of several multiobjective optimization algorithms for solving the constrained MICFMO problem. The computation time required to perform the investigation is reduced through the usage of several artificial neural networks constructed in the dissertation for objective and constraint function evaluations. Eight multiobjective metaheuristics are compared in the context of a test suite of several MICFMO problem instances, based on the SAFARI-1 research reactor in South Africa. The investigation reveals that the NSGA-II, the P-ACO algorithm and the MOOCEM are generally the best-performing metaheuristics across the problem instances in the test suite, while the MOVNS algorithm also performs well in the context of bi-objective problem instances. As part of this investigation, a multiplicative penalty function (MPF) constraint handling technique is also proposed and compared to an existing constraint handling technique, called constrained-domination. The comparison reveals that the MPF technique is a competitive alternative to constrained-domination. In an attempt to raise the level of generality at which MICFMO may be performed and potentially improve the quality of optimization results, a multiobjective hyperheuristic, called the AMALGAM method, is also considered in this dissertation. This hyperheuristic incorporates multiple metaheuristic sub-algorithms simultaneously for optimization. Testing reveals that the AMALGAM method yields superior results in the majority of problem instances in the test suite, thus achieving the dual goal of raising the level of generality and of yielding improved optimization results. The method has also been implemented in the OSCAR-4 system and is applied to solve several MICFMO case study problem instances, based on two research reactors, in order to demonstrate its practical applicability. Finally, in the third part of this dissertation, a conceptual framework is proposed for an optimization-based personal decision support system, dedicated to MICFM. This framework may serve as the basis for developing a computerized tool to aid nuclear reactor operators in designing suitable reload configurations.AFRIKAANSE OPSOMMING : Die doeltreffendheid en doelmatigheid van brandstofverbruik in 'n tipiese kernreaktor word deur die spesieke rangskikking van beskikbare brandstofelemente in die laaiposisies van die reaktor beinvloed. Hierdie rangskikking staan bekend as 'n brandstof herlaaikongurasie en word gewoonlik opnuut bepaal vir elke operasionele siklus van 'n reaktor. Die gelyktydige optimering van veelvuldige doele word dikwels tydens die ontwerp van 'n herlaaikongurasie nagestreef, veral binne die konteks van navorsingsreaktore. Die doelwit van meerdoelige binne-kern brandstofbeheeroptimering (MBKBBO) is om 'n Pareto optimale versameling van herlaaikongurasieafruilings te identiseer. So 'n versameling mag dan vir oorweging (deur byvoorbeeld 'n kernreaktoroperateur) voorgele word sodat 'n voorkeurkongurasie gekies kan word. In die eerste gedeelte van hierdie proefskrif word 'n skalariseringsgebaseerde metodologie vir MBKBBO voorgestel om verskeie tekortkominge in die gewilde gewigverswaringsmetode aan te spreek. Laasgenoemde metode word gereeld in die literatuur gebruik om die MBKBBO probleem op te los. Die voorgestelde metodologie is in 'n reaktorsimulasiestelsel, bekend as die OSCAR-4 stelsel, geimplementeer. Om die praktiese toepasbaarheid daarvan te demonstreer, word die metodologie gebruik om 'n aantal MBKBBO probleemgevalle binne die konteks van twee navorsingsreaktore op te los. In die tweede gedeelte van die proefskrif word 'n uitgebreide ondersoek ingestel om die geskiktheid van verskeie meerdoelige optimeringsalgoritmes vir die oplos van die beperkte MBKBBO probleem te bepaal. Die berekeningstyd wat vir die ondersoek benodig word, word verminder deur die gebruik van kunsmatige neurale netwerke, wat in die proefskrif gekonstrueer word, om doelfunksies en beperkings te evalueer. Agt meerdoelige metaheuristieke word binne die konteks van verskeie MBKBBO toetsprobleemgevalle vergelyk wat op die SAFARI-1 navorsingsreaktor in Suid-Afrika gebaseer is. Toetse dui daarop dat die NSGA-II, die P-ACO algoritme en die MOOCEM oor die algemeen die beste oor al die toetsprobleemgevalle presteer. Die MOVNS algoritme presteer ook goed in die konteks van tweedoelige probleemgevalle. 'n Vermenigvuldigende boetefunksie (VBF) beperkinghanteringstegniek word ook voorgestel en vergelyk met 'n bestaande tegniek bekend as beperkte dominasie. Daar word bevind dat the VBF tegniek 'n mededingende alternatief tot beperkte dominasie is. 'n Poging word aangewend om die vlak van algemeenheid waarmee MBKBBO uitgevoer word, te verhoog, asook om potensieel die kwaliteit van die optimeringsresultate te verbeter. 'n Meerdoelige hiperheuristiek, bekend as die AMALGAM metode, word in die nastreef van hierdie twee doelwitte oorweeg. Die metode funksioneer deur middel van die gelyktydige insluiting van 'n aantal metaheuristieke deel-algoritmes. Toetse dui daarop dat the AMALGAM metode beter resultate vir die meerderheid van toetsprobleme lewer, en dus word die bogenoemde twee doelwitte bereik. Die metode is ook in the OSCAR-4 stelsel ge mplementeer en word gebruik om 'n aantal MBKBBO gevallestudie probleemgevalle (binne die konteks van twee navorsingsreaktore) op te los. Sodoende word die praktiese toepasbaarheid van die metode gedemonstreer. In die derde deel van die proefskrif word 'n konseptuele raamwerk laastens vir 'n optimeringsgebaseerde persoonlike besluitsteunstelsel gemik op MBKBB, voorgestel. Hierdie raamwerk mag as grondslag dien vir die ontwikkeling van 'n gerekenariseerde hulpmiddel vir kernreaktoroperateurs om aanvaarbare herlaaikongurasies te ontwerp.Doctora

A review on the self and dual interactions between machine learning and optimisation

Machine learning and optimisation are two growing fields of artificial intelligence with an enormous number of computer science applications. The techniques in the former area aim to learn knowledge from data or experience, while the techniques from the latter search for the best option or solution to a given problem. To employ these techniques automatically and effectively aligning with the real aim of artificial intelligence, both sets of techniques are frequently hybridised, interacting with each other and themselves. This study focuses on such interactions aiming at (1) presenting a broad overview of the studies on self and dual interactions between machine learning and optimisation; (2) providing a useful tutorial for researchers and practitioners in both fields in support of collaborative work through investigation of the recent advances and analyses of the advantages and disadvantages of different techniques to tackle the same or similar problems; (3) clarifying the overlapping terminologies having different meanings used in both fields; (4) identifying research gaps and potential research directions

Machine learning for improving heuristic optimisation

Heuristics, metaheuristics and hyper-heuristics are search methodologies which have been preferred by many researchers and practitioners for solving computationally hard combinatorial optimisation problems, whenever the exact methods fail to produce high quality solutions in a reasonable amount of time. In this thesis, we introduce an advanced machine learning technique, namely, tensor analysis, into the field of heuristic optimisation. We show how the relevant data should be collected in tensorial form, analysed and used during the search process. Four case studies are presented to illustrate the capability of single and multi-episode tensor analysis processing data with high and low abstraction levels for improving heuristic optimisation. A single episode tensor analysis using data at a high abstraction level is employed to improve an iterated multi-stage hyper-heuristic for cross-domain heuristic search. The empirical results across six different problem domains from a hyper-heuristic benchmark show that significant overall performance improvement is possible. A similar approach embedding a multi-episode tensor analysis is applied to the nurse rostering problem and evaluated on a benchmark of a diverse collection of instances, obtained from different hospitals across the world. The empirical results indicate the success of the tensor-based hyper-heuristic, improving upon the best-known solutions for four particular instances. Genetic algorithm is a nature inspired metaheuristic which uses a population of multiple interacting solutions during the search. Mutation is the key variation operator in a genetic algorithm and adjusts the diversity in a population throughout the evolutionary process. Often, a fixed mutation probability is used to perturb the value at each locus, representing a unique component of a given solution. A single episode tensor analysis using data with a low abstraction level is applied to an online bin packing problem, generating locus dependent mutation probabilities. The tensor approach improves the performance of a standard genetic algorithm on almost all instances, significantly. A multi-episode tensor analysis using data with a low abstraction level is embedded into multi-agent cooperative search approach. The empirical results once again show the success of the proposed approach on a benchmark of flow shop problem instances as compared to the approach which does not make use of tensor analysis. The tensor analysis can handle the data with different levels of abstraction leading to a learning approach which can be used within different types of heuristic optimisation methods based on different underlying design philosophies, indeed improving their overall performance