Hyper-heuristic decision tree induction

A hyper-heuristic is any algorithm that searches or operates in the space of heuristics as opposed to the space of solutions. Hyper-heuristics are increasingly used in function and combinatorial optimization. Rather than attempt to solve a problem using a fixed heuristic, a hyper-heuristic approach attempts to find a combination of heuristics that solve a problem (and in turn may be directly suitable for a class of problem instances). Hyper-heuristics have been little explored in data mining. This work presents novel hyper-heuristic approaches to data mining, by searching a space of attribute selection criteria for decision tree building algorithm. The search is conducted by a genetic algorithm. The result of the hyper-heuristic search in this case is a strategy for selecting attributes while building decision trees. Most hyper-heuristics work by trying to adapt the heuristic to the state of the problem being solved. Our hyper-heuristic is no different. It employs a strategy for adapting the heuristic used to build decision tree nodes according to some set of features of the training set it is working on. We introduce, explore and evaluate five different ways in which this problem state can be represented for a hyper-heuristic that operates within a decisiontree building algorithm. In each case, the hyper-heuristic is guided by a rule set that tries to map features of the data set to be split by the decision tree building algorithm to a heuristic to be used for splitting the same data set. We also explore and evaluate three different sets of low-level heuristics that could be employed by such a hyper-heuristic. This work also makes a distinction between specialist hyper-heuristics and generalist hyper-heuristics. The main difference between these two hyperheuristcs is the number of training sets used by the hyper-heuristic genetic algorithm. Specialist hyper-heuristics are created using a single data set from a particular domain for evolving the hyper-heurisic rule set. Such algorithms are expected to outperform standard algorithms on the kind of data set used by the hyper-heuristic genetic algorithm. Generalist hyper-heuristics are trained on multiple data sets from different domains and are expected to deliver a robust and competitive performance over these data sets when compared to standard algorithms. We evaluate both approaches for each kind of hyper-heuristic presented in this thesis. We use both real data sets as well as synthetic data sets. Our results suggest that none of the hyper-heuristics presented in this work are suited for specialization – in most cases, the hyper-heuristic’s performance on the data set it was specialized for was not significantly better than that of the best performing standard algorithm. On the other hand, the generalist hyper-heuristics delivered results that were very competitive to the best standard methods. In some cases we even achieved a significantly better overall performance than all of the standard methods

Técnicas de optimización paralelas : esquema híbrido basado en hiperheurísticas y computación evolutiva

Optimisation is the process of selecting the best element fr om a set of available alternatives. Solutions are termed good or bad depending on its performance for a set of objectives. Several algorithms to deal with such kind of problems have been defined in the literature. Metaheuristics are one of the most prominent techniques. They are a class of modern heuristics whose main goal is to com bine heuristics in a problem independent way with the aim of improving their per formance. Meta- heuristics have reported high-quality solutions in severa l fields. One of the reasons of the good behaviour of metaheuristics is that they are defin ed in general terms. Therefore, metaheuristic algorithms can be adapted to fit th e needs of most real-life optimisation. However, such an adaptation is a hard task, and it requires a high computational and user effort. There are two main ways of reducing the effort associated to th e usage of meta- heuristics. First, the application of hyperheuristics and parameter setting strategies facilitates the process of tackling novel optimisation pro blems and instances. A hyperheuristic can be viewed as a heuristic that iterativel y chooses between a set of given low-level metaheuristics in order to solve an optim isation problem. By using hyperheuristics, metaheuristic practitioners do no t need to manually test a large number of metaheuristics and parameterisations for d iscovering the proper algorithms to use. Instead, they can define the set of configur ations which must be tested, and the model tries to automatically detect the be st-behaved ones, in order to grant more resources to them. Second, the usage of pa rallel environments might speedup the process of automatic testing, so high qual ity solutions might be achieved in less time. This research focuses on the design of novel hyperheuristic s and defines a set of models to allow their usage in parallel environments. Differ ent hyperheuristics for controlling mono-objective and multi-objective multi-po int optimisation strategies have been defined. Moreover, a set of novel multiobjectivisa tion techniques has been proposed. In addition, with the aim of facilitating the usage of multiobjectivi- sation, the performance of models that combine the usage of m ultiobjectivisation and hyperheuristics has been studied. The proper performance of the proposed techniques has been v alidated with a set of well-known benchmark optimisation problems. In addi tion, several practical and complex optimisation problems have been addressed. Som e of the analysed problems arise in the communication field. In addition, a pac king problem proposed in a competition has been faced up. The proposals for such pro blems have not been limited to use the problem-independent schemes. Inste ad, new metaheuristics, operators and local search strategies have been defined. Suc h schemes have been integrated with the designed parallel hyperheuristics wit h the aim of accelerating the achievement of high quality solutions, and with the aim of fa cilitating their usage. In several complex optimisation problems, the current best -known solutions have been found with the methods defined in this dissertation.Los problemas de optimización son aquellos en los que hay que elegir cuál es la solución más adecuada entre un conjunto de alternativas. Actualmente existe una gran cantidad de algoritmos que permiten abordar este tipo de problemas. Entre ellos, las metaheurísticas son una de las técnicas más usadas. El uso de metaheurísticas ha posibilitado la resolución de una gran cantidad de problemas en diferentes campos. Esto se debe a que las metaheurísticas son técnicas generales, con lo que disponen de una gran cantidad de elementos o parámetros que pueden ser adaptados a la hora de afrontar diferentes problemas de optimización. Sin embargo, la elección de dichos parámetros no es sencilla, por lo que generalmente se requiere un gran esfuerzo computacional, y un gran esfuerzo por parte del usuario de estas técnicas. Existen diversas técnicas que atenúan este inconveniente. Por un lado, existen varios mecanismos que permiten seleccionar los valores de dichos parámetros de forma automática. Las técnicas más simples utilizan valores fijos durante toda la ejecución, mientras que las técnicas más avanzadas, como las hiperheurísticas, adaptan los valores usados a las necesidades de cada fase de optimización. Además, estas técnicas permiten usar varias metaheurísticas de forma simultánea. Por otro lado, el uso de técnicas paralelas permite acelerar el proceso de testeo automático, reduciendo el tiempo necesario para obtener soluciones de alta calidad. El objetivo principal de esta tesis ha sido diseñar nuevas hiperheurísticas e integrarlas en el modelo paralelo basado en islas. Estas técnicas se han usado para controlar los parámetros de varias metaheurísticas evolutivas. Se han definido diversas hiperheurísticas que han permitido abordar tanto problemas mono-objetivo como problemas multi-objetivo. Además, se han definido un conjunto de multiobjetivizaciones, que a su vez se han beneficiado de las hiperheurísticas propuestas. Las técnicas diseñadas se han validado con algunos de los problemas de test más ampliamente utilizados. Además, se han abordado un conjunto de problemas de optimización prácticos. Concretamente, se han tratado tres problemas que surgen en el ámbito de las telecomunicaciones, y un problema de empaquetado. En dichos problemas, además de usar las hiperheurísticas y multiobjetivizaciones, se han definido nuevos algoritmos, operadores, y estrategias de búsqueda local. En varios de los problemas, el uso combinado de todas estas técnicas ha posibilitado obtener las mejores soluciones encontradas hasta el momento

A genetic programming hyper-heuristic approach to automated packing

This thesis presents a programme of research which investigated a genetic programming hyper-heuristic methodology to automate the heuristic design process for one, two and three dimensional packing problems. Traditionally, heuristic search methodologies operate on a space of potential solutions to a problem. In contrast, a hyper-heuristic is a heuristic which searches a space of heuristics, rather than a solution space directly. The majority of hyper-heuristic research papers, so far, have involved selecting a heuristic, or sequence of heuristics, from a set pre-defined by the practitioner. Less well studied are hyper-heuristics which can create new heuristics, from a set of potential components. This thesis presents a genetic programming hyper-heuristic which makes it possible to automatically generate heuristics for a wide variety of packing problems. The genetic programming algorithm creates heuristics by intelligently combining components. The evolved heuristics are shown to be highly competitive with human created heuristics. The methodology is first applied to one dimensional bin packing, where the evolved heuristics are analysed to determine their quality, specialisation, robustness, and scalability. Importantly, it is shown that these heuristics are able to be reused on unseen problems. The methodology is then applied to the two dimensional packing problem to determine if automatic heuristic generation is possible for this domain. The three dimensional bin packing and knapsack problems are then addressed. It is shown that the genetic programming hyper-heuristic methodology can evolve human competitive heuristics, for the one, two, and three dimensional cases of both of these problems. No change of parameters or code is required between runs. This represents the first packing algorithm in the literature able to claim human competitive results in such a wide variety of packing domains

Problemas de corte: métodos exactos y aproximados para formulaciones mono y multi-objetivo

Los problemas de corte y empaquetado son una familia de problemas de optimización combinatoria que han sido ampliamente estudiados en numerosas áreas de la industria y la investigación, debido a su relevancia en una enorme variedad de aplicaciones reales. Son problemas que surgen en muchas industrias de producción donde se debe realizar la subdivisión de un material o espacio disponible en partes más pequeñas. Existe una gran variedad de métodos para resolver este tipo de problemas de optimización. A la hora de proponer un método de resolución para un problema de optimización, es recomendable tener en cuenta el enfoque y las necesidades que se tienen en relación al problema y su solución. Las aproximaciones exactas encuentran la solución óptima, pero sólo es viable aplicarlas a instancias del problema muy pequeñas. Las heurísticas manejan conocimiento específico del problema para obtener soluciones de alta calidad sin necesitar un excesivo esfuerzo computacional. Por otra parte, las metaheurísticas van un paso más allá, ya que son capaces de resolver una clase muy general de problemas computacionales. Finalmente, las hiperheurísticas tratan de automatizar, normalmente incorporando técnicas de aprendizaje, el proceso de selección, combinación, generación o adaptación de heurísticas más simples para resolver eficientemente problemas de optimización. Para obtener lo mejor de estos métodos se requiere conocer, además del tipo de optimización (mono o multi-objetivo) y el tamaño del problema, los medios computacionales de los que se dispone, puesto que el uso de máquinas e implementaciones paralelas puede reducir considerablemente los tiempos para obtener una solución. En las aplicaciones reales de los problemas de corte y empaquetado en la industria, la diferencia entre usar una solución obtenida rápidamente y usar propuestas más sofisticadas para encontrar la solución óptima puede determinar la supervivencia de la empresa. Sin embargo, el desarrollo de propuestas más sofisticadas y efectivas normalmente involucra un gran esfuerzo computacional, que en las aplicaciones reales puede provocar una reducción de la velocidad del proceso de producción. Por lo tanto, el diseño de propuestas efectivas y, al mismo tiempo, eficientes es fundamental. Por esta razón, el principal objetivo de este trabajo consiste en el diseño e implementación de métodos efectivos y eficientes para resolver distintos problemas de corte y empaquetado. Además, si estos métodos se definen como esquemas lo más generales posible, se podrán aplicar a diferentes problemas de corte y empaquetado sin realizar demasiados cambios para adaptarlos a cada uno. Así, teniendo en cuenta el amplio rango de metodologías de resolución de problemas de optimización y las técnicas disponibles para incrementar su eficiencia, se han diseñado e implementado diversos métodos para resolver varios problemas de corte y empaquetado, tratando de mejorar las propuestas existentes en la literatura. Los problemas que se han abordado han sido: el Two-Dimensional Cutting Stock Problem, el Two-Dimensional Strip Packing Problem, y el Container Loading Problem. Para cada uno de estos problemas se ha realizado una amplia y minuciosa revisión bibliográfica, y se ha obtenido la solución de las distintas variantes escogidas aplicando diferentes métodos de resolución: métodos exactos mono-objetivo y paralelizaciones de los mismos, y métodos aproximados multi-objetivo y paralelizaciones de los mismos. Los métodos exactos mono-objetivo aplicados se han basado en técnicas de búsqueda en árbol. Por otra parte, como métodos aproximados multi-objetivo se han seleccionado unas metaheurísticas multi-objetivo, los MOEAs. Además, para la representación de los individuos utilizados por estos métodos se han empleado codificaciones directas mediante una notación postfija, y codificaciones que usan heurísticas de colocación e hiperheurísticas. Algunas de estas metodologías se han mejorado utilizando esquemas paralelos haciendo uso de las herramientas de programación OpenMP y MPI. En el caso d

Heuristics approaches for three-dimensional strip packing and multiple carrier transportation plans

In transport logistic operations, an efficient delivery plan and better utilisation of vehicles will result in fuel cost savings, reduced working hours and even reduction of carbon dioxide emissions. This thesis proposes various algorithmic approaches to generate improved performance in automated vehicle load packing and route planning. First, modifications to best-fit heuristic methodologies are proposed and then incorporated into a simple but effective “look-ahead” heuristic procedure. The results obtained are very competitive and in some cases best-known results are found for different sets of constraints on three-dimensional strip packing problems. Secondly, a review and comparison of different clustering techniques in transport route planning is presented. This study shows that the algorithmic approach performs according to the specific type of real-world transport route planning scenario under consideration. This study helps to achieve a better understanding of how to conduct the automated generation of vehicle routes that meet the specific conditions required in the operations of a transport logistics company. Finally, a new approach to measuring the quality of transportation route plans is presented showing how this procedure has a positive effect on the quality of the generated route plans. In summary, this thesis proposes new tailored and effective heuristic methodologies that have been tested and incorporated into the real-world operations of a transport logistics company. The research work presented here is a modest yet significant advance to better understanding and solving the difficult problems of vehicle loading and routing in real-world scenarios

Novel Hyper-heuristics Applied to the Domain of Bin Packing

Principal to the ideology behind hyper-heuristic research is the desire to increase the level of generality of heuristic procedures so that they can be easily applied to a wide variety of problems to produce solutions of adequate quality within practical timescales.This thesis examines hyper-heuristics within a single problem domain, that of Bin Packing where the benefits to be gained from selecting or generating heuristics for large problem sets with widely differing characteristics is considered. Novel implementations of both selective and generative hyper-heuristics are proposed. The former approach attempts to map the characteristics of a problem to the heuristic that best solves it while the latter uses Genetic Programming techniques to automate the heuristic design process. Results obtained using the selective approach show that solution quality was improved significantly when contrasted to the performance of the best single heuristic when applied to large sets of diverse problem instances. Although enforcing the benefits to be gained by selecting from a range of heuristics the study also highlighted the lack of diversity in human designed algorithms. Using Genetic Programming techniques to automate the heuristic design process allowed both single heuristics and collectives of heuristics to be generated that were shown to perform significantly better than their human designed counterparts. The thesis concludes by combining both selective and generative hyper-heuristic approaches into a novel immune inspired system where heuristics that cover distinct areas of the problem space are generated. The system is shown to have a number of advantages over similar cooperative approaches in terms of its plasticity, efficiency and long term memory. Extensive testing of all of the hyper-heuristics developed on large sets of both benchmark and newly generated problem instances enforces the utility of hyper-heuristics in their goal of producing fast understandable procedures that give good quality solutions for a range of problems with widely varying characteristics

Heuristics approaches for three-dimensional strip packing and multiple carrier transportation plans

In transport logistic operations, an efficient delivery plan and better utilisation of vehicles will result in fuel cost savings, reduced working hours and even reduction of carbon dioxide emissions. This thesis proposes various algorithmic approaches to generate improved performance in automated vehicle load packing and route planning. First, modifications to best-fit heuristic methodologies are proposed and then incorporated into a simple but effective “look-ahead” heuristic procedure. The results obtained are very competitive and in some cases best-known results are found for different sets of constraints on three-dimensional strip packing problems. Secondly, a review and comparison of different clustering techniques in transport route planning is presented. This study shows that the algorithmic approach performs according to the specific type of real-world transport route planning scenario under consideration. This study helps to achieve a better understanding of how to conduct the automated generation of vehicle routes that meet the specific conditions required in the operations of a transport logistics company. Finally, a new approach to measuring the quality of transportation route plans is presented showing how this procedure has a positive effect on the quality of the generated route plans. In summary, this thesis proposes new tailored and effective heuristic methodologies that have been tested and incorporated into the real-world operations of a transport logistics company. The research work presented here is a modest yet significant advance to better understanding and solving the difficult problems of vehicle loading and routing in real-world scenarios

An investigation of multi-objective hyper-heuristics for multi-objective optimisation

In this thesis, we investigate and develop a number of online learning selection choice function based hyper-heuristic methodologies that attempt to solve multi-objective unconstrained optimisation problems. For the first time, we introduce an online learning selection choice function based hyperheuristic framework for multi-objective optimisation. Our multi-objective hyper-heuristic controls and combines the strengths of three well-known multi-objective evolutionary algorithms (NSGAII, SPEA2, and MOGA), which are utilised as the low level heuristics. A choice function selection heuristic acts as a high level strategy which adaptively ranks the performance of those low-level heuristics according to feedback received during the search process, deciding which one to call at each decision point. Four performance measurements are integrated into a ranking scheme which acts as a feedback learning mechanism to provide knowledge of the problem domain to the high level strategy. To the best of our knowledge, for the first time, this thesis investigates the influence of the move acceptance component of selection hyper-heuristics for multi-objective optimisation. Three multi-objective choice function based hyper-heuristics, combined with different move acceptance strategies including All-Moves as a deterministic move acceptance and the Great Deluge Algorithm (GDA) and Late Acceptance (LA) as a nondeterministic move acceptance function. GDA and LA require a change in the value of a single objective at each step and so a well-known hypervolume metric, referred to as D metric, is proposed for their applicability to the multi-objective optimisation problems. D metric is used as a way of comparing two non-dominated sets with respect to the objective space. The performance of the proposed multi-objective selection choice function based hyper-heuristics is evaluated on the Walking Fish Group (WFG) test suite which is a common benchmark for multi-objective optimisation. Additionally, the proposed approaches are applied to the vehicle crashworthiness design problem, in order to test its effectiveness on a realworld multi-objective problem. The results of both benchmark test problems demonstrate the capability and potential of the multi-objective hyper-heuristic approaches in solving continuous multi-objective optimisation problems. The multi-objective choice function Great Deluge Hyper-Heuristic (HHMO_CF_GDA) turns out to be the best choice for solving these types of problems