New strategies for efficient and practical genetic programming.

2006/2007In the last decades, engineers and decision makers expressed a growing interest in the development of effective modeling and simulation methods to understand or predict the behavior of many phenomena in science and engineering. Many of these phenomena are translated in mathematical models for convenience and to carry out an easy interpretation. Methods commonly employed for this purpose include, for example, Neural Networks, Simulated Annealing, Genetic Algorithms, Tabu search, and so on. These methods all seek for the optimal or near optimal values of a predefined set of parameters of a model built a priori. But in this case, a suitable model should be known beforehand. When the form of this model cannot be found, the problem can be seen from another level where the goal is to find a program or a mathematical representation which can solve the problem. According to this idea the modeling step is performed automatically thanks to a quality criterion which drives the building process. In this thesis, we focus on the Genetic Programming (GP) approach as an automatic method for creating computer programs by means of artificial evolution based upon the original contributions of Darwin and Mendel. While GP has proven to be a powerful means for coping with problems in which finding a solution and its representation is difficult, its practical applicability is still severely limited by several factors. First, the GP approach is inherently a stochastic process. It means there is no guarantee to obtain a satisfactory solution at the end of the evolutionary loop. Second, the performances on a given problem may be strongly dependent on a broad range of parameters, including the number of variables involved, the quantity of data for each variable, the size and composition of the initial population, the number of generations and so on. On the contrary, when one uses Genetic Programming to solve a problem, he has two expectancies: on the one hand, maximize the probability to obtain an acceptable solution, and on the other hand, minimize the amount of computational resources to get this solution. Initially we present innovative and challenging applications related to several fields in science (computer science and mechanical science) which participate greatly in the experience gained in the GP field. Then we propose new strategies for improving the performances of the GP approach in terms of efficiency and accuracy. We probe our approach on a large set of benchmark problems in three different domains. Furthermore we introduce a new approach based on GP dedicated to symbolic regression of multivariate data-sets where the underlying phenomenon is best characterized by a discontinuous function. These contributions aim to provide a better understanding of the key features and the underlying relationships which make enhancements successful in improving the original algorithm.Negli ultimi anni, ingegneri e progettisti hanno espresso un interesse crescente nello sviluppo di nuovi metodi di simulazione e di modellazione per comprendere e predire il comportamento di diversi fenomeni sia in ambito scientifico che ingegneristico. Molti di questi fenomeni vengono descritti attraverso modelli matematici che ne facilitano l'interpretazione. A questo fine, i metodi più comunemente impiegati sono, le tecniche basate sui Reti Neurali, Simulated Annealing, gli Algoritmi Genetici, la ricerca Tabu, ecc. Questi metodi vanno a determinare i valori ottimali o quasi ottimali dei parametri di un modello costruito a priori. E evidente che in tal caso, si dovrebbe conoscere in anticipo un modello idoneo. Quando ciò non è possibile, il problema deve essere considerato da un altro punto di vista: l'obiettivo è trovare un programma o una rappresentazione matematica che possano risolvere il problema. A questo scopo, la fase di modellazione è svolta automaticamente in funzione di un criterio qualitativo che guida il processo di ricerca. Il tema di ricerca di questa tesi è la programmazione genetica (“Genetic Programming” che chiameremo GP) e le sue applicazioni. La programmazione genetica si può definire come un metodo automatico per la generazione di programmi attraverso una simulazione artificiale dei principi relativi all'evoluzione naturale basata sui contributi originali di Darwin e di Mendel. La programmazione genetica ha dimostrato di essere un potente mezzo per affrontare quei problemi in cui trovare una soluzione e la sua rappresentazione è difficile. Però la sua applicabilità rimane severamente limitata da diversi fattori. In primo luogo, il metodo GP è inerentemente un processo stocastico. Ciò significa che non garantisce che una soluzione soddisfacente sarà trovata alla fine del ciclo evolutivo. In secondo luogo, le prestazioni su un dato problema dipendono fortemente da una vasta gamma di parametri, compresi il numero di variabili impiegate, la quantità di dati per ogni variabile, la dimensione e la composizione della popolazione iniziale, il numero di generazioni e così via. Al contrario, un utente della programmazione genetica ha due aspettative: da una parte, massimizzare la probabilità di ottenere una soluzione accettabile, e dall'altra, minimizzare la quantità di risorse di calcolo per ottenerla. Nella fase iniziale di questo lavoro sono state considerate delle applicazioni particolarmente innovative relative a diversi campi della scienza (informatica e meccanica) che hanno contributo notevolmente all'esperienza acquisita nel campo della programmazione genetica. In questa tesi si propone un nuovo procedimento con lo scopo di migliorare le prestazioni della programmazione genetica in termini di efficienza ed accuratezza. Abbiamo testato il nostro approccio su un ampio insieme di benchmarks in tre domini applicativi diversi. Si propone inoltre una tecnica basata sul GP per la regressione simbolica di data-set multivariati dove il fenomeno di fondo è caratterizzato da una funzione discontinua. Questi contributi cercano di fornire una comprensione migliore degli elementi chiave e dei meccanismi interni che hanno consentito il miglioramento dell'algoritmo originale.XX Ciclo198

Symbolic regression of discontinuous and multivariate functions by Hyper-Volume Error Separation (HVES)

Symbolic regression is aimed at discovering mathematical expressions, in symbolic form, that fit a given sample of data points. While genetic programming (GP) constitutes a powerful tool for solving this class of problems, its effectiveness is still severely limited when the data sample requires different expressions in different regions of the input space - i.e., when the approximating function should be discontinuous. In this paper we present a new GP-based approach for symbolic regression of discontinuous functions in multivariate data-sets. We identify the portions of the input space that require different approximating functions by means of a new algorithm that we call hyper-volume error separation (HVES). To this end we run a preliminary GP evolution and partition the input space based on the error exhibited by the best individual across the data-set. Then we partition the data-set based on the partition of the input space and use each such partition for driving an independent, preliminary GP evolution. The populations resulting from such preliminary evolutions are finally merged and evolved again. We compared our approach to the standard GP search and to a GP search for discontinuous functions in univariate data-sets. Our results show that coupling HVES with GP is an effective approach and provides significant accuracy improvements while requiring less computational resources

Generación automática de modelos de pronóstico usando bloques funcionales y programación genética

En el marco de la predicción de series temporales, la Programación Genética ha tomado gran fuerza en los últimos años debido a su capacidad de deducir la ecuación y aquellos parámetros que mejor aproximan la relación entre la variable de salida y el conjunto de variables de entrada; sin embargo, al ser aplicada en la predicción de series de tiempo, aún presenta limitaciones en la incorporación de las componentes de ciclo, tendencia, estacionalidad y error; en el uso de aquellos rezagos de interés en todos los individuos durante el proceso de búsqueda; en la inclusión de los modelos bechmark de predicción de series de tiempo presentes en la literatura; y la redundancia de nodos (terminales y operadores) que no aportan a la aptitud del modelo. Para abordarlos, en este trabajo se modificaron: la estructura del algoritmo de programación genética original, la función de aptitud, los operadores de selección, intensificación, reproducción, mutación y cruce; además, fueron incorporadas las componentes de ciclo, tendencia, estacionalidad y error, a los bloques funcionales. Lo anterior permite la inclusión de las componentes de los modelos actuales de predicción de series de tiempo, la focalización de los individuos en regiones de interés durante el proceso de exploración, y la incorporación de conocimiento experto en la generación de la población inicial del algoritmo. Las modificaciones propuestas fueron implementadas en un prototipo en el lenguaje R, y validadas contra series de tiempo con ecuación de generación conocida (para verificar la capacidad de deducción de la ecuación a partir de los datos) y series benchmark de la literatura de predicción de series de tiempo, como son las series: AIRLINE, SUNSPOT, LYNX, INTERNET y POLLUTION. Los resultados obtenidos en términos de medidas de error comparados contra modelos ARIMA, SVM (Maquinas de vectores de soporte), MLP (perceptrones multicapa), NN (redes neuronales artificiales), DAN (redes neuronales de arquitectura dinamica) y el algoritmo original de programación genética, fueron mejores tanto en el entrenamiento como la predicción.Abstract: In the framework of time-series forecasting, Genetic Programming has taken great strength in recent years due to their ability to derive the equation and the parameters that best approximate the relationship between the output variable and the set of input variables; but when applied to the prediction time series, is still limited in the incorporation of cycle, trend, seasonality and error components; in the use of lags of interest in all individuals during the search process; in the inclusion of bechmark models of literature of time series forcasting, and the redundancy of nodes (terminals and operators) that do not contribute to the fitness of the model. To address them, in this work were modified the structure of the original genetic programming algorithm, the fitness function, selection operators, intensification, reproduction, mutation and crossover, in addition, it included cycle components, trend, seasonality and error, to the functional blocks. This allows the inclusion of components of current models of time series forecasting, the targeting of individuals in regions of interest during the exploration process, and the incorporation of expert knowledge in the generation of the initial population of the algorithm. The proposed changes were implemented in a prototype in the R language, and validated against time series generation equation with known (to verify deductibility of the equation from the data) and bechmarks of series of time series forecasting, such as the series: AIRLINE, SUNSPOT, LYNX, INTERNET and POLLUTION. The results in terms of error measures compared with ARIMA models, SVM (support vector machines), MLP (multilayer perceptron), NN (artificial neural network), DAN (Dynamic Architecture for Artificial Neural Networks) and original genetic programming algorithm, were both better training and prediction.Doctorad

Hypnotizability and dissociation in dietary restraint

The literature on hypnotizability and dissociation was reviewed in relation to patterns of clinical and non-clinical eating behaviours. Included was a theoretical interpretation of the significantly higher rates of hypnotizability and dissociation in certain eating disordered groups, such as bulimics, compared to age matched controls. However, non-clinical investigations between hypnotizability, dissociation, and problematic eating patterns were focused on in particular. Two principle hypotheses emerged from this review. First, the Socio-Hypnotic hypothesis (e.g. Groth-Marnat & Schumaker, 1990; Frasquilho & Oakley, 1997) suggested that hypnotic suggestibility may influence the internalisation of socio-cultural pressure to be thin. Second, the Dissociative Escape Hypothesis proposes that a vulnerability to experience dissociative phenomena as a potential defence mechanism may lead individuals to disinhibit eating behaviour when faced with aversive cognitions (based on Heatherton & Baumeister, 1991, McManus, 1995; Frasquilho & Oakley, 1997). These hypotheses were framed in terms of a social-cultural model of problematic eating (Stice, 1994). After reviewing the concepts behind the definitions and measurements of hypnotizability, different types of dissociation, dietary restraint, and disinhibited eating, four studies sought to explore associations between these factors within a non-clinical female college student population. Study 1 (n = 40) examined the relationship between types of restraint and non-hypnotic suggestions relating to imagining body size increase and decrease, in the context of body related anxiety, disinhibition of eating, and imagery based suggestibility. Study 2 (n = 87) and Study 3 (n = 123) used correlational and regression techniques to examine relationships between a widely used test of hypnotizability and tests of cognitive and affective dissociation in relation to a number of dietary restraint and disinhibition of eating measures. Study 4 (n = 50) examined differences between restrainers and non-restrainers on a behavioural index of body shape and food concerns based on a Stroop-type paradigm and examined these concerns in relation to dissociation and hypnotizability. The Socio-Hypnotic hypothesis was weakly supported by correlation evidence, though regression analyses revealed more complex effects. While correlations involving dissociation needed to control for depression, different types of dissociation may relate to different features of both disinhibited eating and dietary restraint. These results were summarised and discussed in the final chapter and future research possibilities examined