8,736 research outputs found
A Multi-objective Exploratory Procedure for Regression Model Selection
Variable selection is recognized as one of the most critical steps in
statistical modeling. The problems encountered in engineering and social
sciences are commonly characterized by over-abundance of explanatory variables,
non-linearities and unknown interdependencies between the regressors. An added
difficulty is that the analysts may have little or no prior knowledge on the
relative importance of the variables. To provide a robust method for model
selection, this paper introduces the Multi-objective Genetic Algorithm for
Variable Selection (MOGA-VS) that provides the user with an optimal set of
regression models for a given data-set. The algorithm considers the regression
problem as a two objective task, and explores the Pareto-optimal (best subset)
models by preferring those models over the other which have less number of
regression coefficients and better goodness of fit. The model exploration can
be performed based on in-sample or generalization error minimization. The model
selection is proposed to be performed in two steps. First, we generate the
frontier of Pareto-optimal regression models by eliminating the dominated
models without any user intervention. Second, a decision making process is
executed which allows the user to choose the most preferred model using
visualisations and simple metrics. The method has been evaluated on a recently
published real dataset on Communities and Crime within United States.Comment: in Journal of Computational and Graphical Statistics, Vol. 24, Iss.
1, 201
The Kalai-Smorodinski solution for many-objective Bayesian optimization
An ongoing aim of research in multiobjective Bayesian optimization is to
extend its applicability to a large number of objectives. While coping with a
limited budget of evaluations, recovering the set of optimal compromise
solutions generally requires numerous observations and is less interpretable
since this set tends to grow larger with the number of objectives. We thus
propose to focus on a specific solution originating from game theory, the
Kalai-Smorodinsky solution, which possesses attractive properties. In
particular, it ensures equal marginal gains over all objectives. We further
make it insensitive to a monotonic transformation of the objectives by
considering the objectives in the copula space. A novel tailored algorithm is
proposed to search for the solution, in the form of a Bayesian optimization
algorithm: sequential sampling decisions are made based on acquisition
functions that derive from an instrumental Gaussian process prior. Our approach
is tested on four problems with respectively four, six, eight, and nine
objectives. The method is available in the Rpackage GPGame available on CRAN at
https://cran.r-project.org/package=GPGame
A generic optimising feature extraction method using multiobjective genetic programming
In this paper, we present a generic, optimising feature extraction method using multiobjective genetic programming. We re-examine the feature extraction problem and show that effective feature extraction can significantly enhance the performance of pattern recognition systems with simple classifiers. A framework is presented to evolve optimised feature extractors that transform an input pattern space into a decision space in which maximal class separability is obtained. We have applied this method to real world datasets from the UCI Machine Learning and StatLog databases to verify our approach and compare our proposed method with other reported results. We conclude that our algorithm is able to produce classifiers of superior (or equivalent) performance to the conventional classifiers examined, suggesting removal of the need to exhaustively evaluate a large family of conventional classifiers on any new problem. (C) 2010 Elsevier B.V. All rights reserved
- …