7,986 research outputs found
Recommended from our members
Optimal exact designs of experiments via Mixed Integer Nonlinear Programming
Optimal exact designs are problematic to find and study because there is no unified theory for determining them and studyingtheir properties. Each has its own challenges and when a method exists to confirm the design optimality, it is invariablyapplicable to the particular problem only.We propose a systematic approach to construct optimal exact designs by incorporatingthe Cholesky decomposition of the Fisher Information Matrix in a Mixed Integer Nonlinear Programming formulation. Asexamples, we apply the methodology to find D- and A-optimal exact designs for linear and nonlinear models using global orlocal optimizers. Our examples include design problems with constraints on the locations or the number of replicates at theoptimal design points
Recommended from our members
A comparison of general-purpose optimization algorithms forfinding optimal approximate experimental designs
Several common general purpose optimization algorithms are compared for findingA- and D-optimal designs for different types of statistical models of varying complexity,including high dimensional models with five and more factors. The algorithms of interestinclude exact methods, such as the interior point method, the Nelder–Mead method, theactive set method, the sequential quadratic programming, and metaheuristic algorithms,such as particle swarm optimization, simulated annealing and genetic algorithms.Several simulations are performed, which provide general recommendations on theutility and performance of each method, including hybridized versions of metaheuristicalgorithms for finding optimal experimental designs. A key result is that general-purposeoptimization algorithms, both exact methods and metaheuristic algorithms, perform wellfor finding optimal approximate experimental designs
Communications-Inspired Projection Design with Application to Compressive Sensing
We consider the recovery of an underlying signal x \in C^m based on
projection measurements of the form y=Mx+w, where y \in C^l and w is
measurement noise; we are interested in the case l < m. It is assumed that the
signal model p(x) is known, and w CN(w;0,S_w), for known S_W. The objective is
to design a projection matrix M \in C^(l x m) to maximize key
information-theoretic quantities with operational significance, including the
mutual information between the signal and the projections I(x;y) or the Renyi
entropy of the projections h_a(y) (Shannon entropy is a special case). By
capitalizing on explicit characterizations of the gradients of the information
measures with respect to the projections matrix, where we also partially extend
the well-known results of Palomar and Verdu from the mutual information to the
Renyi entropy domain, we unveil the key operations carried out by the optimal
projections designs: mode exposure and mode alignment. Experiments are
considered for the case of compressive sensing (CS) applied to imagery. In this
context, we provide a demonstration of the performance improvement possible
through the application of the novel projection designs in relation to
conventional ones, as well as justification for a fast online projections
design method with which state-of-the-art adaptive CS signal recovery is
achieved.Comment: 25 pages, 7 figures, parts of material published in IEEE ICASSP 2012,
submitted to SIIM
A-optimal designs for an additive quadratic mixture model
Quadratic models are widely used in the analysis of experiments involving mixtures. This paper gives A-optimal designs for an additive quadratic mixture model for q ≥ 3 mixture components. It is proved that in these A-optimal designs, vertices of the simplex S q-1 are support points, and other support points shift gradually from barycentres of depth 1 to barycentres of depth 3 as q increases. A-optimal designs with minimal support are also discussed.published_or_final_versio
Non-convex Optimization for Machine Learning
A vast majority of machine learning algorithms train their models and perform
inference by solving optimization problems. In order to capture the learning
and prediction problems accurately, structural constraints such as sparsity or
low rank are frequently imposed or else the objective itself is designed to be
a non-convex function. This is especially true of algorithms that operate in
high-dimensional spaces or that train non-linear models such as tensor models
and deep networks.
The freedom to express the learning problem as a non-convex optimization
problem gives immense modeling power to the algorithm designer, but often such
problems are NP-hard to solve. A popular workaround to this has been to relax
non-convex problems to convex ones and use traditional methods to solve the
(convex) relaxed optimization problems. However this approach may be lossy and
nevertheless presents significant challenges for large scale optimization.
On the other hand, direct approaches to non-convex optimization have met with
resounding success in several domains and remain the methods of choice for the
practitioner, as they frequently outperform relaxation-based techniques -
popular heuristics include projected gradient descent and alternating
minimization. However, these are often poorly understood in terms of their
convergence and other properties.
This monograph presents a selection of recent advances that bridge a
long-standing gap in our understanding of these heuristics. The monograph will
lead the reader through several widely used non-convex optimization techniques,
as well as applications thereof. The goal of this monograph is to both,
introduce the rich literature in this area, as well as equip the reader with
the tools and techniques needed to analyze these simple procedures for
non-convex problems.Comment: The official publication is available from now publishers via
http://dx.doi.org/10.1561/220000005
Sparse Signal Recovery under Poisson Statistics
We are motivated by problems that arise in a number of applications such as
Online Marketing and explosives detection, where the observations are usually
modeled using Poisson statistics. We model each observation as a Poisson random
variable whose mean is a sparse linear superposition of known patterns. Unlike
many conventional problems observations here are not identically distributed
since they are associated with different sensing modalities. We analyze the
performance of a Maximum Likelihood (ML) decoder, which for our Poisson setting
involves a non-linear optimization but yet is computationally tractable. We
derive fundamental sample complexity bounds for sparse recovery when the
measurements are contaminated with Poisson noise. In contrast to the
least-squares linear regression setting with Gaussian noise, we observe that in
addition to sparsity, the scale of the parameters also fundamentally impacts
sample complexity. We introduce a novel notion of Restricted Likelihood
Perturbation (RLP), to jointly account for scale and sparsity. We derive sample
complexity bounds for regularized ML estimators in terms of RLP and
further specialize these results for deterministic and random sensing matrix
designs.Comment: 13 pages, 11 figures, 2 tables, submitted to IEEE Transactions on
Signal Processin
Bayesian Design and Analysis of Small Multifactor Industrial Experiments
PhDUnreplicated two level fractional factorial designs are a common type of experimental
design used in the early stages of industrial experimentation. They allow considerable
information about the e ects of several factors on the response to be obtained with
a relatively small number of runs.
The aim of this thesis is to improve the guidance available to experimenters in choosing
a good design and analysing data. This is particularly important when there is
commercial pressure to minimise the size of the experiment.
A design is usually chosen based on optimality, either in terms of a variance criterion
or estimability criteria such as resolution. This is given the number of factors, number
of levels of each factor and number of runs available. A decision theory approach is
explored, which allows a more informed choice of design to be made. Prior distributions
on the sizes of e ects are taken into consideration, and then a design chosen
from a candidate set of designs using a utility function relevant to the objectives of
the experiment. Comparisons of the decision theoretic methods with simple rules of
thumb are made to determine when the more complex approach is necessary.
Fully Bayesian methods are rarely used in multifactor experiments. However there
is virtually always some prior knowledge about the sizes of e ects and so using this
in a Bayesian data analysis seems natural. Vague and more informative priors are
6
explored.
The analysis of this type of experiment can be impacted in a disastrous way in the
presence of outliers. An analysis that is robust to outliers is sought by applying di erent
model distributions of the data and prior assumptions on the parameters. Results
obtained are compared with those from standard analyses to assess the bene ts of
the Bayesian analysis
Speaker verification using sequence discriminant support vector machines
This paper presents a text-independent speaker verification system using support vector machines (SVMs) with score-space kernels. Score-space kernels generalize Fisher kernels and are based on underlying generative models such as Gaussian mixture models (GMMs). This approach provides direct discrimination between whole sequences, in contrast with the frame-level approaches at the heart of most current systems. The resultant SVMs have a very high dimensionality since it is related to the number of parameters in the underlying generative model. To address problems that arise in the resultant optimization we introduce a technique called spherical normalization that preconditions the Hessian matrix. We have performed speaker verification experiments using the PolyVar database. The SVM system presented here reduces the relative error rates by 34% compared to a GMM likelihood ratio system
- …