6,055 research outputs found
Faster Convex Optimization: Simulated Annealing with an Efficient Universal Barrier
This paper explores a surprising equivalence between two seemingly-distinct
convex optimization methods. We show that simulated annealing, a well-studied
random walk algorithms, is directly equivalent, in a certain sense, to the
central path interior point algorithm for the the entropic universal barrier
function. This connection exhibits several benefits. First, we are able improve
the state of the art time complexity for convex optimization under the
membership oracle model. We improve the analysis of the randomized algorithm of
Kalai and Vempala by utilizing tools developed by Nesterov and Nemirovskii that
underly the central path following interior point algorithm. We are able to
tighten the temperature schedule for simulated annealing which gives an
improved running time, reducing by square root of the dimension in certain
instances. Second, we get an efficient randomized interior point method with an
efficiently computable universal barrier for any convex set described by a
membership oracle. Previously, efficiently computable barriers were known only
for particular convex sets
Learning Convex Partitions and Computing Game-theoretic Equilibria from Best Response Queries
Suppose that an -simplex is partitioned into convex regions having
disjoint interiors and distinct labels, and we may learn the label of any point
by querying it. The learning objective is to know, for any point in the
simplex, a label that occurs within some distance from that point.
We present two algorithms for this task: Constant-Dimension Generalised Binary
Search (CD-GBS), which for constant uses queries, and Constant-Region Generalised Binary
Search (CR-GBS), which uses CD-GBS as a subroutine and for constant uses
queries.
We show via Kakutani's fixed-point theorem that these algorithms provide
bounds on the best-response query complexity of computing approximate
well-supported equilibria of bimatrix games in which one of the players has a
constant number of pure strategies. We also partially extend our results to
games with multiple players, establishing further query complexity bounds for
computing approximate well-supported equilibria in this setting.Comment: 38 pages, 7 figures, second version strengthens lower bound in
Theorem 6, adds footnotes with additional comments and fixes typo
Piecewise Linear Control Systems
This thesis treats analysis and design of piecewise linear control systems. Piecewise linear systems capture many of the most common nonlinearities in engineering systems, and they can also be used for approximation of other nonlinear systems. Several aspects of linear systems with quadratic constraints are generalized to piecewise linear systems with piecewise quadratic constraints. It is shown how uncertainty models for linear systems can be extended to piecewise linear systems, and how these extensions give insight into the classical trade-offs between fidelity and complexity of a model. Stability of piecewise linear systems is investigated using piecewise quadratic Lyapunov functions. Piecewise quadratic Lyapunov functions are much more powerful than the commonly used quadratic Lyapunov functions. It is shown how piecewise quadratic Lyapunov functions can be computed via convex optimization in terms of linear matrix inequalities. The computations are based on a compact parameterization of continuous piecewise quadratic functions and conditional analysis using the S-procedure. A unifying framework for computation of a variety of Lyapunov functions via convex optimization is established based on this parameterization. Systems with attractive sliding modes and systems with bounded regions of attraction are also treated. Dissipativity analysis and optimal control problems with piecewise quadratic cost functions are solved via convex optimization. The basic results are extended to fuzzy systems, hybrid systems and smooth nonlinear systems. It is shown how Lyapunov functions with a discontinuous dependence on the discrete state can be computed via convex optimization. An automated procedure for increasing the flexibility of the Lyapunov function candidate is suggested based on linear programming duality. A Matlab toolbox that implements several of the results derived in the thesis is presented
On Communication Protocols that Compute Almost Privately
A traditionally desired goal when designing auction mechanisms is incentive
compatibility, i.e., ensuring that bidders fare best by truthfully reporting
their preferences. A complementary goal, which has, thus far, received
significantly less attention, is to preserve privacy, i.e., to ensure that
bidders reveal no more information than necessary. We further investigate and
generalize the approximate privacy model for two-party communication recently
introduced by Feigenbaum et al.[8]. We explore the privacy properties of a
natural class of communication protocols that we refer to as "dissection
protocols". Dissection protocols include, among others, the bisection auction
in [9,10] and the bisection protocol for the millionaires problem in [8].
Informally, in a dissection protocol the communicating parties are restricted
to answering simple questions of the form "Is your input between the values
\alpha and \beta (under a predefined order over the possible inputs)?".
We prove that for a large class of functions, called tiling functions, which
include the 2nd-price Vickrey auction, there always exists a dissection
protocol that provides a constant average-case privacy approximation ratio for
uniform or "almost uniform" probability distributions over inputs. To establish
this result we present an interesting connection between the approximate
privacy framework and basic concepts in computational geometry. We show that
such a good privacy approximation ratio for tiling functions does not, in
general, exist in the worst case. We also discuss extensions of the basic setup
to more than two parties and to non-tiling functions, and provide calculations
of privacy approximation ratios for two functions of interest.Comment: to appear in Theoretical Computer Science (series A
Training Gaussian Mixture Models at Scale via Coresets
How can we train a statistical mixture model on a massive data set? In this
work we show how to construct coresets for mixtures of Gaussians. A coreset is
a weighted subset of the data, which guarantees that models fitting the coreset
also provide a good fit for the original data set. We show that, perhaps
surprisingly, Gaussian mixtures admit coresets of size polynomial in dimension
and the number of mixture components, while being independent of the data set
size. Hence, one can harness computationally intensive algorithms to compute a
good approximation on a significantly smaller data set. More importantly, such
coresets can be efficiently constructed both in distributed and streaming
settings and do not impose restrictions on the data generating process. Our
results rely on a novel reduction of statistical estimation to problems in
computational geometry and new combinatorial complexity results for mixtures of
Gaussians. Empirical evaluation on several real-world datasets suggests that
our coreset-based approach enables significant reduction in training-time with
negligible approximation error
Projected and Hidden Markov Models for calculating kinetics and metastable states of complex molecules
Markov state models (MSMs) have been successful in computing metastable
states, slow relaxation timescales and associated structural changes, and
stationary or kinetic experimental observables of complex molecules from large
amounts of molecular dynamics simulation data. However, MSMs approximate the
true dynamics by assuming a Markov chain on a clusters discretization of the
state space. This approximation is difficult to make for high-dimensional
biomolecular systems, and the quality and reproducibility of MSMs has therefore
been limited. Here, we discard the assumption that dynamics are Markovian on
the discrete clusters. Instead, we only assume that the full phase- space
molecular dynamics is Markovian, and a projection of this full dynamics is
observed on the discrete states, leading to the concept of Projected Markov
Models (PMMs). Robust estimation methods for PMMs are not yet available, but we
derive a practically feasible approximation via Hidden Markov Models (HMMs). It
is shown how various molecular observables of interest that are often computed
from MSMs can be computed from HMMs / PMMs. The new framework is applicable to
both, simulation and single-molecule experimental data. We demonstrate its
versatility by applications to educative model systems, an 1 ms Anton MD
simulation of the BPTI protein, and an optical tweezer force probe trajectory
of an RNA hairpin
- âŠ