Dispersion for Data-Driven Algorithm Design, Online Learning, and Private Optimization

Data-driven algorithm design, that is, choosing the best algorithm for a specific application, is a crucial problem in modern data science. Practitioners often optimize over a parameterized algorithm family, tuning parameters based on problems from their domain. These procedures have historically come with no guarantees, though a recent line of work studies algorithm selection from a theoretical perspective. We advance the foundations of this field in several directions: we analyze online algorithm selection, where problems arrive one-by-one and the goal is to minimize regret, and private algorithm selection, where the goal is to find good parameters over a set of problems without revealing sensitive information contained therein. We study important algorithm families, including SDP-rounding schemes for problems formulated as integer quadratic programs, and greedy techniques for canonical subset selection problems. In these cases, the algorithm's performance is a volatile and piecewise Lipschitz function of its parameters, since tweaking the parameters can completely change the algorithm's behavior. We give a sufficient and general condition, dispersion, defining a family of piecewise Lipschitz functions that can be optimized online and privately, which includes the functions measuring the performance of the algorithms we study. Intuitively, a set of piecewise Lipschitz functions is dispersed if no small region contains many of the functions' discontinuities. We present general techniques for online and private optimization of the sum of dispersed piecewise Lipschitz functions. We improve over the best-known regret bounds for a variety of problems, prove regret bounds for problems not previously studied, and give matching lower bounds. We also give matching upper and lower bounds on the utility loss due to privacy. Moreover, we uncover dispersion in auction design and pricing problems

The Effect of Quadrature Errors in the Computation of L^2 Piecewise Polynomial Approximations

In this paper we investigate the L^2 piecewise polynomial approximation problem. L^2 bounds for the derivatives of the error in approximating sufficiently smooth functions by polynomial splines follow immediately from the analogous results for polynomial spline interpolation. We derive L^2 bounds for the errors introduced by the use of two types of quadrature rules for the numerical computation of L^2 piecewise polynomial approximations. These bounds enable us to present some asymptotic results and to examine the consistent convergence of appropriately chosen sequences of such approximations. Some numerical results are also included

Development and Analysis of Deterministic Privacy-Preserving Policies Using Non-Stochastic Information Theory

A deterministic privacy metric using non-stochastic information theory is developed. Particularly, minimax information is used to construct a measure of information leakage, which is inversely proportional to the measure of privacy. Anyone can submit a query to a trusted agent with access to a non-stochastic uncertain private dataset. Optimal deterministic privacy-preserving policies for responding to the submitted query are computed by maximizing the measure of privacy subject to a constraint on the worst-case quality of the response (i.e., the worst-case difference between the response by the agent and the output of the query computed on the private dataset). The optimal privacy-preserving policy is proved to be a piecewise constant function in the form of a quantization operator applied on the output of the submitted query. The measure of privacy is also used to analyze the performance of $k$-anonymity methodology (a popular deterministic mechanism for privacy-preserving release of datasets using suppression and generalization techniques), proving that it is in fact not privacy-preserving.Comment: improved introduction and numerical exampl

Effect of Chaotic Noise on Multistable Systems

In a recent letter [Phys.Rev.Lett. {\bf 30}, 3269 (1995), chao-dyn/9510011], we reported that a macroscopic chaotic determinism emerges in a multistable system: the unidirectional motion of a dissipative particle subject to an apparently symmetric chaotic noise occurs even if the particle is in a spatially symmetric potential. In this paper, we study the global dynamics of a dissipative particle by investigating the barrier crossing probability of the particle between two basins of the multistable potential. We derive analytically an expression of the barrier crossing probability of the particle subject to a chaotic noise generated by a general piecewise linear map. We also show that the obtained analytical barrier crossing probability is applicable to a chaotic noise generated not only by a piecewise linear map with a uniform invariant density but also by a non-piecewise linear map with non-uniform invariant density. We claim, from the viewpoint of the noise induced motion in a multistable system, that chaotic noise is a first realization of the effect of {\em dynamical asymmetry} of general noise which induces the symmetry breaking dynamics.Comment: 14 pages, 9 figures, to appear in Phys.Rev.

Stabilised finite element methods for ill-posed problems with conditional stability

In this paper we discuss the adjoint stabilised finite element method introduced in, E. Burman, Stabilized finite element methods for nonsymmetric, noncoercive and ill-posed problems. Part I: elliptic equations, SIAM Journal on Scientific Computing, and how it may be used for the computation of solutions to problems for which the standard stability theory given by the Lax-Milgram Lemma or the Babuska-Brezzi Theorem fails. We pay particular attention to ill-posed problems that have some conditional stability property and prove (conditional) error estimates in an abstract framework. As a model problem we consider the elliptic Cauchy problem and provide a complete numerical analysis for this case. Some numerical examples are given to illustrate the theory.Comment: Accepted in the proceedings from the EPSRC Durham Symposium Building Bridges: Connections and Challenges in Modern Approaches to Numerical Partial Differential Equation