806 research outputs found
Dispersion for Data-Driven Algorithm Design, Online Learning, and Private Optimization
Data-driven algorithm design, that is, choosing the best algorithm for a
specific application, is a crucial problem in modern data science.
Practitioners often optimize over a parameterized algorithm family, tuning
parameters based on problems from their domain. These procedures have
historically come with no guarantees, though a recent line of work studies
algorithm selection from a theoretical perspective. We advance the foundations
of this field in several directions: we analyze online algorithm selection,
where problems arrive one-by-one and the goal is to minimize regret, and
private algorithm selection, where the goal is to find good parameters over a
set of problems without revealing sensitive information contained therein. We
study important algorithm families, including SDP-rounding schemes for problems
formulated as integer quadratic programs, and greedy techniques for canonical
subset selection problems. In these cases, the algorithm's performance is a
volatile and piecewise Lipschitz function of its parameters, since tweaking the
parameters can completely change the algorithm's behavior. We give a sufficient
and general condition, dispersion, defining a family of piecewise Lipschitz
functions that can be optimized online and privately, which includes the
functions measuring the performance of the algorithms we study. Intuitively, a
set of piecewise Lipschitz functions is dispersed if no small region contains
many of the functions' discontinuities. We present general techniques for
online and private optimization of the sum of dispersed piecewise Lipschitz
functions. We improve over the best-known regret bounds for a variety of
problems, prove regret bounds for problems not previously studied, and give
matching lower bounds. We also give matching upper and lower bounds on the
utility loss due to privacy. Moreover, we uncover dispersion in auction design
and pricing problems
Transmit without regrets: Online optimization in MIMO-OFDM cognitive radio systems
In this paper, we examine cognitive radio systems that evolve dynamically
over time due to changing user and environmental conditions. To combine the
advantages of orthogonal frequency division multiplexing (OFDM) and
multiple-input, multiple-output (MIMO) technologies, we consider a MIMO-OFDM
cognitive radio network where wireless users with multiple antennas communicate
over several non-interfering frequency bands. As the network's primary users
(PUs) come and go in the system, the communication environment changes
constantly (and, in many cases, randomly). Accordingly, the network's
unlicensed, secondary users (SUs) must adapt their transmit profiles "on the
fly" in order to maximize their data rate in a rapidly evolving environment
over which they have no control. In this dynamic setting, static solution
concepts (such as Nash equilibrium) are no longer relevant, so we focus on
dynamic transmit policies that lead to no regret: specifically, we consider
policies that perform at least as well as (and typically outperform) even the
best fixed transmit profile in hindsight. Drawing on the method of matrix
exponential learning and online mirror descent techniques, we derive a
no-regret transmit policy for the system's SUs which relies only on local
channel state information (CSI). Using this method, the system's SUs are able
to track their individually evolving optimum transmit profiles remarkably well,
even under rapidly (and randomly) changing conditions. Importantly, the
proposed augmented exponential learning (AXL) policy leads to no regret even if
the SUs' channel measurements are subject to arbitrarily large observation
errors (the imperfect CSI case), thus ensuring the method's robustness in the
presence of uncertainties.Comment: 25 pages, 3 figures, to appear in the IEEE Journal on Selected Areas
in Communication
- …