186 research outputs found
Tight Lower Bounds for Greedy Routing in Higher-Dimensional Small-World Grids
We consider Kleinberg's celebrated small world graph model (Kleinberg, 2000),
in which a D-dimensional grid {0,...,n-1}^D is augmented with a constant number
of additional unidirectional edges leaving each node. These long range edges
are determined at random according to a probability distribution (the
augmenting distribution), which is the same for each node. Kleinberg suggested
using the inverse D-th power distribution, in which node v is the long range
contact of node u with a probability proportional to ||u-v||^(-D). He showed
that such an augmenting distribution allows to route a message efficiently in
the resulting random graph: The greedy algorithm, where in each intermediate
node the message travels over a link that brings the message closest to the
target w.r.t. the Manhattan distance, finds a path of expected length O(log^2
n) between any two nodes. In this paper we prove that greedy routing does not
perform asymptotically better for any uniform and isotropic augmenting
distribution, i.e., the probability that node u has a particular long range
contact v is independent of the labels of u and v and only a function of
||u-v||.
In order to obtain the result, we introduce a novel proof technique: We
define a budget game, in which a token travels over a game board, while the
player manages a "probability budget". In each round, the player bets part of
her remaining probability budget on step sizes. A step size is chosen at random
according to a probability distribution of the player's bet. The token then
makes progress as determined by the chosen step size, while some of the
player's bet is removed from her probability budget. We prove a tight lower
bound for such a budget game, and then obtain a lower bound for greedy routing
in the D-dimensional grid by a reduction
The Right Mutation Strength for Multi-Valued Decision Variables
The most common representation in evolutionary computation are bit strings.
This is ideal to model binary decision variables, but less useful for variables
taking more values. With very little theoretical work existing on how to use
evolutionary algorithms for such optimization problems, we study the run time
of simple evolutionary algorithms on some OneMax-like functions defined over
. More precisely, we regard a variety of
problem classes requesting the component-wise minimization of the distance to
an unknown target vector . For such problems we see a crucial
difference in how we extend the standard-bit mutation operator to these
multi-valued domains. While it is natural to select each position of the
solution vector to be changed independently with probability , there are
various ways to then change such a position. If we change each selected
position to a random value different from the original one, we obtain an
expected run time of . If we change each selected position
by either or (random choice), the optimization time reduces to
. If we use a random mutation strength with probability inversely proportional to and change
the selected position by either or (random choice), then the
optimization time becomes , bringing down
the dependence on from linear to polylogarithmic. One of our results
depends on a new variant of the lower bounding multiplicative drift theorem.Comment: an extended abstract of this work is to appear at GECCO 201
Asymptotic Bias of Stochastic Gradient Search
The asymptotic behavior of the stochastic gradient algorithm with a biased
gradient estimator is analyzed. Relying on arguments based on the dynamic
system theory (chain-recurrence) and the differential geometry (Yomdin theorem
and Lojasiewicz inequality), tight bounds on the asymptotic bias of the
iterates generated by such an algorithm are derived. The obtained results hold
under mild conditions and cover a broad class of high-dimensional nonlinear
algorithms. Using these results, the asymptotic properties of the
policy-gradient (reinforcement) learning and adaptive population Monte Carlo
sampling are studied. Relying on the same results, the asymptotic behavior of
the recursive maximum split-likelihood estimation in hidden Markov models is
analyzed, too.Comment: arXiv admin note: text overlap with arXiv:0907.102
More Haste, Less Waste: Lowering the Redundancy in Fully Indexable Dictionaries
We consider the problem of representing, in a compressed format, a bit-vector
of bits with 1s, supporting the following operations, where : returns the number of occurrences of bit in the
prefix ; returns the position of the th occurrence
of bit in . Such a data structure is called \emph{fully indexable
dictionary (FID)} [Raman et al.,2007], and is at least as powerful as
predecessor data structures. Our focus is on space-efficient FIDs on the
\textsc{ram} model with word size and constant time for all
operations, so that the time cost is independent of the input size. Given the
bitstring to be encoded, having length and containing ones, the
minimal amount of information that needs to be stored is . The state of the art in building a FID for is
given in [Patrascu,2008] using
bits, to support the operations in time. Here, we propose a parametric
data structure exhibiting a time/space trade-off such that, for any real
constants , it
uses B(n,m) + O(n^{1+\delta} + n (\frac{m}{n^s})^\eps) bits and performs
all the operations in time O(s\delta^{-1} + \eps^{-1}). The improvement is
twofold: our redundancy can be lowered parametrically and, fixing ,
we get a constant-time FID whose space is B(n,m) + O(m^\eps/\poly{n}) bits,
for sufficiently large . This is a significant improvement compared to the
previous bounds for the general case
Convergence Rate of Stochastic Gradient Search in the Case of Multiple and Non-Isolated Minima
The convergence rate of stochastic gradient search is analyzed in this paper.
Using arguments based on differential geometry and Lojasiewicz inequalities,
tight bounds on the convergence rate of general stochastic gradient algorithms
are derived. As opposed to the existing results, the results presented in this
paper allow the objective function to have multiple, non-isolated minima,
impose no restriction on the values of the Hessian (of the objective function)
and do not require the algorithm estimates to have a single limit point.
Applying these new results, the convergence rate of recursive prediction error
identification algorithms is studied. The convergence rate of supervised and
temporal-difference learning algorithms is also analyzed using the results
derived in the paper
Convergence and Convergence Rate of Stochastic Gradient Search in the Case of Multiple and Non-Isolated Extrema
The asymptotic behavior of stochastic gradient algorithms is studied. Relying
on results from differential geometry (Lojasiewicz gradient inequality), the
single limit-point convergence of the algorithm iterates is demonstrated and
relatively tight bounds on the convergence rate are derived. In sharp contrast
to the existing asymptotic results, the new results presented here allow the
objective function to have multiple and non-isolated minima. The new results
also offer new insights into the asymptotic properties of several classes of
recursive algorithms which are routinely used in engineering, statistics,
machine learning and operations research
Settling for limited privacy: how much does it help?
This thesis explores practical and theoretical aspects of several privacy-providing technologies, including tools for anonymous web-browsing, verifiable electronic voting schemes, and private information retrieval from databases. State-of-art privacy-providing schemes are frequently impractical for implementational reasons or for sheer information-theoretical reasons due to the amount of information that needs to be transmitted. We have been researching the question of whether relaxing the requirements on such schemes, in particular settling for imperfect but sufficient in real-world situations privacy, as opposed to perfect privacy, may be helpful in producing more practical or more efficient schemes. This thesis presents three results. The first result is the introduction of caching as a technique for providing anonymous web-browsing at the cost of sacrificing some functionality provided by anonymizing systems that do not use caching. The second result is a coercion-resistant electronic voting scheme with nearly perfect privacy and nearly perfect voter verifiability. The third result consists of some lower bounds and some simple upper bounds on the amount of communication in nearly private information retrieval schemes; our work is the first in-depth exploration of private information schemes with imperfect privacy
- …