4 research outputs found
Information Theory and Noisy Computation
We report on two types of results. The first is a study of the rate of decay of information carried by a signal which is being propagated over a noisy channel. The second is a series of lower bounds on the depth, size, and component reliability of noisy logic circuits which are required to compute some function reliably. The arguments used for the circuit results are information-theoretic, and in particular, the signal decay result is essential to the depth lower bound. Our first result can be viewed as a quantified version of the data processing lemma, for the case of Boolean random variables
Exploration with Limited Memory: Streaming Algorithms for Coin Tossing, Noisy Comparisons, and Multi-Armed Bandits
Consider the following abstract coin tossing problem: Given a set of
coins with unknown biases, find the most biased coin using a minimal number of
coin tosses. This is a common abstraction of various exploration problems in
theoretical computer science and machine learning and has been studied
extensively over the years. In particular, algorithms with optimal sample
complexity (number of coin tosses) have been known for this problem for quite
some time.
Motivated by applications to processing massive datasets, we study the space
complexity of solving this problem with optimal number of coin tosses in the
streaming model. In this model, the coins are arriving one by one and the
algorithm is only allowed to store a limited number of coins at any point --
any coin not present in the memory is lost and can no longer be tossed or
compared to arriving coins. Prior algorithms for the coin tossing problem with
optimal sample complexity are based on iterative elimination of coins which
inherently require storing all the coins, leading to memory-inefficient
streaming algorithms.
We remedy this state-of-affairs by presenting a series of improved streaming
algorithms for this problem: we start with a simple algorithm which require
storing only coins and then iteratively refine it further and
further, leading to algorithms with memory,
memory, and finally a one that only stores a single extra coin in memory -- the
same exact space needed to just store the best coin throughout the stream.
Furthermore, we extend our algorithms to the problem of finding the most
biased coins as well as other exploration problems such as finding top-
elements using noisy comparisons or finding an -best arm in
stochastic multi-armed bandits, and obtain efficient streaming algorithms for
these problems
Energy-Efficient Circuit Design
We initiate the theoretical investigation of energy-efficient circuit design.
We assume that the circuit design specifies the circuit layout as well as the
supply voltages for the gates. To obtain maximum energy efficiency, the circuit
design must balance the conflicting demands of minimizing the energy used per gate,
and minimizing the number of gates in the circuit; If the energy supplied to the
gates is small, then functional failures are likely, necessitating a circuit layout
that is more fault-tolerant, and thus that has more gates.
By leveraging previous
work on fault-tolerant circuit design, we show general upper and lower bounds on
the amount of energy required by a circuit to compute a given relation. We show
that some circuits would be asymptotically more energy-efficient if heterogeneous
supply voltages were allowed, and show that for some circuits the most energy-efficient
supply voltages are homogeneous over all gates.
In the traditional approach to circuit design the
supply voltages for each transistor/gate are set sufficiently high so that with
sufficiently high probability no transistor fails.
We show that if there is a better (in terms of worst-case relative error with respect to energy) method than the traditional approach
then ,
and thus there is a complexity theoretic obstacle to achieving energy savings with Near-Threshold computing.
We show that almost all
Boolean functions require circuits that use exponential energy. This is not an immediate
consequence of Shannon's classic result that most functions require exponential
sized circuits of faultless gates because, as we show, the same circuit layout can
compute many different functions, depending on the value of the supply voltage.
If the error bound must vanish as the number of inputs increases, we show that a natural class of functions can be computed with asymptotically less energy using heterogeneous supply voltages than is possible using homogeneous supply voltages.
We also prove upper bounds on the asymptotic energy savings achieved by using heterogeneous supply voltages over homogeneous supply voltages for a class of functions, and also show a relation that can bypass this bound