10,193 research outputs found
Query Learning with Exponential Query Costs
In query learning, the goal is to identify an unknown object while minimizing
the number of "yes" or "no" questions (queries) posed about that object. A
well-studied algorithm for query learning is known as generalized binary search
(GBS). We show that GBS is a greedy algorithm to optimize the expected number
of queries needed to identify the unknown object. We also generalize GBS in two
ways. First, we consider the case where the cost of querying grows
exponentially in the number of queries and the goal is to minimize the expected
exponential cost. Then, we consider the case where the objects are partitioned
into groups, and the objective is to identify only the group to which the
object belongs. We derive algorithms to address these issues in a common,
information-theoretic framework. In particular, we present an exact formula for
the objective function in each case involving Shannon or Renyi entropy, and
develop a greedy algorithm for minimizing it. Our algorithms are demonstrated
on two applications of query learning, active learning and emergency response.Comment: 15 page
Information and Decision Theoretic Approaches to Problems in Active Diagnosis.
In applications such as active learning or disease/fault diagnosis, one often encounters the problem of identifying an unknown object while minimizing the number of ``yes" or ``no" questions (queries) posed about that object. This problem has been commonly referred to as object/entity identification or active diagnosis in the literature. In this thesis, we consider several extensions of this fundamental problem that are motivated by practical considerations in real-world, time-critical identification tasks such as emergency response.
First, we consider the problem where the objects are partitioned into groups, and the goal is to identify only the group to which the object belongs. We then consider the case where the cost of identifying an object grows exponentially in the number of queries. To address these problems we show that a standard algorithm for object identification, known as the splitting algorithm or generalized binary search (GBS), may be viewed as a generalization of Shannon-Fano coding. We then extend this result to the group-based and the exponential cost settings, leading to new, improved algorithms.
We then study the problem of active diagnosis under persistent query noise. Previous work in this area either assumed that the noise is independent or that the underlying query noise distribution is completely known. We make no such assumptions, and introduce an algorithm that returns a ranked list of objects, such that the expected rank of the true object is optimized. Finally, we study the problem of active diagnosis where multiple objects are present, such as in disease/fault diagnosis. Current algorithms in this area have an exponential time complexity making them slow and intractable. We address this issue by proposing an extension of our rank-based approach to the multiple object scenario, where we optimize the area under the ROC curve of the rank-based output. The AUC criterion allows us to make a simplifying assumption that significantly reduces the complexity of active diagnosis (from exponential to near quadratic), with little or no compromise on the performance quality. Further, we demonstrate the performance of the proposed algorithms through extensive experiments on both synthetic and real world datasets.Ph.D.Electrical Engineering: SystemsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/91606/1/gowtham_1.pd
Single-Step Quantum Search Using Problem Structure
The structure of satisfiability problems is used to improve search algorithms
for quantum computers and reduce their required coherence times by using only a
single coherent evaluation of problem properties. The structure of random k-SAT
allows determining the asymptotic average behavior of these algorithms, showing
they improve on quantum algorithms, such as amplitude amplification, that
ignore detailed problem structure but remain exponential for hard problem
instances. Compared to good classical methods, the algorithm performs better,
on average, for weakly and highly constrained problems but worse for hard
cases. The analytic techniques introduced here also apply to other quantum
algorithms, supplementing the limited evaluation possible with classical
simulations and showing how quantum computing can use ensemble properties of NP
search problems.Comment: 39 pages, 12 figures. Revision describes further improvement with
multiple steps (section 7). See also
http://www.parc.xerox.com/dynamics/www/quantum.htm
Solving the Optimal Trading Trajectory Problem Using a Quantum Annealer
We solve a multi-period portfolio optimization problem using D-Wave Systems'
quantum annealer. We derive a formulation of the problem, discuss several
possible integer encoding schemes, and present numerical examples that show
high success rates. The formulation incorporates transaction costs (including
permanent and temporary market impact), and, significantly, the solution does
not require the inversion of a covariance matrix. The discrete multi-period
portfolio optimization problem we solve is significantly harder than the
continuous variable problem. We present insight into how results may be
improved using suitable software enhancements, and why current quantum
annealing technology limits the size of problem that can be successfully solved
today. The formulation presented is specifically designed to be scalable, with
the expectation that as quantum annealing technology improves, larger problems
will be solvable using the same techniques.Comment: 7 pages; expanded and update
Let's Make Block Coordinate Descent Go Fast: Faster Greedy Rules, Message-Passing, Active-Set Complexity, and Superlinear Convergence
Block coordinate descent (BCD) methods are widely-used for large-scale
numerical optimization because of their cheap iteration costs, low memory
requirements, amenability to parallelization, and ability to exploit problem
structure. Three main algorithmic choices influence the performance of BCD
methods: the block partitioning strategy, the block selection rule, and the
block update rule. In this paper we explore all three of these building blocks
and propose variations for each that can lead to significantly faster BCD
methods. We (i) propose new greedy block-selection strategies that guarantee
more progress per iteration than the Gauss-Southwell rule; (ii) explore
practical issues like how to implement the new rules when using "variable"
blocks; (iii) explore the use of message-passing to compute matrix or Newton
updates efficiently on huge blocks for problems with a sparse dependency
between variables; and (iv) consider optimal active manifold identification,
which leads to bounds on the "active set complexity" of BCD methods and leads
to superlinear convergence for certain problems with sparse solutions (and in
some cases finite termination at an optimal solution). We support all of our
findings with numerical results for the classic machine learning problems of
least squares, logistic regression, multi-class logistic regression, label
propagation, and L1-regularization
- …