3,488 research outputs found
A Comparison between Deep Neural Nets and Kernel Acoustic Models for Speech Recognition
We study large-scale kernel methods for acoustic modeling and compare to DNNs
on performance metrics related to both acoustic modeling and recognition.
Measuring perplexity and frame-level classification accuracy, kernel-based
acoustic models are as effective as their DNN counterparts. However, on
token-error-rates DNN models can be significantly better. We have discovered
that this might be attributed to DNN's unique strength in reducing both the
perplexity and the entropy of the predicted posterior probabilities. Motivated
by our findings, we propose a new technique, entropy regularized perplexity,
for model selection. This technique can noticeably improve the recognition
performance of both types of models, and reduces the gap between them. While
effective on Broadcast News, this technique could be also applicable to other
tasks.Comment: arXiv admin note: text overlap with arXiv:1411.400
Computation in generalised probabilistic theories
From the existence of an efficient quantum algorithm for factoring, it is
likely that quantum computation is intrinsically more powerful than classical
computation. At present, the best upper bound known for the power of quantum
computation is that BQP is in AWPP. This work investigates limits on
computational power that are imposed by physical principles. To this end, we
define a circuit-based model of computation in a class of operationally-defined
theories more general than quantum theory, and ask: what is the minimal set of
physical assumptions under which the above inclusion still holds? We show that
given only an assumption of tomographic locality (roughly, that multipartite
states can be characterised by local measurements), efficient computations are
contained in AWPP. This inclusion still holds even without assuming a basic
notion of causality (where the notion is, roughly, that probabilities for
outcomes cannot depend on future measurement choices). Following Aaronson, we
extend the computational model by allowing post-selection on measurement
outcomes. Aaronson showed that the corresponding quantum complexity class is
equal to PP. Given only the assumption of tomographic locality, the inclusion
in PP still holds for post-selected computation in general theories. Thus in a
world with post-selection, quantum theory is optimal for computation in the
space of all general theories. We then consider if relativised complexity
results can be obtained for general theories. It is not clear how to define a
sensible notion of an oracle in the general framework that reduces to the
standard notion in the quantum case. Nevertheless, it is possible to define
computation relative to a `classical oracle'. Then, we show there exists a
classical oracle relative to which efficient computation in any theory
satisfying the causality assumption and tomographic locality does not include
NP.Comment: 14+9 pages. Comments welcom
Cutset Sampling for Bayesian Networks
The paper presents a new sampling methodology for Bayesian networks that
samples only a subset of variables and applies exact inference to the rest.
Cutset sampling is a network structure-exploiting application of the
Rao-Blackwellisation principle to sampling in Bayesian networks. It improves
convergence by exploiting memory-based inference algorithms. It can also be
viewed as an anytime approximation of the exact cutset-conditioning algorithm
developed by Pearl. Cutset sampling can be implemented efficiently when the
sampled variables constitute a loop-cutset of the Bayesian network and, more
generally, when the induced width of the networks graph conditioned on the
observed sampled variables is bounded by a constant w. We demonstrate
empirically the benefit of this scheme on a range of benchmarks
Complexity Results and Approximation Strategies for MAP Explanations
MAP is the problem of finding a most probable instantiation of a set of
variables given evidence. MAP has always been perceived to be significantly
harder than the related problems of computing the probability of a variable
instantiation Pr, or the problem of computing the most probable explanation
(MPE). This paper investigates the complexity of MAP in Bayesian networks.
Specifically, we show that MAP is complete for NP^PP and provide further
negative complexity results for algorithms based on variable elimination. We
also show that MAP remains hard even when MPE and Pr become easy. For example,
we show that MAP is NP-complete when the networks are restricted to polytrees,
and even then can not be effectively approximated. Given the difficulty of
computing MAP exactly, and the difficulty of approximating MAP while providing
useful guarantees on the resulting approximation, we investigate best effort
approximations. We introduce a generic MAP approximation framework. We provide
two instantiations of the framework; one for networks which are amenable to
exact inference Pr, and one for networks for which even exact inference is too
hard. This allows MAP approximation on networks that are too complex to even
exactly solve the easier problems, Pr and MPE. Experimental results indicate
that using these approximation algorithms provides much better solutions than
standard techniques, and provide accurate MAP estimates in many cases
Complexity of Discrete Energy Minimization Problems
Discrete energy minimization is widely-used in computer vision and machine
learning for problems such as MAP inference in graphical models. The problem,
in general, is notoriously intractable, and finding the global optimal solution
is known to be NP-hard. However, is it possible to approximate this problem
with a reasonable ratio bound on the solution quality in polynomial time? We
show in this paper that the answer is no. Specifically, we show that general
energy minimization, even in the 2-label pairwise case, and planar energy
minimization with three or more labels are exp-APX-complete. This finding rules
out the existence of any approximation algorithm with a sub-exponential
approximation ratio in the input size for these two problems, including
constant factor approximations. Moreover, we collect and review the
computational complexity of several subclass problems and arrange them on a
complexity scale consisting of three major complexity classes -- PO, APX, and
exp-APX, corresponding to problems that are solvable, approximable, and
inapproximable in polynomial time. Problems in the first two complexity classes
can serve as alternative tractable formulations to the inapproximable ones.
This paper can help vision researchers to select an appropriate model for an
application or guide them in designing new algorithms.Comment: ECCV'16 accepte
- …