27,073 research outputs found
Joint M-Best-Diverse Labelings as a Parametric Submodular Minimization
We consider the problem of jointly inferring the M-best diverse labelings for
a binary (high-order) submodular energy of a graphical model. Recently, it was
shown that this problem can be solved to a global optimum, for many practically
interesting diversity measures. It was noted that the labelings are, so-called,
nested. This nestedness property also holds for labelings of a class of
parametric submodular minimization problems, where different values of the
global parameter give rise to different solutions. The popular example
of the parametric submodular minimization is the monotonic parametric max-flow
problem, which is also widely used for computing multiple labelings. As the
main contribution of this work we establish a close relationship between
diversity with submodular energies and the parametric submodular minimization.
In particular, the joint M-best diverse labelings can be obtained by running a
non-parametric submodular minimization (in the special case - max-flow) solver
for M different values of in parallel, for certain diversity measures.
Importantly, the values for can be computed in a closed form in
advance, prior to any optimization. These theoretical results suggest two
simple yet efficient algorithms for the joint M-best diverse problem, which
outperform competitors in terms of runtime and quality of results. In
particular, as we show in the paper, the new methods compute the exact M-best
diverse labelings faster than a popular method of Batra et al., which in some
sense only obtains approximate solutions
An Adaptive Markov Random Field for Structured Compressive Sensing
Exploiting intrinsic structures in sparse signals underpins the recent
progress in compressive sensing (CS). The key for exploiting such structures is
to achieve two desirable properties: generality (\ie, the ability to fit a wide
range of signals with diverse structures) and adaptability (\ie, being adaptive
to a specific signal). Most existing approaches, however, often only achieve
one of these two properties. In this study, we propose a novel adaptive Markov
random field sparsity prior for CS, which not only is able to capture a broad
range of sparsity structures, but also can adapt to each sparse signal through
refining the parameters of the sparsity prior with respect to the compressed
measurements. To maximize the adaptability, we also propose a new sparse signal
estimation where the sparse signals, support, noise and signal parameter
estimation are unified into a variational optimization problem, which can be
effectively solved with an alternative minimization scheme. Extensive
experiments on three real-world datasets demonstrate the effectiveness of the
proposed method in recovery accuracy, noise tolerance, and runtime.Comment: 13 pages, submitted to IEEE Transactions on Image Processin
PTAS for MAP Assignment on Pairwise Markov Random Fields in Planar Graphs
We present a PTAS for computing the maximum a posteriori assignment on
Pairwise Markov Random Fields with non-negative weights in planar graphs. This
algorithm is practical and not far behind state-of-the-art techniques in image
processing. MAP on Pairwise Markov Random Fields with (possibly) negative
weights cannot be approximated unless P = NP, even on planar graphs. We also
show via reduction that this yields a PTAS for one scoring function of
Correlation Clustering in planar graphs
Submodular meets Structured: Finding Diverse Subsets in Exponentially-Large Structured Item Sets
To cope with the high level of ambiguity faced in domains such as Computer
Vision or Natural Language processing, robust prediction methods often search
for a diverse set of high-quality candidate solutions or proposals. In
structured prediction problems, this becomes a daunting task, as the solution
space (image labelings, sentence parses, etc.) is exponentially large. We study
greedy algorithms for finding a diverse subset of solutions in
structured-output spaces by drawing new connections between submodular
functions over combinatorial item sets and High-Order Potentials (HOPs) studied
for graphical models. Specifically, we show via examples that when marginal
gains of submodular diversity functions allow structured representations, this
enables efficient (sub-linear time) approximate maximization by reducing the
greedy augmentation step to inference in a factor graph with appropriately
constructed HOPs. We discuss benefits, tradeoffs, and show that our
constructions lead to significantly better proposals
Collaborative filtering via sparse Markov random fields
Recommender systems play a central role in providing individualized access to
information and services. This paper focuses on collaborative filtering, an
approach that exploits the shared structure among mind-liked users and similar
items. In particular, we focus on a formal probabilistic framework known as
Markov random fields (MRF). We address the open problem of structure learning
and introduce a sparsity-inducing algorithm to automatically estimate the
interaction structures between users and between items. Item-item and user-user
correlation networks are obtained as a by-product. Large-scale experiments on
movie recommendation and date matching datasets demonstrate the power of the
proposed method
Hinge-Loss Markov Random Fields and Probabilistic Soft Logic
A fundamental challenge in developing high-impact machine learning
technologies is balancing the need to model rich, structured domains with the
ability to scale to big data. Many important problem areas are both richly
structured and large scale, from social and biological networks, to knowledge
graphs and the Web, to images, video, and natural language. In this paper, we
introduce two new formalisms for modeling structured data, and show that they
can both capture rich structure and scale to big data. The first, hinge-loss
Markov random fields (HL-MRFs), is a new kind of probabilistic graphical model
that generalizes different approaches to convex inference. We unite three
approaches from the randomized algorithms, probabilistic graphical models, and
fuzzy logic communities, showing that all three lead to the same inference
objective. We then define HL-MRFs by generalizing this unified objective. The
second new formalism, probabilistic soft logic (PSL), is a probabilistic
programming language that makes HL-MRFs easy to define using a syntax based on
first-order logic. We introduce an algorithm for inferring most-probable
variable assignments (MAP inference) that is much more scalable than
general-purpose convex optimization methods, because it uses message passing to
take advantage of sparse dependency structures. We then show how to learn the
parameters of HL-MRFs. The learned HL-MRFs are as accurate as analogous
discrete models, but much more scalable. Together, these algorithms enable
HL-MRFs and PSL to model rich, structured data at scales not previously
possible
Interactions of Computational Complexity Theory and Mathematics
[This paper is a (self contained) chapter in a new book, Mathematics and
Computation, whose draft is available on my homepage at
https://www.math.ias.edu/avi/book ].
We survey some concrete interaction areas between computational complexity
theory and different fields of mathematics. We hope to demonstrate here that
hardly any area of modern mathematics is untouched by the computational
connection (which in some cases is completely natural and in others may seem
quite surprising). In my view, the breadth, depth, beauty and novelty of these
connections is inspiring, and speaks to a great potential of future
interactions (which indeed, are quickly expanding). We aim for variety. We give
short, simple descriptions (without proofs or much technical detail) of ideas,
motivations, results and connections; this will hopefully entice the reader to
dig deeper. Each vignette focuses only on a single topic within a large
mathematical filed. We cover the following:
Number Theory: Primality testing
Combinatorial Geometry: Point-line incidences
Operator Theory: The Kadison-Singer problem
Metric Geometry: Distortion of embeddings
Group Theory: Generation and random generation
Statistical Physics: Monte-Carlo Markov chains
Analysis and Probability: Noise stability
Lattice Theory: Short vectors
Invariant Theory: Actions on matrix tuplesComment: 27 page
Learning Bayesian Networks from Incomplete Data with Stochastic Search Algorithms
This paper describes stochastic search approaches, including a new stochastic
algorithm and an adaptive mutation operator, for learning Bayesian networks
from incomplete data. This problem is characterized by a huge solution space
with a highly multimodal landscape. State-of-the-art approaches all involve
using deterministic approaches such as the expectation-maximization algorithm.
These approaches are guaranteed to find local maxima, but do not explore the
landscape for other modes. Our approach evolves structure and the missing data.
We compare our stochastic algorithms and show they all produce accurate
results.Comment: Appears in Proceedings of the Fifteenth Conference on Uncertainty in
Artificial Intelligence (UAI1999
Noisy Parallel Approximate Decoding for Conditional Recurrent Language Model
Recent advances in conditional recurrent language modelling have mainly
focused on network architectures (e.g., attention mechanism), learning
algorithms (e.g., scheduled sampling and sequence-level training) and novel
applications (e.g., image/video description generation, speech recognition,
etc.) On the other hand, we notice that decoding algorithms/strategies have not
been investigated as much, and it has become standard to use greedy or beam
search. In this paper, we propose a novel decoding strategy motivated by an
earlier observation that nonlinear hidden layers of a deep neural network
stretch the data manifold. The proposed strategy is embarrassingly
parallelizable without any communication overhead, while improving an existing
decoding algorithm. We extensively evaluate it with attention-based neural
machine translation on the task of En->Cz translation
Simulating outcomes of interventions using a multipurpose simulation program based on the Evolutionary Causal Matrices and Markov Chain
Predicting long-term outcomes of interventions is necessary for educational
and social policy-making processes that might widely influence our society for
the long-term. However, performing such predictions based on data from
large-scale experiments might be challenging due to the lack of time and
resources. In order to address this issue, computer simulations based on
Evolutionary Causal Matrices and Markov Chain can be used to predict long-term
outcomes with relatively small-scale lab data. In this paper, we introduce
Python classes implementing a computer simulation model and presented some
pilots implementations demonstrating how the model can be utilized for
predicting outcomes of diverse interventions. We also introduce the
class-structured simulation module both with real experimental data and with
hypothetical data formulated based on social psychological theories. Classes
developed and tested in the present study provide researchers and practitioners
with a feasible and practical method to simulate intervention outcomes
prospectively
- β¦