26,307 research outputs found
Outlier robust system identification: a Bayesian kernel-based approach
In this paper, we propose an outlier-robust regularized kernel-based method
for linear system identification. The unknown impulse response is modeled as a
zero-mean Gaussian process whose covariance (kernel) is given by the recently
proposed stable spline kernel, which encodes information on regularity and
exponential stability. To build robustness to outliers, we model the
measurement noise as realizations of independent Laplacian random variables.
The identification problem is cast in a Bayesian framework, and solved by a new
Markov Chain Monte Carlo (MCMC) scheme. In particular, exploiting the
representation of the Laplacian random variables as scale mixtures of
Gaussians, we design a Gibbs sampler which quickly converges to the target
distribution. Numerical simulations show a substantial improvement in the
accuracy of the estimates over state-of-the-art kernel-based methods.Comment: 5 figure
Making Consensus Tractable
We study a model of consensus decision making, in which a finite group of
Bayesian agents has to choose between one of two courses of action. Each member
of the group has a private and independent signal at his or her disposal,
giving some indication as to which action is optimal. To come to a common
decision, the participants perform repeated rounds of voting. In each round,
each agent casts a vote in favor of one of the two courses of action,
reflecting his or her current belief, and observes the votes of the rest.
We provide an efficient algorithm for the calculation the agents have to
perform, and show that consensus is always reached and that the probability of
reaching a wrong decision decays exponentially with the number of agents.Comment: 18 pages. To appear in Transactions on Economics and Computatio
Consensus Message Passing for Layered Graphical Models
Generative models provide a powerful framework for probabilistic reasoning.
However, in many domains their use has been hampered by the practical
difficulties of inference. This is particularly the case in computer vision,
where models of the imaging process tend to be large, loopy and layered. For
this reason bottom-up conditional models have traditionally dominated in such
domains. We find that widely-used, general-purpose message passing inference
algorithms such as Expectation Propagation (EP) and Variational Message Passing
(VMP) fail on the simplest of vision models. With these models in mind, we
introduce a modification to message passing that learns to exploit their
layered structure by passing 'consensus' messages that guide inference towards
good solutions. Experiments on a variety of problems show that the proposed
technique leads to significantly more accurate inference results, not only when
compared to standard EP and VMP, but also when compared to competitive
bottom-up conditional models.Comment: Appearing in Proceedings of the 18th International Conference on
Artificial Intelligence and Statistics (AISTATS) 201
Constant Step Size Stochastic Gradient Descent for Probabilistic Modeling
Stochastic gradient methods enable learning probabilistic models from large
amounts of data. While large step-sizes (learning rates) have shown to be best
for least-squares (e.g., Gaussian noise) once combined with parameter
averaging, these are not leading to convergent algorithms in general. In this
paper, we consider generalized linear models, that is, conditional models based
on exponential families. We propose averaging moment parameters instead of
natural parameters for constant-step-size stochastic gradient descent. For
finite-dimensional models, we show that this can sometimes (and surprisingly)
lead to better predictions than the best linear model. For infinite-dimensional
models, we show that it always converges to optimal predictions, while
averaging natural parameters never does. We illustrate our findings with
simulations on synthetic data and classical benchmarks with many observations.Comment: Published in Proc. UAI 2018, was accepted as oral presentation Camera
read
Voting with preferences over margins of victory
This paper analyzes a two-alternative voting model with the distinctive feature that voters have preferences over margins of victory. We study voting contests with a finite as well as an infinite number of voters, and with and without mandatory voting. The main result of the paper is the existence and characterization of a unique equilibrium outcome in all those situations. At equilibrium, voters who prefer a larger support for one of the alternatives vote for such alternative. The model also provides a formal argument for the conditional sincerity voting condition in Alesina and Rosenthal (1995) and the benefit of voting function in Llavador (2006). Finally, we offer new insights on explaining why some citizens may vote strategically for an alternative different from the one declared as the most preferred.Margin of victory, plurality, abstention, strategic voting, committee voting, elections
Structural graph matching using the EM algorithm and singular value decomposition
This paper describes an efficient algorithm for inexact graph matching. The method is purely structural, that is, it uses only the edge or connectivity structure of the graph and does not draw on node or edge attributes. We make two contributions: 1) commencing from a probability distribution for matching errors, we show how the problem of graph matching can be posed as maximum-likelihood estimation using the apparatus of the EM algorithm; and 2) we cast the recovery of correspondence matches between the graph nodes in a matrix framework. This allows one to efficiently recover correspondence matches using the singular value decomposition. We experiment with the method on both real-world and synthetic data. Here, we demonstrate that the method offers comparable performance to more computationally demanding method
A Bayesian Approach to Discovering Truth from Conflicting Sources for Data Integration
In practical data integration systems, it is common for the data sources
being integrated to provide conflicting information about the same entity.
Consequently, a major challenge for data integration is to derive the most
complete and accurate integrated records from diverse and sometimes conflicting
sources. We term this challenge the truth finding problem. We observe that some
sources are generally more reliable than others, and therefore a good model of
source quality is the key to solving the truth finding problem. In this work,
we propose a probabilistic graphical model that can automatically infer true
records and source quality without any supervision. In contrast to previous
methods, our principled approach leverages a generative process of two types of
errors (false positive and false negative) by modeling two different aspects of
source quality. In so doing, ours is also the first approach designed to merge
multi-valued attribute types. Our method is scalable, due to an efficient
sampling-based inference algorithm that needs very few iterations in practice
and enjoys linear time complexity, with an even faster incremental variant.
Experiments on two real world datasets show that our new method outperforms
existing state-of-the-art approaches to the truth finding problem.Comment: VLDB201
A new kernel-based approach to system identification with quantized output data
In this paper we introduce a novel method for linear system identification
with quantized output data. We model the impulse response as a zero-mean
Gaussian process whose covariance (kernel) is given by the recently proposed
stable spline kernel, which encodes information on regularity and exponential
stability. This serves as a starting point to cast our system identification
problem into a Bayesian framework. We employ Markov Chain Monte Carlo methods
to provide an estimate of the system. In particular, we design two methods
based on the so-called Gibbs sampler that allow also to estimate the kernel
hyperparameters by marginal likelihood maximization via the
expectation-maximization method. Numerical simulations show the effectiveness
of the proposed scheme, as compared to the state-of-the-art kernel-based
methods when these are employed in system identification with quantized data.Comment: 10 pages, 4 figure
- …