25,348 research outputs found
On Marton's inner bound for broadcast channels
Marton's inner bound is the best known achievable region for a general
discrete memoryless broadcast channel. To compute Marton's inner bound one has
to solve an optimization problem over a set of joint distributions on the input
and auxiliary random variables. The optimizers turn out to be structured in
many cases. Finding properties of optimizers not only results in efficient
evaluation of the region, but it may also help one to prove factorization of
Marton's inner bound (and thus its optimality). The first part of this paper
formulates this factorization approach explicitly and states some conjectures
and results along this line. The second part of this paper focuses primarily on
the structure of the optimizers. This section is inspired by a new binary
inequality that recently resulted in a very simple characterization of the
sum-rate of Marton's inner bound for binary input broadcast channels. This
prompted us to investigate whether this inequality can be extended to larger
cardinality input alphabets. We show that several of the results for the binary
input case do carry over for higher cardinality alphabets and we present a
collection of results that help restrict the search space of probability
distributions to evaluate the boundary of Marton's inner bound in the general
case. We also prove a new inequality for the binary skew-symmetric broadcast
channel that yields a very simple characterization of the entire Marton inner
bound for this channel.Comment: Submitted to ISIT 201
Discrete chain graph models
The statistical literature discusses different types of Markov properties for
chain graphs that lead to four possible classes of chain graph Markov models.
The different models are rather well understood when the observations are
continuous and multivariate normal, and it is also known that one model class,
referred to as models of LWF (Lauritzen--Wermuth--Frydenberg) or block
concentration type, yields discrete models for categorical data that are
smooth. This paper considers the structural properties of the discrete models
based on the three alternative Markov properties. It is shown by example that
two of the alternative Markov properties can lead to non-smooth models. The
remaining model class, which can be viewed as a discrete version of
multivariate regressions, is proven to comprise only smooth models. The proof
employs a simple change of coordinates that also reveals that the model's
likelihood function is unimodal if the chain components of the graph are
complete sets.Comment: Published in at http://dx.doi.org/10.3150/08-BEJ172 the Bernoulli
(http://isi.cbs.nl/bernoulli/) by the International Statistical
Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm
Hierarchical Compound Poisson Factorization
Non-negative matrix factorization models based on a hierarchical
Gamma-Poisson structure capture user and item behavior effectively in extremely
sparse data sets, making them the ideal choice for collaborative filtering
applications. Hierarchical Poisson factorization (HPF) in particular has proved
successful for scalable recommendation systems with extreme sparsity. HPF,
however, suffers from a tight coupling of sparsity model (absence of a rating)
and response model (the value of the rating), which limits the expressiveness
of the latter. Here, we introduce hierarchical compound Poisson factorization
(HCPF) that has the favorable Gamma-Poisson structure and scalability of HPF to
high-dimensional extremely sparse matrices. More importantly, HCPF decouples
the sparsity model from the response model, allowing us to choose the most
suitable distribution for the response. HCPF can capture binary, non-negative
discrete, non-negative continuous, and zero-inflated continuous responses. We
compare HCPF with HPF on nine discrete and three continuous data sets and
conclude that HCPF captures the relationship between sparsity and response
better than HPF.Comment: Will appear on Proceedings of the 33 rd International Conference on
Machine Learning, New York, NY, USA, 2016. JMLR: W&CP volume 4
Bell's theorem as a signature of nonlocality: a classical counterexample
For a system composed of two particles Bell's theorem asserts that averages
of physical quantities determined from local variables must conform to a family
of inequalities. In this work we show that a classical model containing a local
probabilistic interaction in the measurement process can lead to a violation of
the Bell inequalities. We first introduce two-particle phase-space
distributions in classical mechanics constructed to be the analogs of quantum
mechanical angular momentum eigenstates. These distributions are then employed
in four schemes characterized by different types of detectors measuring the
angular momenta. When the model includes an interaction between the detector
and the measured particle leading to ensemble dependencies, the relevant Bell
inequalities are violated if total angular momentum is required to be
conserved. The violation is explained by identifying assumptions made in the
derivation of Bell's theorem that are not fulfilled by the model. These
assumptions will be argued to be too restrictive to see in the violation of the
Bell inequalities a faithful signature of nonlocality.Comment: Extended manuscript. Significant change
Bell's theorem as a signature of nonlocality: a classical counterexample
For a system composed of two particles Bell's theorem asserts that averages
of physical quantities determined from local variables must conform to a family
of inequalities. In this work we show that a classical model containing a local
probabilistic interaction in the measurement process can lead to a violation of
the Bell inequalities. We first introduce two-particle phase-space
distributions in classical mechanics constructed to be the analogs of quantum
mechanical angular momentum eigenstates. These distributions are then employed
in four schemes characterized by different types of detectors measuring the
angular momenta. When the model includes an interaction between the detector
and the measured particle leading to ensemble dependencies, the relevant Bell
inequalities are violated if total angular momentum is required to be
conserved. The violation is explained by identifying assumptions made in the
derivation of Bell's theorem that are not fulfilled by the model. These
assumptions will be argued to be too restrictive to see in the violation of the
Bell inequalities a faithful signature of nonlocality.Comment: Extended manuscript. Significant change
Smooth, identifiable supermodels of discrete DAG models with latent variables
We provide a parameterization of the discrete nested Markov model, which is a
supermodel that approximates DAG models (Bayesian network models) with latent
variables. Such models are widely used in causal inference and machine
learning. We explicitly evaluate their dimension, show that they are curved
exponential families of distributions, and fit them to data. The
parameterization avoids the irregularities and unidentifiability of latent
variable models. The parameters used are all fully identifiable and
causally-interpretable quantities.Comment: 30 page
Graphical models in Macaulay2
The Macaulay2 package GraphicalModels contains algorithms for the algebraic
study of graphical models associated to undirected, directed and mixed graphs,
and associated collections of conditional independence statements. Among the
algorithms implemented are procedures for computing the vanishing ideal of
graphical models, for generating conditional independence ideals of families of
independence statements associated to graphs, and for checking for identifiable
parameters in Gaussian mixed graph models. These procedures can be used to
study fundamental problems about graphical models.Comment: Several changes to address referee comments and suggestions. We will
eventually include this package in the standard distribution of Macaulay2.
But until then, the associated Macaulay2 file can be found at
http://www.shsu.edu/~ldg005/papers.htm
Bayesian Estimation for Continuous-Time Sparse Stochastic Processes
We consider continuous-time sparse stochastic processes from which we have
only a finite number of noisy/noiseless samples. Our goal is to estimate the
noiseless samples (denoising) and the signal in-between (interpolation
problem).
By relying on tools from the theory of splines, we derive the joint a priori
distribution of the samples and show how this probability density function can
be factorized. The factorization enables us to tractably implement the maximum
a posteriori and minimum mean-square error (MMSE) criteria as two statistical
approaches for estimating the unknowns. We compare the derived statistical
methods with well-known techniques for the recovery of sparse signals, such as
the norm and Log (- relaxation) regularization
methods. The simulation results show that, under certain conditions, the
performance of the regularization techniques can be very close to that of the
MMSE estimator.Comment: To appear in IEEE TS
- …