87,166 research outputs found
Statistical and Linear Independence of Binary Random Variables
Linear cryptanalysis makes use of statistical models that consider linear approximations over practical and ideal block ciphers as binary random variables. Recently, more complex models have been proposed that take also into account the statistical behavior of correlations of linear approximations over the key space of the cipher and over the randomness of the ideal cipher. The goal of this ongoing work is to investigate independence properties of linear approximations and their relationships. In this third revised version we show that the assumptions of Proposition~1 of the previous version are contradictory and hence renders that result useless. In particular, we prove that linear and statistical independence of binary random variables are equivalent properties in a vector space of variables if and only if all non-zero variables in this vector space are balanced, that is, correspond to components of a permutation. This study is motivated by finding reasonable wrong-key hypotheses for linear cryptanalysis and its generalizations which will also be discussed
Binary Independent Component Analysis with OR Mixtures
Independent component analysis (ICA) is a computational method for separating
a multivariate signal into subcomponents assuming the mutual statistical
independence of the non-Gaussian source signals. The classical Independent
Components Analysis (ICA) framework usually assumes linear combinations of
independent sources over the field of realvalued numbers R. In this paper, we
investigate binary ICA for OR mixtures (bICA), which can find applications in
many domains including medical diagnosis, multi-cluster assignment, Internet
tomography and network resource management. We prove that bICA is uniquely
identifiable under the disjunctive generation model, and propose a
deterministic iterative algorithm to determine the distribution of the latent
random variables and the mixing matrix. The inverse problem concerning
inferring the values of latent variables are also considered along with noisy
measurements. We conduct an extensive simulation study to verify the
effectiveness of the propose algorithm and present examples of real-world
applications where bICA can be applied.Comment: Manuscript submitted to IEEE Transactions on Signal Processin
Multivariate Bernoulli distribution
In this paper, we consider the multivariate Bernoulli distribution as a model
to estimate the structure of graphs with binary nodes. This distribution is
discussed in the framework of the exponential family, and its statistical
properties regarding independence of the nodes are demonstrated. Importantly
the model can estimate not only the main effects and pairwise interactions
among the nodes but also is capable of modeling higher order interactions,
allowing for the existence of complex clique effects. We compare the
multivariate Bernoulli model with existing graphical inference models - the
Ising model and the multivariate Gaussian model, where only the pairwise
interactions are considered. On the other hand, the multivariate Bernoulli
distribution has an interesting property in that independence and
uncorrelatedness of the component random variables are equivalent. Both the
marginal and conditional distributions of a subset of variables in the
multivariate Bernoulli distribution still follow the multivariate Bernoulli
distribution. Furthermore, the multivariate Bernoulli logistic model is
developed under generalized linear model theory by utilizing the canonical link
function in order to include covariate information on the nodes, edges and
cliques. We also consider variable selection techniques such as LASSO in the
logistic model to impose sparsity structure on the graph. Finally, we discuss
extending the smoothing spline ANOVA approach to the multivariate Bernoulli
logistic model to enable estimation of non-linear effects of the predictor
variables.Comment: Published in at http://dx.doi.org/10.3150/12-BEJSP10 the Bernoulli
(http://isi.cbs.nl/bernoulli/) by the International Statistical
Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm
Binary Models for Marginal Independence
Log-linear models are a classical tool for the analysis of contingency
tables. In particular, the subclass of graphical log-linear models provides a
general framework for modelling conditional independences. However, with the
exception of special structures, marginal independence hypotheses cannot be
accommodated by these traditional models. Focusing on binary variables, we
present a model class that provides a framework for modelling marginal
independences in contingency tables. The approach taken is graphical and draws
on analogies to multivariate Gaussian models for marginal independence. For the
graphical model representation we use bi-directed graphs, which are in the
tradition of path diagrams. We show how the models can be parameterized in a
simple fashion, and how maximum likelihood estimation can be performed using a
version of the Iterated Conditional Fitting algorithm. Finally we consider
combining these models with symmetry restrictions
Hierarchical Models for Independence Structures of Networks
We introduce a new family of network models, called hierarchical network
models, that allow us to represent in an explicit manner the stochastic
dependence among the dyads (random ties) of the network. In particular, each
member of this family can be associated with a graphical model defining
conditional independence clauses among the dyads of the network, called the
dependency graph. Every network model with dyadic independence assumption can
be generalized to construct members of this new family. Using this new
framework, we generalize the Erd\"os-R\'enyi and beta-models to create
hierarchical Erd\"os-R\'enyi and beta-models. We describe various methods for
parameter estimation as well as simulation studies for models with sparse
dependency graphs.Comment: 19 pages, 7 figure
Algebraic Aspects of Conditional Independence and Graphical Models
This chapter of the forthcoming Handbook of Graphical Models contains an
overview of basic theorems and techniques from algebraic geometry and how they
can be applied to the study of conditional independence and graphical models.
It also introduces binomial ideals and some ideas from real algebraic geometry.
When random variables are discrete or Gaussian, tools from computational
algebraic geometry can be used to understand implications between conditional
independence statements. This is accomplished by computing primary
decompositions of conditional independence ideals. As examples the chapter
presents in detail the graphical model of a four cycle and the intersection
axiom, a certain implication of conditional independence statements. Another
important problem in the area is to determine all constraints on a graphical
model, for example, equations determined by trek separation. The full set of
equality constraints can be determined by computing the model's vanishing
ideal. The chapter illustrates these techniques and ideas with examples from
the literature and provides references for further reading.Comment: 20 pages, 1 figur
Markov Properties of Discrete Determinantal Point Processes
Determinantal point processes (DPPs) are probabilistic models for repulsion.
When used to represent the occurrence of random subsets of a finite base set,
DPPs allow to model global negative associations in a mathematically elegant
and direct way. Discrete DPPs have become popular and computationally tractable
models for solving several machine learning tasks that require the selection of
diverse objects, and have been successfully applied in numerous real-life
problems. Despite their popularity, the statistical properties of such models
have not been adequately explored. In this note, we derive the Markov
properties of discrete DPPs and show how they can be expressed using graphical
models.Comment: 9 pages, 1 figur
Exchangeable Variable Models
A sequence of random variables is exchangeable if its joint distribution is
invariant under variable permutations. We introduce exchangeable variable
models (EVMs) as a novel class of probabilistic models whose basic building
blocks are partially exchangeable sequences, a generalization of exchangeable
sequences. We prove that a family of tractable EVMs is optimal under zero-one
loss for a large class of functions, including parity and threshold functions,
and strictly subsumes existing tractable independence-based model families.
Extensive experiments show that EVMs outperform state of the art classifiers
such as SVMs and probabilistic models which are solely based on independence
assumptions.Comment: ICML 201
- …