8 research outputs found
Binary Independent Component Analysis with OR Mixtures
Independent component analysis (ICA) is a computational method for separating
a multivariate signal into subcomponents assuming the mutual statistical
independence of the non-Gaussian source signals. The classical Independent
Components Analysis (ICA) framework usually assumes linear combinations of
independent sources over the field of realvalued numbers R. In this paper, we
investigate binary ICA for OR mixtures (bICA), which can find applications in
many domains including medical diagnosis, multi-cluster assignment, Internet
tomography and network resource management. We prove that bICA is uniquely
identifiable under the disjunctive generation model, and propose a
deterministic iterative algorithm to determine the distribution of the latent
random variables and the mixing matrix. The inverse problem concerning
inferring the values of latent variables are also considered along with noisy
measurements. We conduct an extensive simulation study to verify the
effectiveness of the propose algorithm and present examples of real-world
applications where bICA can be applied.Comment: Manuscript submitted to IEEE Transactions on Signal Processin
Decorrelation of Neutral Vector Variables: Theory and Applications
In this paper, we propose novel strategies for neutral vector variable
decorrelation. Two fundamental invertible transformations, namely serial
nonlinear transformation and parallel nonlinear transformation, are proposed to
carry out the decorrelation. For a neutral vector variable, which is not
multivariate Gaussian distributed, the conventional principal component
analysis (PCA) cannot yield mutually independent scalar variables. With the two
proposed transformations, a highly negatively correlated neutral vector can be
transformed to a set of mutually independent scalar variables with the same
degrees of freedom. We also evaluate the decorrelation performances for the
vectors generated from a single Dirichlet distribution and a mixture of
Dirichlet distributions. The mutual independence is verified with the distance
correlation measurement. The advantages of the proposed decorrelation
strategies are intensively studied and demonstrated with synthesized data and
practical application evaluations
Infinite Divisibility of Information
We study an information analogue of infinitely divisible probability
distributions, where the i.i.d. sum is replaced by the joint distribution of an
i.i.d. sequence. A random variable is called informationally infinitely
divisible if, for any , there exists an i.i.d. sequence of random
variables that contains the same information as , i.e.,
there exists an injective function such that .
While there does not exist informationally infinitely divisible discrete random
variable, we show that any discrete random variable has a bounded
multiplicative gap to infinite divisibility, that is, if we remove the
injectivity requirement on , then there exists i.i.d.
and satisfying , and the entropy satisfies
. We also study a new class of discrete
probability distributions, called spectral infinitely divisible distributions,
where we can remove the multiplicative gap . Furthermore, we study the
case where is itself an i.i.d. sequence, , for
which the multiplicative gap can be replaced by .
This means that as increases, becomes closer to
being spectral infinitely divisible in a uniform manner. This can be regarded
as an information analogue of Kolmogorov's uniform theorem. Applications of our
result include independent component analysis, distributed storage with a
secrecy constraint, and distributed random number generation.Comment: 22 page
Information Theory and Machine Learning
The recent successes of machine learning, especially regarding systems based on deep neural networks, have encouraged further research activities and raised a new set of challenges in understanding and designing complex machine learning algorithms. New applications require learning algorithms to be distributed, have transferable learning results, use computation resources efficiently, convergence quickly on online settings, have performance guarantees, satisfy fairness or privacy constraints, incorporate domain knowledge on model structures, etc. A new wave of developments in statistical learning theory and information theory has set out to address these challenges. This Special Issue, "Machine Learning and Information Theory", aims to collect recent results in this direction reflecting a diverse spectrum of visions and efforts to extend conventional theories and develop analysis tools for these complex machine learning systems
Interactions on Complex Networks: Inference Algorithms and Applications
Complex networks are ubiquitous – from social and information systems to biological and technological systems. Such networks are platforms for interaction, communication, and collaboration among distributed entities. Studying and analyzing observable network interactions are therefore crucial to understand the hidden complex network properties.
However, with pervasive adoption of the Internet and technology advancements, networks under study today are not only substantially larger than those in the past, but are often highly distributed over large geographical areas. Along with this massive scale, the volume of interaction data also presents a serious challenge to network analysis and data mining techniques. This dissertation focuses on developing inference solutions to complex networks from different domains and applying them in solving practical problems in information and social sciences.
In the first part of the dissertation, we propose Binary Independent Component Analysis with OR Mixtures (bICA), an inference algorithm specialized for communication networks that can be formulated as a bipartite graph. Then we apply bICA and its variants to solve a wide range of networking problems, ranging from optimal monitoring and primary user separation in wireless networks to multicast network tree topology inference. Evaluation results show that the methodology is not only more accurate than previous approaches, but also more robust against measurement noise.
In the second part, we extend our study to the online social networking domain, where the networks are both massive and dynamic. We conduct an extensive analysis on Twitter and associated influence ranking services. Several interesting discoveries have been made, which challenge some of the basic assumptions that many researchers made in the past. We also investigate the problem of finding the set of most influential entities on social networks given a limited budget. Experiments conducted on both large-scale social networks and synthetically generated networks demonstrate the effectiveness of the proposed solution.Computer Science, Department o