688 research outputs found
Bayesian Joint Matrix Decomposition for Data Integration with Heterogeneous Noise
Matrix decomposition is a popular and fundamental approach in machine
learning and data mining. It has been successfully applied into various fields.
Most matrix decomposition methods focus on decomposing a data matrix from one
single source. However, it is common that data are from different sources with
heterogeneous noise. A few of matrix decomposition methods have been extended
for such multi-view data integration and pattern discovery. While only few
methods were designed to consider the heterogeneity of noise in such multi-view
data for data integration explicitly. To this end, we propose a joint matrix
decomposition framework (BJMD), which models the heterogeneity of noise by
Gaussian distribution in a Bayesian framework. We develop two algorithms to
solve this model: one is a variational Bayesian inference algorithm, which
makes full use of the posterior distribution; and another is a maximum a
posterior algorithm, which is more scalable and can be easily paralleled.
Extensive experiments on synthetic and real-world datasets demonstrate that
BJMD considering the heterogeneity of noise is superior or competitive to the
state-of-the-art methods.Comment: 14 pages, 7 figures, 8 table
The Discovery of Mutated Driver Pathways in Cancer: Models and Algorithms
The pathogenesis of cancer in human is still poorly understood. With the
rapid development of high-throughput sequencing technologies, huge volumes of
cancer genomics data have been generated. Deciphering those data poses great
opportunities and challenges to computational biologists. One of such key
challenges is to distinguish driver mutations, genes as well as pathways from
passenger ones. Mutual exclusivity of gene mutations (each patient has no more
than one mutation in the gene set) has been observed in various cancer types
and thus has been used as an important property of a driver gene set or
pathway. In this article, we aim to review the recent development of
computational models and algorithms for discovering driver pathways or modules
in cancer with the focus on mutual exclusivity-based ones.Comment: 11 pages, 4 figure
A Unified Joint Matrix Factorization Framework for Data Integration
Nonnegative matrix factorization (NMF) is a powerful tool in data exploratory
analysis by discovering the hidden features and part-based patterns from
high-dimensional data. NMF and its variants have been successfully applied into
diverse fields such as pattern recognition, signal processing, data mining,
bioinformatics and so on. Recently, NMF has been extended to analyze multiple
matrices simultaneously. However, a unified framework is still lacking. In this
paper, we introduce a sparse multiple relationship data regularized joint
matrix factorization (JMF) framework and two adapted prediction models for
pattern recognition and data integration. Next, we present four update
algorithms to solve this framework. The merits and demerits of these algorithms
are systematically explored. Furthermore, extensive computational experiments
using both synthetic data and real data demonstrate the effectiveness of JMF
framework and related algorithms on pattern recognition and data mining.Comment: 14 pages, 7 figure
Tessellated Wasserstein Auto-Encoders
Non-adversarial generative models such as variational auto-encoder (VAE),
Wasserstein auto-encoders with maximum mean discrepancy (WAE-MMD),
sliced-Wasserstein auto-encoder (SWAE) are relatively easy to train and have
less mode collapse compared to Wasserstein auto-encoder with generative
adversarial network (WAE-GAN). However, they are not very accurate in
approximating the target distribution in the latent space because they don't
have a discriminator to detect the minor difference between real and fake. To
this end, we develop a novel non-adversarial framework called Tessellated
Wasserstein Auto-encoders (TWAE) to tessellate the support of the target
distribution into a given number of regions by the centroidal Voronoi
tessellation (CVT) technique and design batches of data according to the
tessellation instead of random shuffling for accurate computation of
discrepancy. Theoretically, we demonstrate that the error of estimate to the
discrepancy decreases when the numbers of samples and regions of the
tessellation become larger with rates of and
, respectively. Given fixed and , a
necessary condition for the upper bound of measurement error to be minimized is
that the tessellation is the one determined by CVT. TWAE is very flexible to
different non-adversarial metrics and can substantially enhance their
generative performance in terms of Fr\'{e}chet inception distance (FID)
compared to VAE, WAE-MMD, SWAE. Moreover, numerical results indeed demonstrate
that TWAE is competitive to the adversarial model WAE-GAN, demonstrating its
powerful generative ability.Comment: 38 pages, 8 figure
Sparse Deep Nonnegative Matrix Factorization
Nonnegative matrix factorization is a powerful technique to realize dimension
reduction and pattern recognition through single-layer data representation
learning. Deep learning, however, with its carefully designed hierarchical
structure, is able to combine hidden features to form more representative
features for pattern recognition. In this paper, we proposed sparse deep
nonnegative matrix factorization models to analyze complex data for more
accurate classification and better feature interpretation. Such models are
designed to learn localized features or generate more discriminative
representations for samples in distinct classes by imposing -norm penalty
on the columns of certain factors. By extending one-layer model into
multi-layer one with sparsity, we provided a hierarchical way to analyze big
data and extract hidden features intuitively due to nonnegativity. We adopted
the Nesterov's accelerated gradient algorithm to accelerate the computing
process with the convergence rate of after steps iteration. We
also analyzed the computing complexity of our framework to demonstrate their
efficiency. To improve the performance of dealing with linearly inseparable
data, we also considered to incorporate popular nonlinear functions into this
framework and explored their performance. We applied our models onto two
benchmarking image datasets, demonstrating our models can achieve competitive
or better classification performance and produce intuitive interpretations
compared with the typical NMF and competing multi-layer models.Comment: 13 pages, 8 figure
Local community extraction in directed networks
Network is a simple but powerful representation of real-world complex
systems. Network community analysis has become an invaluable tool to explore
and reveal the internal organization of nodes. However, only a few methods were
directly designed for community-detection in directed networks. In this
article, we introduce the concept of local community structure in directed
networks and provide a generic criterion to describe a local community with two
properties. We further propose a stochastic optimization algorithm to rapidly
detect a local community, which allows for uncovering the directional modular
characteristics in directed networks. Numerical results show that the proposed
method can resolve detailed local communities with directional information and
provide more structural characteristics of directed networks than previous
methods.Comment: 8 pages, 6 figure
Group-sparse SVD Models and Their Applications in Biological Data
Sparse Singular Value Decomposition (SVD) models have been proposed for
biclustering high dimensional gene expression data to identify block patterns
with similar expressions. However, these models do not take into account prior
group effects upon variable selection. To this end, we first propose
group-sparse SVD models with group Lasso (GL1-SVD) and group L0-norm penalty
(GL0-SVD) for non-overlapping group structure of variables. However, such
group-sparse SVD models limit their applicability in some problems with
overlapping structure. Thus, we also propose two group-sparse SVD models with
overlapping group Lasso (OGL1-SVD) and overlapping group L0-norm penalty
(OGL0-SVD). We first adopt an alternating iterative strategy to solve GL1-SVD
based on a block coordinate descent method, and GL0-SVD based on a projection
method. The key of solving OGL1-SVD is a proximal operator with overlapping
group Lasso penalty. We employ an alternating direction method of multipliers
(ADMM) to solve the proximal operator. Similarly, we develop an approximate
method to solve OGL0-SVD. Applications of these methods and comparison with
competing ones using simulated data demonstrate their effectiveness. Extensive
applications of them onto several real gene expression data with gene prior
group knowledge identify some biologically interpretable gene modules.Comment: 14 pages, 4 figure
L0-norm Sparse Graph-regularized SVD for Biclustering
Learning the "blocking" structure is a central challenge for high dimensional
data (e.g., gene expression data). Recently, a sparse singular value
decomposition (SVD) has been used as a biclustering tool to achieve this goal.
However, this model ignores the structural information between variables (e.g.,
gene interaction graph). Although typical graph-regularized norm can
incorporate such prior graph information to get accurate discovery and better
interpretability, it fails to consider the opposite effect of variables with
different signs. Motivated by the development of sparse coding and
graph-regularized norm, we propose a novel sparse graph-regularized SVD as a
powerful biclustering tool for analyzing high-dimensional data. The key of this
method is to impose two penalties including a novel graph-regularized norm
() and -norm () on singular
vectors to induce structural sparsity and enhance interpretability. We design
an efficient Alternating Iterative Sparse Projection (AISP) algorithm to solve
it. Finally, we apply our method and related ones to simulated and real data to
show its efficiency in capturing natural blocking structures.Comment: 8 pages, 2 figure
A neurodynamic framework for local community extraction in networks
To understand the structure and organization of a large-scale social,
biological or technological network, it can be helpful to describe and extract
local communities or modules of the network. In this article, we develop a
neurodynamic framework to describe the local communities which correspond to
the stable states of a neuro-system built based on the network. The
quantitative criteria to describe the neurodynamic system can cover a large
range of objective functions. The resolution limit of these functions enable us
to propose a generic criterion to explore multi-resolution local communities.
We explain the advantages of this framework and illustrate them by testing on a
number of model and real-world networks.Comment: 4 figure
Sparse Weighted Canonical Correlation Analysis
Given two data matrices and , sparse canonical correlation analysis
(SCCA) is to seek two sparse canonical vectors and to maximize the
correlation between and . However, classical and sparse CCA models
consider the contribution of all the samples of data matrices and thus cannot
identify an underlying specific subset of samples. To this end, we propose a
novel sparse weighted canonical correlation analysis (SWCCA), where weights are
used for regularizing different samples. We solve the -regularized SWCCA
(-SWCCA) using an alternating iterative algorithm. We apply -SWCCA to
synthetic data and real-world data to demonstrate its effectiveness and
superiority compared to related methods. Lastly, we consider also SWCCA with
different penalties like LASSO (Least absolute shrinkage and selection
operator) and Group LASSO, and extend it for integrating more than three data
matrices.Comment: 8 pages, 5 figure
- β¦