204 research outputs found
Asymptotic normality of maximum likelihood and its variational approximation for stochastic blockmodels
Variational methods for parameter estimation are an active research area,
potentially offering computationally tractable heuristics with theoretical
performance bounds. We build on recent work that applies such methods to
network data, and establish asymptotic normality rates for parameter estimates
of stochastic blockmodel data, by either maximum likelihood or variational
estimation. The result also applies to various sub-models of the stochastic
blockmodel found in the literature.Comment: Published in at http://dx.doi.org/10.1214/13-AOS1124 the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org
Linear Convergence of Adaptively Iterative Thresholding Algorithms for Compressed Sensing
This paper studies the convergence of the adaptively iterative thresholding
(AIT) algorithm for compressed sensing. We first introduce a generalized
restricted isometry property (gRIP). Then we prove that the AIT algorithm
converges to the original sparse solution at a linear rate under a certain gRIP
condition in the noise free case. While in the noisy case, its convergence rate
is also linear until attaining a certain error bound. Moreover, as by-products,
we also provide some sufficient conditions for the convergence of the AIT
algorithm based on the two well-known properties, i.e., the coherence property
and the restricted isometry property (RIP), respectively. It should be pointed
out that such two properties are special cases of gRIP. The solid improvements
on the theoretical results are demonstrated and compared with the known
results. Finally, we provide a series of simulations to verify the correctness
of the theoretical assertions as well as the effectiveness of the AIT
algorithm.Comment: 15 pages, 5 figure
Snap-Shot Decentralized Stochastic Gradient Tracking Methods
In decentralized optimization, agents form a network and only communicate
with their neighbors, which gives advantages in data ownership, privacy, and
scalability. At the same time, decentralized stochastic gradient descent
(\texttt{SGD}) methods, as popular decentralized algorithms for training
large-scale machine learning models, have shown their superiority over
centralized counterparts. Distributed stochastic gradient
tracking~(\texttt{DSGT})~\citep{pu2021distributed} has been recognized as the
popular and state-of-the-art decentralized \texttt{SGD} method due to its
proper theoretical guarantees. However, the theoretical analysis of
\dsgt~\citep{koloskova2021improved} shows that its iteration complexity is
, where is a double stochastic mixing matrix that presents the
network topology and is a parameter that depends on . Thus, it
indicates that the convergence property of \texttt{DSGT} is heavily affected by
the topology of the communication network. To overcome the weakness of
\texttt{DSGT}, we resort to the snap-shot gradient tracking skill and propose
two novel algorithms. We further justify that the proposed two algorithms are
more robust to the topology of communication networks under similar algorithmic
structures and the same communication strategy to \dsgt~. Compared with \dsgt,
their iteration complexity are and which reduce the impact on
network topology (no )
A multi-task learning CNN for image steganalysis
Convolutional neural network (CNN) based image steganalysis are increasingly popular because of their superiority in accuracy. The most straightforward way to employ CNN for image steganalysis is to learn a CNN-based classifier to distinguish whether secret messages have been embedded into an image. However, it is difficult to learn such a classifier because of the weak stego signals and the limited useful information. To address this issue, in this paper, a multi-task learning CNN is proposed. In addition to the typical use of CNN, learning a CNN-based classifier for the whole image, our multi-task CNN is learned with an auxiliary task of the pixel binary classification, estimating whether each pixel in an image has been modified due to steganography. To the best of our knowledge, we are the first to employ CNN to perform the pixel-level classification of such type. Experimental results have justified the effectiveness and efficiency of the proposed multi-task learning CNN
- …