3 research outputs found
Stochastic Learning for Sparse Discrete Markov Random Fields with Controlled Gradient Approximation Error
We study the -regularized maximum likelihood estimator/estimation (MLE)
problem for discrete Markov random fields (MRFs), where efficient and scalable
learning requires both sparse regularization and approximate inference. To
address these challenges, we consider a stochastic learning framework called
stochastic proximal gradient (SPG; Honorio 2012a, Atchade et al.
2014,Miasojedow and Rejchel 2016). SPG is an inexact proximal gradient
algorithm [Schmidtet al., 2011], whose inexactness stems from the stochastic
oracle (Gibbs sampling) for gradient approximation - exact gradient evaluation
is infeasible in general due to the NP-hard inference problem for discrete MRFs
[Koller and Friedman, 2009]. Theoretically, we provide novel verifiable bounds
to inspect and control the quality of gradient approximation. Empirically, we
propose the tighten asymptotically (TAY) learning strategy based on the
verifiable bounds to boost the performance of SPG
Structure learning for CTBN's via penalized maximum likelihood methods
The continuous-time Bayesian networks (CTBNs) represent a class of stochastic
processes, which can be used to model complex phenomena, for instance, they can
describe interactions occurring in living processes, in social science models
or in medicine. The literature on this topic is usually focused on the case
when the dependence structure of a system is known and we are to determine
conditional transition intensities (parameters of the network). In the paper,
we study the structure learning problem, which is a more challenging task and
the existing research on this topic is limited. The approach, which we propose,
is based on a penalized likelihood method. We prove that our algorithm, under
mild regularity conditions, recognizes the dependence structure of the graph
with high probability. We also investigate the properties of the procedure in
numerical studies to demonstrate its effectiveness
Concentration Inequalities for Statistical Inference
This paper gives a review of concentration inequalities which are widely
employed in non-asymptotical analyses of mathematical statistics in a wide
range of settings, from distribution-free to distribution-dependent, from
sub-Gaussian to sub-exponential, sub-Gamma, and sub-Weibull random variables,
and from the mean to the maximum concentration. This review provides results in
these settings with some fresh new results. Given the increasing popularity of
high-dimensional data and inference, results in the context of high-dimensional
linear and Poisson regressions are also provided. We aim to illustrate the
concentration inequalities with known constants and to improve existing bounds
with sharper constants.Comment: Invited review article on constants-specified concentration
inequalities published in Communications in Mathematical Researc