9,934 research outputs found
Kernel Exponential Family Estimation via Doubly Dual Embedding
We investigate penalized maximum log-likelihood estimation for exponential
family distributions whose natural parameter resides in a reproducing kernel
Hilbert space. Key to our approach is a novel technique, doubly dual embedding,
that avoids computation of the partition function. This technique also allows
the development of a flexible sampling strategy that amortizes the cost of
Monte-Carlo sampling in the inference stage. The resulting estimator can be
easily generalized to kernel conditional exponential families. We establish a
connection between kernel exponential family estimation and MMD-GANs, revealing
a new perspective for understanding GANs. Compared to the score matching based
estimators, the proposed method improves both memory and time efficiency while
enjoying stronger statistical properties, such as fully capturing smoothness in
its statistical convergence rate while the score matching estimator appears to
saturate. Finally, we show that the proposed estimator empirically outperforms
state-of-the-artComment: 22 pages, 20 figures; AISTATS 201
Weakly Submodular Functions
Submodular functions are well-studied in combinatorial optimization, game
theory and economics. The natural diminishing returns property makes them
suitable for many applications. We study an extension of monotone submodular
functions, which we call {\em weakly submodular functions}. Our extension
includes some (mildly) supermodular functions. We show that several natural
functions belong to this class and relate our class to some other recent
submodular function extensions.
We consider the optimization problem of maximizing a weakly submodular
function subject to uniform and general matroid constraints. For a uniform
matroid constraint, the "standard greedy algorithm" achieves a constant
approximation ratio where the constant (experimentally) converges to 5.95 as
the cardinality constraint increases. For a general matroid constraint, a
simple local search algorithm achieves a constant approximation ratio where the
constant (analytically) converges to 10.22 as the rank of the matroid
increases
- …