10 research outputs found
A PDE Approach to the Problems of Optimality of Expectations
Let (X, Z) be a bivariate random vector. A predictor of X based on Z is just a Borel function g(Z). The problem of "least squares prediction" of X given the observation Z is to find the global minimum point of the functional E[(X − g(Z))2] with respect to all random variables g(Z), where g is a Borel function. It is well known that the solution of this problem is the conditional expectation E(X|Z). We also know that, if for a nonnegative smooth function F: R×R → R, arg ming(Z)E[F(X, g(Z))] = E[X|Z], for all X and Z, then F(x, y) is a Bregmann loss function. It is also of interest, for a fixed ϕ to find F (x, y), satisfying, arg ming(Z)E[F(X, g(Z))] = ϕ(E[X|Z]), for all X and Z. In more general setting, a stronger problem is to find F (x, y) satisfying arg miny∈RE[F (X, y)] = ϕ(E[X]), ∀X. We study this problem and develop a partial differential equation (PDE) approach to solution of these problems
Inertial Block Proximal Methods for Non-Convex Non-Smooth Optimization
We propose inertial versions of block coordinate descent methods for solving
non-convex non-smooth composite optimization problems. Our methods possess
three main advantages compared to current state-of-the-art accelerated
first-order methods: (1) they allow using two different extrapolation points to
evaluate the gradients and to add the inertial force (we will empirically show
that it is more efficient than using a single extrapolation point), (2) they
allow to randomly picking the block of variables to update, and (3) they do not
require a restarting step. We prove the subsequential convergence of the
generated sequence under mild assumptions, prove the global convergence under
some additional assumptions, and provide convergence rates. We deploy the
proposed methods to solve non-negative matrix factorization (NMF) and show that
they compete favorably with the state-of-the-art NMF algorithms. Additional
experiments on non-negative approximate canonical polyadic decomposition, also
known as non-negative tensor factorization, are also provided