11,381 research outputs found
Distributed Stochastic Optimization under Imperfect Information
We consider a stochastic convex optimization problem that requires minimizing
a sum of misspecified agentspecific expectation-valued convex functions over
the intersection of a collection of agent-specific convex sets. This
misspecification is manifested in a parametric sense and may be resolved
through solving a distinct stochastic convex learning problem. Our interest
lies in the development of distributed algorithms in which every agent makes
decisions based on the knowledge of its objective and feasibility set while
learning the decisions of other agents by communicating with its local
neighbors over a time-varying connectivity graph. While a significant body of
research currently exists in the context of such problems, we believe that the
misspecified generalization of this problem is both important and has seen
little study, if at all. Accordingly, our focus lies on the simultaneous
resolution of both problems through a joint set of schemes that combine three
distinct steps: (i) An alignment step in which every agent updates its current
belief by averaging over the beliefs of its neighbors; (ii) A projected
(stochastic) gradient step in which every agent further updates this averaged
estimate; and (iii) A learning step in which agents update their belief of the
misspecified parameter by utilizing a stochastic gradient step. Under an
assumption of mere convexity on agent objectives and strong convexity of the
learning problems, we show that the sequences generated by this collection of
update rules converge almost surely to the solution of the correctly specified
stochastic convex optimization problem and the stochastic learning problem,
respectively
Four lectures on probabilistic methods for data science
Methods of high-dimensional probability play a central role in applications
for statistics, signal processing theoretical computer science and related
fields. These lectures present a sample of particularly useful tools of
high-dimensional probability, focusing on the classical and matrix Bernstein's
inequality and the uniform matrix deviation inequality. We illustrate these
tools with applications for dimension reduction, network analysis, covariance
estimation, matrix completion and sparse signal recovery. The lectures are
geared towards beginning graduate students who have taken a rigorous course in
probability but may not have any experience in data science applications.Comment: Lectures given at 2016 PCMI Graduate Summer School in Mathematics of
Data. Some typos, inaccuracies fixe
- …