7,547 research outputs found
Learning to Optimize under Non-Stationarity
We introduce algorithms that achieve state-of-the-art \emph{dynamic regret}
bounds for non-stationary linear stochastic bandit setting. It captures natural
applications such as dynamic pricing and ads allocation in a changing
environment. We show how the difficulty posed by the non-stationarity can be
overcome by a novel marriage between stochastic and adversarial bandits
learning algorithms. Defining and as the problem dimension, the
\emph{variation budget}, and the total time horizon, respectively, our main
contributions are the tuned Sliding Window UCB (\texttt{SW-UCB}) algorithm with
optimal dynamic regret, and the
tuning free bandit-over-bandit (\texttt{BOB}) framework built on top of the
\texttt{SW-UCB} algorithm with best
dynamic regret
Dynamic reconfiguration of human brain networks during learning
Human learning is a complex phenomenon requiring flexibility to adapt
existing brain function and precision in selecting new neurophysiological
activities to drive desired behavior. These two attributes -- flexibility and
selection -- must operate over multiple temporal scales as performance of a
skill changes from being slow and challenging to being fast and automatic. Such
selective adaptability is naturally provided by modular structure, which plays
a critical role in evolution, development, and optimal network function. Using
functional connectivity measurements of brain activity acquired from initial
training through mastery of a simple motor skill, we explore the role of
modularity in human learning by identifying dynamic changes of modular
organization spanning multiple temporal scales. Our results indicate that
flexibility, which we measure by the allegiance of nodes to modules, in one
experimental session predicts the relative amount of learning in a future
session. We also develop a general statistical framework for the identification
of modular architectures in evolving systems, which is broadly applicable to
disciplines where network adaptability is crucial to the understanding of
system performance.Comment: Main Text: 19 pages, 4 figures Supplementary Materials: 34 pages, 4
figures, 3 table
Distributed Big-Data Optimization via Block Communications
We study distributed multi-agent large-scale optimization problems, wherein
the cost function is composed of a smooth possibly nonconvex sum-utility plus a
DC (Difference-of-Convex) regularizer. We consider the scenario where the
dimension of the optimization variables is so large that optimizing and/or
transmitting the entire set of variables could cause unaffordable computation
and communication overhead. To address this issue, we propose the first
distributed algorithm whereby agents optimize and communicate only a portion of
their local variables. The scheme hinges on successive convex approximation
(SCA) to handle the nonconvexity of the objective function, coupled with a
novel block-signal tracking scheme, aiming at locally estimating the average of
the agents' gradients. Asymptotic convergence to stationary solutions of the
nonconvex problem is established. Numerical results on a sparse regression
problem show the effectiveness of the proposed algorithm and the impact of the
block size on its practical convergence speed and communication cost
- …