21,448 research outputs found
A Finite Time Analysis of Two Time-Scale Actor Critic Methods
Actor-critic (AC) methods have exhibited great empirical success compared
with other reinforcement learning algorithms, where the actor uses the policy
gradient to improve the learning policy and the critic uses temporal difference
learning to estimate the policy gradient. Under the two time-scale learning
rate schedule, the asymptotic convergence of AC has been well studied in the
literature. However, the non-asymptotic convergence and finite sample
complexity of actor-critic methods are largely open. In this work, we provide a
non-asymptotic analysis for two time-scale actor-critic methods under
non-i.i.d. setting. We prove that the actor-critic method is guaranteed to find
a first-order stationary point (i.e., ) of the non-concave performance function
, with sample
complexity. To the best of our knowledge, this is the first work providing
finite-time analysis and sample complexity bound for two time-scale
actor-critic methods.Comment: 45 page
A machine learning approach to portfolio pricing and risk management for high-dimensional problems
We present a general framework for portfolio risk management in discrete
time, based on a replicating martingale. This martingale is learned from a
finite sample in a supervised setting. The model learns the features necessary
for an effective low-dimensional representation, overcoming the curse of
dimensionality common to function approximation in high-dimensional spaces. We
show results based on polynomial and neural network bases. Both offer superior
results to naive Monte Carlo methods and other existing methods like
least-squares Monte Carlo and replicating portfolios.Comment: 30 pages (main), 10 pages (appendix), 3 figures, 22 table
- …