Learning to play table tennis is a challenging task for robots, as a wide
variety of strokes required. Recent advances have shown that deep Reinforcement
Learning (RL) is able to successfully learn the optimal actions in a simulated
environment. However, the applicability of RL in real scenarios remains limited
due to the high exploration effort. In this work, we propose a realistic
simulation environment in which multiple models are built for the dynamics of
the ball and the kinematics of the robot. Instead of training an end-to-end RL
model, a novel policy gradient approach with TD3 backbone is proposed to learn
the racket strokes based on the predicted state of the ball at the hitting
time. In the experiments, we show that the proposed approach significantly
outperforms the existing RL methods in simulation. Furthermore, to cross the
domain from simulation to reality, we adopt an efficient retraining method and
test it in three real scenarios. The resulting success rate is 98% and the
distance error is around 24.9 cm. The total training time is about 1.5 hours