The state space of Android apps is huge and its thorough exploration during
testing remains a major challenge. In fact, the best exploration strategy is
highly dependent on the features of the app under test. Reinforcement Learning
(RL) is a machine learning technique that learns the optimal strategy to solve
a task by trial and error, guided by positive or negative reward, rather than
by explicit supervision. Deep RL is a recent extension of RL that takes
advantage of the learning capabilities of neural networks. Such capabilities
make Deep RL suitable for complex exploration spaces such as the one of Android
apps. However, state of the art, publicly available tools only support basic,
tabular RL. We have developed ARES, a Deep RL approach for black-box testing of
Android apps. Experimental results show that it achieves higher coverage and
fault revelation than the baselines, which include state of the art RL based
tools, such as TimeMachine and Q-Testing. We also investigated qualitatively
the reasons behind such performance and we have identified the key features of
Android apps that make Deep RL particularly effective on them to be the
presence of chained and blocking activities
