4 research outputs found
Deep reinforcement learning for drone delivery
This work was funded by the Ministry of Science, Innovation and Universities of Spain under Grant No. TRA2016-77012-R.Drones are expected to be used extensively for delivery tasks in the future. In the absence of obstacles, satellite based navigation from departure to the geo-located destination is a simple task. When obstacles are known to be in the path, pilots must build a flight plan to avoid them. However, when they are unknown, there are too many or they are in places that are not fixed positions, then to build a safe flight plan becomes very challenging. Moreover, in a weak satellite signal environment, such as indoors, under trees canopy or in urban canyons, the current drone navigation systems may fail. Artificial intelligence, a research area with increasing activity, can be used to overcome such challenges. Initially focused on robots and now mostly applied to ground vehicles, artificial intelligence begins to be used also to train drones. Reinforcement learning is the branch of artificial intelligence able to train machines. The application of reinforcement learning to drones will provide them with more intelligence, eventually converting drones in fully-autonomous machines. In this work, reinforcement learning is studied for drone delivery. As sensors, the drone only has a stereo-vision front camera, from which depth information is obtained. The drone is trained to fly to a destination in a neighborhood environment that has plenty of obstacles such as trees, cables, cars and houses. The flying area is also delimited by a geo-fence; this is a virtual (non-visible) fence that prevents the drone from entering or leaving a defined area. The drone has to avoid visible obstacles and has to reach a goal. Results show that, in comparison with the previous results, the new algorithms have better results, not only with a better reward, but also with a reduction of its variance. The second contribution is the checkpoints. They consist of saving a trained model every time a better reward is achieved. Results show how checkpoints improve the test results.Peer ReviewedPostprint (published version
Spatiotemporally Constrained Action Space Attacks on Deep Reinforcement Learning Agents
Robustness of Deep Reinforcement Learning (DRL) algorithms towards
adversarial attacks in real world applications such as those deployed in
cyber-physical systems (CPS) are of increasing concern. Numerous studies have
investigated the mechanisms of attacks on the RL agent's state space.
Nonetheless, attacks on the RL agent's action space (AS) (corresponding to
actuators in engineering systems) are equally perverse; such attacks are
relatively less studied in the ML literature. In this work, we first frame the
problem as an optimization problem of minimizing the cumulative reward of an RL
agent with decoupled constraints as the budget of attack. We propose a
white-box Myopic Action Space (MAS) attack algorithm that distributes the
attacks across the action space dimensions. Next, we reformulate the
optimization problem above with the same objective function, but with a
temporally coupled constraint on the attack budget to take into account the
approximated dynamics of the agent. This leads to the white-box Look-ahead
Action Space (LAS) attack algorithm that distributes the attacks across the
action and temporal dimensions. Our results shows that using the same amount of
resources, the LAS attack deteriorates the agent's performance significantly
more than the MAS attack. This reveals the possibility that with limited
resource, an adversary can utilize the agent's dynamics to malevolently craft
attacks that causes the agent to fail. Additionally, we leverage these attack
strategies as a possible tool to gain insights on the potential vulnerabilities
of DRL agents.Comment: Version 2 with supplementary material