Efficient collective swimming by harnessing vortices through deep reinforcement learning

Fish in schooling formations navigate complex flow-fields replete with mechanical energy in the vortex wakes of their companions. Their schooling behaviour has been associated with evolutionary advantages including collective energy savings. How fish harvest energy from their complex fluid environment and the underlying physical mechanisms governing energy-extraction during collective swimming, is still unknown. Here we show that fish can improve their sustained propulsive efficiency by actively following, and judiciously intercepting, vortices in the wake of other swimmers. This swimming strategy leads to collective energy-savings and is revealed through the first ever combination of deep reinforcement learning with high-fidelity flow simulations. We find that a `smart-swimmer' can adapt its position and body deformation to synchronise with the momentum of the oncoming vortices, improving its average swimming-efficiency at no cost to the leader. The results show that fish may harvest energy deposited in vortices produced by their peers, and support the conjecture that swimming in formation is energetically advantageous. Moreover, this study demonstrates that deep reinforcement learning can produce navigation algorithms for complex flow-fields, with promising implications for energy savings in autonomous robotic swarms.Comment: 26 pages, 14 figure

Optimizing collective fieldtaxis of swarming agents through reinforcement learning

Swarming of animal groups enthralls scientists in fields ranging from biology to physics to engineering. Complex swarming patterns often arise from simple interactions between individuals to the benefit of the collective whole. The existence and success of swarming, however, nontrivially depend on microscopic parameters governing the interactions. Here we show that a machine-learning technique can be employed to tune these underlying parameters and optimize the resulting performance. As a concrete example, we take an active matter model inspired by schools of golden shiners, which collectively conduct phototaxis. The problem of optimizing the phototaxis capability is then mapped to that of maximizing benefits in a continuum-armed bandit game. The latter problem accepts a simple reinforcement-learning algorithm, which can tune the continuous parameters of the model. This result suggests the utility of machine-learning methodology in swarm-robotics applications.Comment: 6 pages, 3 figure

Intelligent Escape of Robotic Systems: A Survey of Methodologies, Applications, and Challenges

Intelligent escape is an interdisciplinary field that employs artificial intelligence (AI) techniques to enable robots with the capacity to intelligently react to potential dangers in dynamic, intricate, and unpredictable scenarios. As the emphasis on safety becomes increasingly paramount and advancements in robotic technologies continue to advance, a wide range of intelligent escape methodologies has been developed in recent years. This paper presents a comprehensive survey of state-of-the-art research work on intelligent escape of robotic systems. Four main methods of intelligent escape are reviewed, including planning-based methodologies, partitioning-based methodologies, learning-based methodologies, and bio-inspired methodologies. The strengths and limitations of existing methods are summarized. In addition, potential applications of intelligent escape are discussed in various domains, such as search and rescue, evacuation, military security, and healthcare. In an effort to develop new approaches to intelligent escape, this survey identifies current research challenges and provides insights into future research trends in intelligent escape.Comment: This paper is accepted by Journal of Intelligent and Robotic System

Optimum control strategies for maximum thrust production in underwater undulatory swimming

Fish, cetaceans and many other aquatic vertebrates undulate their bodies to propel themselves through water. Numerous studies on natural, artificial or analogous swimmers are dedicated to revealing the links between the kinematics of body oscillation and the production of thrust for swimming. One of the most open and difficult questions concerns the best kinematics to maximize this later quantity for given constraints and how a system strategizes and adjusts its internal parameters to reach this maximum. To address this challenge, we exploit a biomimetic robotic swimmer to determine the control signal that produces the highest thrust. Using machine learning techniques and intuitive models, we find that this optimal control consists of a square wave function, whose frequency is fixed by the interplay between the internal dynamics of the swimmer and the fluid-structure interaction with the surrounding fluid. We then propose a simple implementation for autonomous robotic swimmers that requires no prior knowledge of systems or equations. This application to aquatic locomotion is validated by 2D numerical simulations