A deep reinforcement learning strategy for autonomous robot flocking

Social behaviors in animals such as bees, ants, and birds have shown high levels of intelligence from a multi-agent system perspective. They present viable solutions to real-world problems, particularly in navigating constrained environments with simple robotic platforms. Among these behaviors is swarm flocking, which has been extensively studied for this purpose. Flocking algorithms have been developed from basic behavioral rules, which often require parameter tuning for specific applications. However, the lack of a general formulation for tuning has made these strategies difficult to implement in various real conditions, and even to replicate laboratory behaviors. In this paper, we propose a flocking scheme for small autonomous robots that can self-learn in dynamic environments, derived from a deep reinforcement learning process. Our approach achieves flocking independently of population size and environmental characteristics, with minimal external intervention. Our multi-agent system model considers each agent’s action as a linear function dynamically adjusting the motion according to interactions with other agents and the environment. Our strategy is an important contribution toward real-world flocking implementation. We demonstrate that our approach allows for autonomous flocking in the system without requiring specific parameter tuning, making it ideal for applications where there is a need for simple robotic platforms to navigate in dynamic environments

Guidance and control of an autonomous underwater vehicle

Merged with duplicate record 10026.1/856 on 07.03.2017 by CS (TIS)A cooperative project between the Universities of Plymouth and Cranfield was aimed at designing and developing an autonomous underwater vehicle named Hammerhead. The work presented herein is to formulate an advance guidance and control system and to implement it in the Hammerhead. This involves the description of Hammerhead hardware from a control system perspective. In addition to the control system, an intelligent navigation scheme and a state of the art vision system is also developed. However, the development of these submodules is out of the scope of this thesis. To model an underwater vehicle, the traditional way is to acquire painstaking mathematical models based on laws of physics and then simplify and linearise the models to some operating point. One of the principal novelties of this research is the use of system identification techniques on actual vehicle data obtained from full scale in water experiments. Two new guidance mechanisms have also been formulated for cruising type vehicles. The first is a modification of the proportional navigation guidance for missiles whilst the other is a hybrid law which is a combination of several guidance strategies employed during different phases of the Right. In addition to the modelling process and guidance systems, a number of robust control methodologies have been conceived for Hammerhead. A discrete time linear quadratic Gaussian with loop transfer recovery based autopilot is formulated and integrated with the conventional and more advance guidance laws proposed. A model predictive controller (MPC) has also been devised which is constructed using artificial intelligence techniques such as genetic algorithms (GA) and fuzzy logic. A GA is employed as an online optimization routine whilst fuzzy logic has been exploited as an objective function in an MPC framework. The GA-MPC autopilot has been implemented in Hammerhead in real time and results demonstrate excellent robustness despite the presence of disturbances and ever present modelling uncertainty. To the author's knowledge, this is the first successful application of a GA in real time optimization for controller tuning in the marine sector and thus the thesis makes an extremely novel and useful contribution to control system design in general. The controllers are also integrated with the proposed guidance laws and is also considered to be an invaluable contribution to knowledge. Moreover, the autopilots are used in conjunction with a vision based altitude information sensor and simulation results demonstrate the efficacy of the controllers to cope with uncertain altitude demands.J&S MARINE LTD., QINETIQ, SUBSEA 7 AND SOUTH WEST WATER PL

Deep Reinforcement Learning-Based Mapless Crowd Navigation with Perceived Risk of the Moving Crowd for Mobile Robots

Current state-of-the-art crowd navigation approaches are mainly deep reinforcement learning (DRL)-based. However, DRL-based methods suffer from the issues of generalization and scalability. To overcome these challenges, we propose a method that includes a Collision Probability (CP) in the observation space to give the robot a sense of the level of danger of the moving crowd to help the robot navigate safely through crowds with unseen behaviors. We studied the effects of changing the number of moving obstacles to pay attention during navigation. During training, we generated local waypoints to increase the reward density and improve the learning efficiency of the system. Our approach was developed using deep reinforcement learning (DRL) and trained using the Gazebo simulator in a non-cooperative crowd environment with obstacles moving at randomized speeds and directions. We then evaluated our model on four different crowd-behavior scenarios. The results show that our method achieved a 100% success rate in all test settings. We compared our approach with a current state-of-the-art DRL-based approach, and our approach has performed significantly better, especially in terms of social safety. Importantly, our method can navigate in different crowd behaviors and requires no fine-tuning after being trained once. We further demonstrated the crowd navigation capability of our model in real-world tests.Comment: 6 pages, 7 figure

The effect of surface treatment on composite interface, tensile properties and water absorption of suger palm fiber/polypropylene composites

The rising concern towards environmental issues besides the requirement for more flexible polymer-based material has led to increasing of interest in studying about green composite. Sugar palm fiber (SPF) is a versatile fiber plant employed with wide range of application such as in automotive, packaging and buildings construction. This research was aimed to study the effect of surface treatment on composite interface, tensile properties and water absorption of sugar palm fiber/polypropylene (SPFPP) composite by using different surface treatments such as silane (Si), atmospheric glow discharge plasma (Agd) and maleic anhydride (Ma). Silane treatment was carried out by using immersion method, the Agd plasma was conducted using polymerization and lastly polypropylene grafted maleic anhydride by using melting approach. The SPFPP composite was prepared by using injection moulding with fiber content var­ied from 10-30wt%. The effect of interface enhancement on morphology, mechanical properties and water uptakes of SPFPP composites were then investigated by using FfIR, FESEM, tensile test and water absorption test. Overall, the outcome shows that aJl types of surface treatments had improved the interface of SPFPP composite, thus improving its tensile properties compared to the benchmark untreated SPFPP (Ut­SPFPP) composites and polypropylene. The 30wt% Ma-SPFPP composite shows the highest improvement in tensile properties with 58% and 27% increase in the respective Young's Modulus and tensile strength value compared to Ut-SPFPP composite, while 10wt% Ma-SPFPP composite shows the smallest reduction in elongation compared to Neat PP. On the other hand, the 30wt% Si-SPFPP composite shows the lowest water absorption with 20% reduction respective to Ut-SPFPP composite. In conclusion, the surface treatments have proven succesfull in enhancing the natural fiber-polymer in­terface and improve the tensile properties of SPFPP composite with Ma-SPFPP shows the highest improvement, foJlowed by Agd-SPFPP and Si-SPFPP composites

The effect of surface treatment on composite interface, tensile properties and water absorption of suger palm fiber/polypropylene composites

The rising concern towards environmental issues besides the requirement for more flexible polymer-based material has led to increasing of interest in studying about green composite. Sugar palm fiber (SPF) is a versatile fiber plant employed with wide range of application such as in automotive, packaging and buildings construction. This research was aimed to study the effect of surface treatment on composite interface, tensile properties and water absorption of sugar palm fiber/polypropylene (SPFPP) composite by using different surface treatments such as silane (Si), atmospheric glow discharge plasma (Agd) and maleic anhydride (Ma). Silane treatment was carried out by using immersion method, the Agd plasma was conducted using polymerization and lastly polypropylene grafted maleic anhydride by using melting approach. The SPFPP composite was prepared by using injection moulding with fiber content var­ied from 10-30wt%. The effect of interface enhancement on morphology, mechanical properties and water uptakes of SPFPP composites were then investigated by using FfIR, FESEM, tensile test and water absorption test. Overall, the outcome shows that aJl types of surface treatments had improved the interface of SPFPP composite, thus improving its tensile properties compared to the benchmark untreated SPFPP (Ut­SPFPP) composites and polypropylene. The 30wt% Ma-SPFPP composite shows the highest improvement in tensile properties with 58% and 27% increase in the respective Young's Modulus and tensile strength value compared to Ut-SPFPP composite, while 10wt% Ma-SPFPP composite shows the smallest reduction in elongation compared to Neat PP. On the other hand, the 30wt% Si-SPFPP composite shows the lowest water absorption with 20% reduction respective to Ut-SPFPP composite. In conclusion, the surface treatments have proven succesfull in enhancing the natural fiber-polymer in­terface and improve the tensile properties of SPFPP composite with Ma-SPFPP shows the highest improvement, foJlowed by Agd-SPFPP and Si-SPFPP composites