Autonomous Drone Landings on an Unmanned Marine Vehicle using Deep Reinforcement Learning

This thesis describes with the integration of an Unmanned Surface Vehicle (USV) and an Unmanned Aerial Vehicle (UAV, also commonly known as drone) in a single Multi-Agent System (MAS). In marine robotics, the advantage offered by a MAS consists of exploiting the key features of a single robot to compensate for the shortcomings in the other. In this way, a USV can serve as the landing platform to alleviate the need for a UAV to be airborne for long periods time, whilst the latter can increase the overall environmental awareness thanks to the possibility to cover large portions of the prevailing environment with a camera (or more than one) mounted on it. There are numerous potential applications in which this system can be used, such as deployment in search and rescue missions, water and coastal monitoring, and reconnaissance and force protection, to name but a few. The theory developed is of a general nature. The landing manoeuvre has been accomplished mainly identifying, through artificial vision techniques, a fiducial marker placed on a flat surface serving as a landing platform. The raison d'etre for the thesis was to propose a new solution for autonomous landing that relies solely on onboard sensors and with minimum or no communications between the vehicles. To this end, initial work solved the problem while using only data from the cameras mounted on the in-flight drone. In the situation in which the tracking of the marker is interrupted, the current position of the USV is estimated and integrated into the control commands. The limitations of classic control theory used in this approached suggested the need for a new solution that empowered the flexibility of intelligent methods, such as fuzzy logic or artificial neural networks. The recent achievements obtained by deep reinforcement learning (DRL) techniques in end-to-end control in playing the Atari video-games suite represented a fascinating while challenging new way to see and address the landing problem. Therefore, novel architectures were designed for approximating the action-value function of a Q-learning algorithm and used to map raw input observation to high-level navigation actions. In this way, the UAV learnt how to land from high latitude without any human supervision, using only low-resolution grey-scale images and with a level of accuracy and robustness. Both the approaches have been implemented on a simulated test-bed based on Gazebo simulator and the model of the Parrot AR-Drone. The solution based on DRL was further verified experimentally using the Parrot Bebop 2 in a series of trials. The outcomes demonstrate that both these innovative methods are both feasible and practicable, not only in an outdoor marine scenario but also in indoor ones as well

Using a Combination of PID Control and Kalman Filter to Design of IoT-based Telepresence Self-balancing Robots during COVID-19 Pandemic

COVID-19 is a very dangerous respiratory disease that can spread quickly through the air. Doctors, nurses, and medical personnel need protective clothing and are very careful in treating COVID-19 patients to avoid getting infected with the COVID-19 virus. Hence, a medical telepresence robot, which resembles a humanoid robot, is necessary to treat COVID-19 patients. The proposed self-balancing COVID-19 medical telepresence robot is a medical robot that handles COVID-19 patients, which resembles a stand-alone humanoid soccer robot with two wheels that can maneuver freely in hospital hallways. The proposed robot design has some control problems; it requires steady body positioning and is subjected to disturbance. A control method that functions to find the stability value such that the system response can reach the set-point is required to control the robot's stability and repel disturbances; this is known as disturbance rejection control. This study aimed to control the robot using a combination of Proportional-Integral-Derivative (PID) control and a Kalman filter. Mathematical equations were required to obtain a model of the robot's characteristics. The state-space model was derived from the self-balancing robot's mathematical equation. Since a PID control technique was used to keep the robot balanced, this state-space model was converted into a transfer function model. The second Ziegler-Nichols's rule oscillation method was used to tune the PID parameters. The values of the amplifier constants obtained were Kp=31.002, Ki=5.167, and Kd=125.992128. The robot was designed to be able to maintain its balance for more than one hour by using constant tuning, even when an external disturbance is applied to it. Doi: 10.28991/esj-2021-SP1-016 Full Text: PD

Robotics 2010

Without a doubt, robotics has made an incredible progress over the last decades. The vision of developing, designing and creating technical systems that help humans to achieve hard and complex tasks, has intelligently led to an incredible variety of solutions. There are barely technical fields that could exhibit more interdisciplinary interconnections like robotics. This fact is generated by highly complex challenges imposed by robotic systems, especially the requirement on intelligent and autonomous operation. This book tries to give an insight into the evolutionary process that takes place in robotics. It provides articles covering a wide range of this exciting area. The progress of technical challenges and concepts may illuminate the relationship between developments that seem to be completely different at first sight. The robotics remains an exciting scientific and engineering field. The community looks optimistically ahead and also looks forward for the future challenges and new development

Advances in Intelligent Robotics and Collaborative Automation

This book provides an overview of a series of advanced research lines in robotics as well as of design and development methodologies for intelligent robots and their intelligent components. It represents a selection of extended versions of the best papers presented at the Seventh IEEE International Workshop on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications IDAACS 2013 that were related to these topics. Its contents integrate state of the art computational intelligence based techniques for automatic robot control to novel distributed sensing and data integration methodologies that can be applied to intelligent robotics and automation systems. The objective of the text was to provide an overview of some of the problems in the field of robotic systems and intelligent automation and the approaches and techniques that relevant research groups within this area are employing to try to solve them.The contributions of the different authors have been grouped into four main sections:• Robots• Control and Intelligence• Sensing• Collaborative automationThe chapters have been structured to provide an easy to follow introduction to the topics that are addressed, including the most relevant references, so that anyone interested in this field can get started in the area

Advances in Intelligent Robotics and Collaborative Automation

This book provides an overview of a series of advanced research lines in robotics as well as of design and development methodologies for intelligent robots and their intelligent components. It represents a selection of extended versions of the best papers presented at the Seventh IEEE International Workshop on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications IDAACS 2013 that were related to these topics. Its contents integrate state of the art computational intelligence based techniques for automatic robot control to novel distributed sensing and data integration methodologies that can be applied to intelligent robotics and automation systems. The objective of the text was to provide an overview of some of the problems in the field of robotic systems and intelligent automation and the approaches and techniques that relevant research groups within this area are employing to try to solve them.The contributions of the different authors have been grouped into four main sections:• Robots• Control and Intelligence• Sensing• Collaborative automationThe chapters have been structured to provide an easy to follow introduction to the topics that are addressed, including the most relevant references, so that anyone interested in this field can get started in the area

Fall Prediction and Controlled Fall for Humanoid Robots

Humanoids which resemble humans in their body structure and degrees of freedom are anticipated to work like them within infrastructures and environments constructed for humans. In such scenarios, even humans who have exceptional manipulation, balancing, and locomotion skills are vulnerable to fall, humanoids being their approximate imitators are no exception to this. Furthermore, their high center of gravity position in relation to their small support polygon makes them more prone to fall, unlike other robots such as quadrupeds. The consequences of these falls are so devastating that it can instantly annihilate both the robot and its surroundings. This has become one of the major stumbling blocks which humanoids have to overcome to operate in real environments. As a result, in this thesis, we have strived to address the imminent fall over of humanoids by developing different control techniques. The fall over problem as such can be divided into three subissues: fall prediction, controlled fall, and its recovery. In the presented work, the first two issues have been addressed, and they are presented in three parts. First, we define what is fall over for humanoids, different sources for it to happen, the effect fall over has both on the robot and to its surroundings, and how to deal with them. Following which, we give a brief introduction to the overall system which includes both the hardware and software components which have been used throughout the work for varied purposes. Second, the first sub-issue is addressed by proposing a generic method to predict the falling over of humanoid robots in a reliable, robust, and agile manner across various terrains, and also amidst arbitrary disturbances. The aforementioned characteristics are strived to attain by proposing a prediction principle inspired by the human balance sensory systems. Accordingly, the fusion of multiple sensors such as inertial measurement unit and gyroscope (IMU), foot pressure sensor (FPS), joint encoders, and stereo vision sensor, which are equivalent to the human\u2019s vestibular, proprioception, and vision systems are considered. We first define a set of feature-based fall indicator variables (FIVs) from the different sensors, and the thresholds for those FIVs are extracted analytically for four major disturbance scenarios. Further, an online threshold interpolation technique and an impulse adaptive counter limit are proposed to manage more generic disturbances. For the generalized prediction process, both the instantaneous and cumulative sum of each FIVs are normalized, and a suitable value is set as the critical limit to predict the fall over. To determine the best combination and the usefulness of multiple sensors, the prediction performance is evaluated on four different types of terrains, in three unique combinations: first, each feature individually with their respective FIVs; second, an intuitive performance based (PF); and finally, Kalman filter based (KF) techniques, which involve the usage of multiple features. For PF and KF techniques, prediction performance evaluations are carried out with and without adding noise. Overall, it is reported that KF performs better than PF and individual sensor features under different conditions. Also, the method\u2019s ability to predict fall overs during the robot\u2019s simple dynamic motion is also tested and verified through simulations. Experimental verification of the proposed prediction method on flat and uneven terrains was carried out with the WALK-MAN humanoid robot. Finally, in reference to the second sub-issue, i.e., the controlled fall, we propose two novel fall control techniques based on energy concepts, which can be applied online to mitigate the impact forces incurred during the falling over of humanoids. Both the techniques are inspired by the break-fall motions, in particular, Ukemi motion practiced by martial arts people. The first technique reduces the total energy using a nonlinear control tool, called energy shaping (ES) and further distributes the reduced energy over multiple contacts by means of energy distribution polygons (EDP). We also include an effective orientation control to safeguard the end-effectors in the event of ground impacts. The performance of the proposed method is numerically evaluated by dynamic simulations under the sudden falling over scenario of the humanoid robot for both lateral and sagittal falls. The effectiveness of the proposed ES and EDP concepts are verified by diverse comparative simulations regarding total energy, distribution, and impact forces. Following the first technique, we proposed another controller to generate an online rolling over motion based on the hypothesis that multi-contact motions can reduce the impact forces even further. To generate efficient rolling motion, critical parameters are defined by the insights drawn from a study on rolling, which are contact positions and attack angles. In addition, energy-injection velocity is proposed as an auxiliary rolling parameter to ensure sequential multiple contacts in rolling. An online rolling controller is synthesized to compute the optimal values of the rolling parameters. The first two parameters are to construct a polyhedron, by selecting suitable contacts around the humanoid\u2019s body. This polyhedron distributes the energy gradually across multiple contacts, thus called energy distribution polyhedron. The last parameter is to inject some additional energy into the system during the fall, to overcome energy drought and tip over successive contacts. The proposed controller, incorporated with energy injection, minimization, and distribution techniques result in a rolling like motion and significantly reduces the impact forces, and it is verified in numerical experiments with a segmented planar robot and a full humanoid model

RoboBat: dynamics and control of flapping flight micro aerial vehicles

Flapping flight micro aerial vehicles (MAVs) are of interest to the aerospace and robotics communities for their maneuverability in comparison to tradition fixed wing and rotary aircraft. However they present numerous challenges in the fields of dynamics, stability and control. This thesis examines the dynamics and kinematics of robotic flapping flight, the design and construction of a robotic bat test bed mounted on a 3-DOF pendulum, and subsequent control experiments using the test bed. The robotic bat test bed is capable of exhibiting different wing motions and is used to test the feasibility of controlling the motions of the robotic bat by using the phase differences between coupled nonlinear oscillators called central pattern generators (CPGs). A dynamic model for the robotic bat based on the complex wing kinematics is presented, and the wing kinematic motions themselves are analyzed using a high-speed motion capture system. Mechanical coupling effects which deviate from theoretical assumptions are investigated as well. Open loop experiments analyzing the steady state behavior of the bat's flight with varying phase differences showed a change of the pitch angle while elevation and forward velocity remains constant. Closed loop experiments indeed validate that control dimension reduction is achievable by controlling the phase differences of CPG oscillators. Unstable longitudinal modes are stabilized and controlled using only control of two parameters: phase difference and flapping frequency. Transition between flapping flight and gliding flight is analyzed. This shows promising results regarding the relation between phase differences of wing motions and longitudinal stability, and lays the groundwork for future research and experimentation in flapping flight MAVs

Human-Inspired Balancing and Recovery Stepping for Humanoid Robots

Robustly maintaining balance on two legs is an important challenge for humanoid robots. The work presented in this book represents a contribution to this area. It investigates efficient methods for the decision-making from internal sensors about whether and where to step, several improvements to efficient whole-body postural balancing methods, and proposes and evaluates a novel method for efficient recovery step generation, leveraging human examples and simulation-based reinforcement learning