Vision based leader-follower formation control for mobile robots

Creating systems with multiple autonomous vehicles places severe demands on the design of control schemes. Robot formation control plays a vital role in coordinating robots. As the number of members in a system rise, the complexity of each member increases. There is a proportional increase in the quantity and complexity of onboard sensing, control and computation. This thesis investigates the control of a group of mobile robots consisting of a leader and several followers to maintain a desired geometric formation --Abstract, page iii

Object Conveyance Algorithm for Multiple Mobile Robots based on Object Shape and Size

This paper describes a determination method of a number of a team for multiple mobile robot object conveyance. The number of robot on multiple mobile robot systems is the factor of complexity on robots formation and motion control. In our previous research, we verified the use of the complex-valued neural network for controlling multiple mobile robots in object conveyance problem. Though it is a significant issue to develop effective determination team member for multiple mobile robot object conveyance, few studies have been done on it. Therefore, we propose an algorithm for determining the number of the team member on multiple mobile robot object conveyance with grasping push. The team member is determined based on object weight to obtain appropriate formation. First, the object shape and size measurement is carried out by a surveyor robot that approaches and surrounds the object. During surrounding the object, the surveyor robot measures its distance to the object and records for estimating the object shape and size. Since the object shape and size are estimated, the surveyor robot makes initial push position on the estimated push point and calls additional robots for cooperative push. The algorithm is validated in several computer simulations with varying object shape and size. As a result, the proposed algorithm is promising for minimizing the number of the robot on multiple mobile robot object conveyance

ENG 1001G-012: Composition and Language

Koordiniranim gibanjem više mobilnih robota umjesto jednoga može se postići značajno povećanje brzine, točnosti i učinkovitosti izvođenja zadataka u velikom broju primjena. Primjeri takvih primjena mogu se naći u inteligentnim transportnim sustavima, zadacima prekrivanja prostora, kooperativnom prijenosu tereta, logističkim sustavima, proizvodnim pogonima itd. Sigurno gibanje mobilnih robota u formaciji konvoj u inteligentnim transportnim sustavima moguće je ako se zadovolji stabilnost konvoja. Stabilnost konvoja modela razlike brzine postignuta je odgođenim povratnim informacijama koje uključuje razliku trenutačne i odgođene udaljenosti između mobilnih robota i razliku trenutačne i odgođene brzine mobilnog robota. Postizanje izvršenja zadataka prekrivanja prostora i kooperativnog prijenosa tereta u najkraćem mogućem vremenu vrlo je izazovan problem. Riješen je planiranjem referentne vremenski optimalne trajektorije formacije mobilnih robota u zakrivljenom koordinatnom sustavu pomoću algoritma optimalnog skaliranja u vremenu, koji vremenski optimalno izmjenjuje najveća i najmanja ubrzanja formacije. Koordinirano gibanje više mobilnih robota u istom radnom prostoru u logističkim sustavima i proizvodnim pogonima ostvareno je primjenom pristupa razdvojenog planiranja gibanja uz unaprijed definirane putanje mobilnih robota i dodavanjem početnog vremena kašnjenja izračunanim predefiniranim trajektorijama mobilnih robota. Slijeđenje zadanih trajektorija za mobilne robote s diferencijalnim pogonom postignuto je algoritmom s nepromjenjivim koeficijentima zasnovanim na kinematičkom modelu mobilnog robota, nelinearnoj dinamici pogreške slijeđenja i Lyapunovljevoj teoriji stabilnosti.Nowadays, the use of mobile robots has become increasingly important for many purposes: medical services, civil transport, domestic work, military, commercial cleaning, sales of consumer goods, agricultural and forestry work, browsing hard to reach and dangerous areas for human, digging ore, construction work, loading/unloading and manipulating materials, outer space and underwater research, space supervision, entertainment etc. New challenges in the field of mobile robotics include controlling a group of mobile robots in an efficient way. Inspiration is found in nature where many animal communities apply cooperative behaviour patterns to achieve a common goal. The most common arguments for the application of a group of mobile robots in comparison to only one mobile robot are increase in speed and accuracy and efficiency of performing tasks. A group of mobile robots is able to execute tasks impossible for a single mobile robot, e.g. transporting or repositioning large objects. Also, a group of mobile robots is more robust to failure than a single mobile robot (one mobile robot can take over the tasks of another mobile robot in case of failure). Since mobile robots can have a variety of roles, the same group of mobile robots can be employed for many different objectives, e.g. intelligent transport systems, areas coverage, cooperative transportation, logistics, etc. In order to achieve the benefits of multi-robot mobile systems it is necessary to solve additional problems in terms of control, which are not present in systems with a single mobile robot, e.g. communication between robots, distribution of tasks, assigning priorities, taking into account kinematic and dynamic constraints of all the mobile robots in the group to successfully plan feasible paths and trajectories for all of them, etc. A mobile robot is a mechanical system and as such is subject to motion equations that follow the laws of physics. Therefore, for each movement, in accordance with kinematic and dynamic constraints, there must be at least one set of input values that affect the motion. Kinematic constraints of mobile robots are the result of limitations in movement of drive wheels and drive configurations. Dynamic constraints of mobile robots refer to limiting the permitted velocity and acceleration, and they are caused by the actuator limitations. Motion control of mobile robots is a complex process. It includes working task planning, reference path and trajectory planning, and tracking the reference trajectory. The focus of this thesis is put on multi-robot land mobile systems and applications in intelligent traffic systems, areas coverage, cooperative transportation of large objects and coordination in the common working environment. All algorithms presented in this thesis are first tested in Matlab® and then experimentally validated on real robots. Experiments were performed using robot soccer platform at the Department of Control and Computer Engineering, Faculty of Electrical Engineering and Computing, University of Zagreb, which is ideal for testing various mobile robot algorithms. It consists of a team of five radio-controlled micro robots of size 0.075 m cubed with differential drive. The playground is of size 2.2×1.8 m. Above the centre of the playground, Basler a301fc IEEE-1394 Bayer digital colour camera with resolution of 656×494 pixels and with maximal frame rate of 80 fps is mounted perpendicular to the playground. The height of the camera to the playground is 2.40 m. A wide angle 6 mm lens is used. Although robot soccer platform is very practical for experiments with formations of mobile robots, it has some technical limitations: (i) relatively large noise in the measured position and velocity of the robot; (ii) delay in the communication between the control computer and microprocessors of the mobile robots and (iii) delay in measurements due to vision (the time required to grab the image from the camera and the time required for image processing). Efficient motion of mobile robots (vehicles) in a convoy formation is very important in transportation of people and goods. The key research problem is ensuring the string stability. In the background of this study are traffic safety and adaptive cruise control system in modern vehicles. The adaptive cruise control system primarily aims to reduce the driver’s effort, which is achieved by controlling the velocity of vehicle according to a predefined control law. Typically, sensors such as radar and lidar are used to measure the relative distance and relative velocity between vehicles. The working principle of adaptive cruise control system is based on these informations. String stability refers to the stability of a series of ''interconnected'' vehicles. The attribute ''interconnected'' does not indicate the physical connection of the vehicles but vehicles moving in a convoy formation where every vehicle in the convoy follows the preceding vehicle at a safe distance. The behaviour of each vehicle in the convoy must be such that the oscillatory behaviour due to a change of velocity of the leading vehicle of the convoy does not increase towards the end of the convoy. Otherwise, unstable convoy may cause collisions between vehicles. In this thesis, a deterministic microscopic full velocity difference model is considered. In the case where the values of model parameters do not meet the requirement of string stability, full velocity difference model should be expanded with additional control signals in order to satisfy string stability. Two additional control signals are based on the difference of the current and the delayed (in a defined time in the past) relative distance between the vehicle, and the difference of the current and the delayed vehicle velocity. The proposed method for achieving string stability is based on the delayed-feedback control. The delayed-feedback control is one of the feasible methods of controlling unstable and chaotic motions. The string stability has been examined by ∞–norm of transfer function of distance between vehicles. The stability of the transfer function is examined using a delay-independent stability criterion, which significantly simplifies the stability test, since characteristic equation of transfer function is transcendental and has infinite number of roots. It should be noted that the adaptive cruise control system is studied in one-dimensional space, e.g. vehicles in the convoy moving on an infinitely long straight road. Using formations of mobile robots, it is possible to significantly increase efficiency of numerous tasks such as areas coverage or cooperative transportation of large objects. However, achieving minimal time of executing tasks is a very challenging problem. The formation of mobile robots observed in this study is most similar to the formation with a leading mobile robot, but instead of a leading mobile robot the reference point of the formation and its reference path are defined. The formation of mobile robots is defined in the curved coordinate system so mobile robots maintain distance in relation to a reference point formation in this coordinate system. The curvature of the coordinate system is actually instantaneous centre of curvature of the formation’s reference trajectory. This results in changing the formation shape during cornering. This has implications in possible applications. For example, a square formation of mobile robots cannot handle rectangular solid object as it moves in curved path. The dynamic model of the mobile robot takes into account intrinsic constraints originating from the robot actuator limits and extrinsic constraints resulting from the limited adhesion force between ground and wheels of the mobile robot such as wheel slip and tip over of the mobile robot. Both types of constraints are important for planning at high velocities. The problem of planning reference time-optimal trajectory for a formation of mobile robot is solved by decoupled approach which could be defined as a problem of planning reference time-optimal trajectories after predefined smooth G2 continuous path of the reference point of formation. First, a path of formation of mobile robots in workspace is found, and then a velocity profile in accordance with the specified criteria and dynamic constraints of mobile robots is calculated. It is assumed that the path of each mobile robot in the formation is feasible and that kinematic constraints of the mobile robot on the path are satisfied. G2 continuous path provides the ability to pass the mobile robot trajectory with a non-zero velocity, thus enabling fast movement of the mobile robot without stopping. G2 path consists of straight lines and clothoids. Planning reference time-optimal trajectory for the formation of mobile robots is based on the optimal time-scaling algorithm providing that formation always moves at its maximum or minimum accelerations. This means that at least one mobile robot of the formation moves at its maximum or minimum accelerations. Emphasis is placed on static formations of mobile robots. Also, one example of dynamic formation of the mobile robots was shown. When multiple mobile robots independently perform tasks in the same workspace, the key problem is the planning collision free coordinated motion of multiple mobile robots sharing the same workspace. This problem is solved using the decoupled approach. First step is to plan individual path of each mobile robot, e.g. predefined path, using methods for planning paths for individual mobile robots in workspace with static obstacles. Second step is to plan a velocity profile for each mobile robot on its predefined path, e.g. predefined trajectory. Third step is to modify predefined trajectory for each mobile robot making sure that collisions between mobile robots in the workspace are avoided. To successfully coordinate motion of multiple mobile robots, a common approach is to assign priority level to each of them. A mobile robot with highest priority takes into account only static obstacles while other mobile robots have to take into account also dynamic obstacles, which are mobile robots with higher priorities. Dynamic obstacles avoidance is based on avoiding time-obstacles in a collision map. Time-obstacles are constructed based on the time mobile robots would spend in possible area of collision. A lower priority mobile robot efficiently avoids time-obstacles, e.g. mobile robot with higher priority level, by inserting a calculated start-up delay time to its predefined trajectory along the predefined path. By applying the same principle to all lower priority robots, a collision free motion coordination of multiple mobile robots can be achieved. It should be noted that the predefined paths of mobile robots do not change during their movements. The change is only in the allocation of time of motion of mobile robots on their predefined paths. A proposed method is computationally fast and intuitive and ensures that always only one mobile robot can be in the possible area of collision. The input of the trajectory tracking algorithm is a feasible planned trajectory, where feasibility refers to the ability of a mobile robot to actually track the planned trajectory. This means that the planned trajectory respects kinematic and dynamic constraints of the mobile robot. The trajectory tracking algorithm is needed because in reality there are many sources of potential errors such as imperfections of a mobile robot model or external disturbances like uneven ground, delayed command control, an imperfect measurement of the state of mobile robots and so on. In this thesis, it is proposed a trajectory tracking algorithm with constant gains for nonholonomic mobile robot with differential drive, which is based on kinematic model of mobile robot, nonlinear dynamics of the tracking error and Lyapunov stability theory. The scientific contributions of the thesis are: 1. Algorithm for mobile robot control in the formation of a convoy that ensures string stability using the delayed-feedback control, based on the difference between current and delayed distance between mobile robots and the difference between current and delayed velocity of a mobile robot. 2. Algorithm for planning smooth time optimal trajectory for a formation of mobile robots with kinematic and dynamic constraints on predefined paths. 3. Algorithm for planning of collision free motion coordination of multiple mobile robots sharing the same workspace on predefined paths assigning initial delay time for predefined trajectories of mobile robots. 4. Algorithm for trajectory tracking for mobile robots with kinematic and dynamic constraints based on Lyapunov stability theory

Safe Multi-Agent Reinforcement Learning for Formation Control without Individual Reference Targets

In recent years, formation control of unmanned vehicles has received considerable interest, driven by the progress in autonomous systems and the imperative for multiple vehicles to carry out diverse missions. In this paper, we address the problem of behavior-based formation control of mobile robots, where we use safe multi-agent reinforcement learning~(MARL) to ensure the safety of the robots by eliminating all collisions during training and execution. To ensure safety, we implemented distributed model predictive control safety filters to override unsafe actions. We focus on achieving behavior-based formation without having individual reference targets for the robots, and instead use targets for the centroid of the formation. This formulation facilitates the deployment of formation control on real robots and improves the scalability of our approach to more robots. The task cannot be addressed through optimization-based controllers without specific individual reference targets for the robots and information about the relative locations of each robot to the others. That is why, for our formulation we use MARL to train the robots. Moreover, in order to account for the interactions between the agents, we use attention-based critics to improve the training process. We train the agents in simulation and later on demonstrate the resulting behavior of our approach on real Turtlebot robots. We show that despite the agents having very limited information, we can still safely achieve the desired behavior.Comment: Submitted to IEEE Transaction on Robotic

Formation Control of Nonholonomic Multi-Agent Systems

This dissertation is concerned with the formation control problem of multiple agents modeled as nonholonomic wheeled mobile robots. Both kinematic and dynamic robot models are considered. Solutions are presented for a class of formation problems that include formation, maneuvering, and flocking. Graph theory and nonlinear systems theory are the key tools used in the design and stability analysis of the proposed control schemes. Simulation and/or experimental results are presented to illustrate the performance of the controllers. In the first part, we present a leader-follower type solution to the formation maneuvering problem. The solution is based on the graph that models the coordination among the robots being a spanning tree. Our control law incorporates two types of position errors: individual tracking errors and coordination errors for leader-follower pairs in the spanning tree. The control ensures that the robots globally acquire a given planar formation while the formation as a whole globally tracks a desired trajectory, both with uniformly ultimately bounded errors. The control law is first designed at the kinematic level and then extended to the dynamic level. In the latter, we consider that parametric uncertainty exists in the equations of motion. These uncertainties are accounted for by employing an adaptive control scheme. In the second part, we design a distance-based control scheme for the flocking of the nonholonomic agents under the assumption that the desired flocking velocity is known to all agents. The control law is designed at the kinematic level and is based on the rigidity properties of the graph modeling the sensing/control interactions among the robots. A simple input transformation is used to facilitate the control design by converting the nonholonomic model into the single-integrator equation. The resulting control ensures exponential convergence to the desired formation while the formation maneuvers according to a desired, time-varying translational velocity. In the third part, we extend the previous flocking control framework to the case where only a subset of the agents know the desired flocking velocity. The resulting controllers include distributed observers to estimate the unknown quantities. The theory of interconnected systems is used to analyze the stability of the observer-controller system

Energetic swarm control with application to multiple vehicle systems

Control and coordination of multiple vehicle systems has been a very active area of research in recent years. Recent advancements in computation, communication, and mechatronics have allowed the development of large groups of vehicles, often referred to as swarms, in order to accomplish complex missions over large areas with redundant fault tolerant capabilities. Existing swarm control work has addressed swarm aggregation, foraging swarms, swarm formation, and swarms that track and enclose targets. Energetic swarm control is another significant recent contribution to the swarm control literature. It allows the control of the internal kinetic energy and potential kinetic energy of the swarm system in order to achieve tasks such as sweeping an area, patrolling, and area coverage. This thesis involves the application of energetic swarm control to wheeled mobile robots. A lower level control layer for wheeled mobile robots, based on feedback linearization, is developed and combined with a higher level particle based energetic swarm controller. Furthermore, input saturation constraints are addressed using a suitable control allocation approach. An experimentally verified model of a wheeled mobile robot is developed and used to demonstrate the capabilities of the new energetic swarm control approach for wheeled mobile robots

Collaborative autonomy in heterogeneous multi-robot systems

As autonomous mobile robots become increasingly connected and widely deployed in different domains, managing multiple robots and their interaction is key to the future of ubiquitous autonomous systems. Indeed, robots are not individual entities anymore. Instead, many robots today are deployed as part of larger fleets or in teams. The benefits of multirobot collaboration, specially in heterogeneous groups, are multiple. Significantly higher degrees of situational awareness and understanding of their environment can be achieved when robots with different operational capabilities are deployed together. Examples of this include the Perseverance rover and the Ingenuity helicopter that NASA has deployed in Mars, or the highly heterogeneous robot teams that explored caves and other complex environments during the last DARPA Sub-T competition. This thesis delves into the wide topic of collaborative autonomy in multi-robot systems, encompassing some of the key elements required for achieving robust collaboration: solving collaborative decision-making problems; securing their operation, management and interaction; providing means for autonomous coordination in space and accurate global or relative state estimation; and achieving collaborative situational awareness through distributed perception and cooperative planning. The thesis covers novel formation control algorithms, and new ways to achieve accurate absolute or relative localization within multi-robot systems. It also explores the potential of distributed ledger technologies as an underlying framework to achieve collaborative decision-making in distributed robotic systems. Throughout the thesis, I introduce novel approaches to utilizing cryptographic elements and blockchain technology for securing the operation of autonomous robots, showing that sensor data and mission instructions can be validated in an end-to-end manner. I then shift the focus to localization and coordination, studying ultra-wideband (UWB) radios and their potential. I show how UWB-based ranging and localization can enable aerial robots to operate in GNSS-denied environments, with a study of the constraints and limitations. I also study the potential of UWB-based relative localization between aerial and ground robots for more accurate positioning in areas where GNSS signals degrade. In terms of coordination, I introduce two new algorithms for formation control that require zero to minimal communication, if enough degree of awareness of neighbor robots is available. These algorithms are validated in simulation and real-world experiments. The thesis concludes with the integration of a new approach to cooperative path planning algorithms and UWB-based relative localization for dense scene reconstruction using lidar and vision sensors in ground and aerial robots

Vision-based control of multi-agent systems

Scope and Methodology of Study: Creating systems with multiple autonomous vehicles places severe demands on the design of decision-making supervisors, cooperative control schemes, and communication strategies. In last years, several approaches have been developed in the literature. Most of them solve the vehicle coordination problem assuming some kind of communications between team members. However, communications make the group sensitive to failure and restrict the applicability of the controllers to teams of friendly robots. This dissertation deals with the problem of designing decentralized controllers that use just local sensor information to achieve some group goals.Findings and Conclusions: This dissertation presents a decentralized architecture for vision-based stabilization of unmanned vehicles moving in formation. The architecture consists of two main components: (i) a vision system, and (ii) vision-based control algorithms. The vision system is capable of recognizing and localizing robots. It is a model-based scheme composed of three main components: image acquisition and processing, robot identification, and pose estimation.Using vision information, we address the problem of stabilizing groups of mobile robots in leader- or two leader-follower formations. The strategies use relative pose between a robot and its designated leader or leaders to achieve formation objectives. Several leader-follower formation control algorithms, which ensure asymptotic coordinated motion, are described and compared. Lyapunov's stability theory-based analysis and numerical simulations in a realistic tridimensional environment show the stability properties of the control approaches

Bounded Distributed Flocking Control of Nonholonomic Mobile Robots

There have been numerous studies on the problem of flocking control for multiagent systems whose simplified models are presented in terms of point-mass elements. Meanwhile, full dynamic models pose some challenging problems in addressing the flocking control problem of mobile robots due to their nonholonomic dynamic properties. Taking practical constraints into consideration, we propose a novel approach to distributed flocking control of nonholonomic mobile robots by bounded feedback. The flocking control objectives consist of velocity consensus, collision avoidance, and cohesion maintenance among mobile robots. A flocking control protocol which is based on the information of neighbor mobile robots is constructed. The theoretical analysis is conducted with the help of a Lyapunov-like function and graph theory. Simulation results are shown to demonstrate the efficacy of the proposed distributed flocking control scheme