Adaptive Critic Neural Network-Based Object Grasping Control using a Three-finger Gripper

Grasping of objects has been a challenging task for robots. The complex grasping task can be defined as object contact control and manipulation subtasks. In this paper, object contact control subtask is defined as the ability to follow a trajectory accurately by the fingers of a gripper. The object manipulation subtask is defined in terms of maintaining a predefined applied force by the fingers on the object. A sophisticated controller is necessary since the process of grasping an object without a priori knowledge of the object\u27s size, texture, softness, gripper, and contact dynamics is rather difficult. Moreover, the object has to be secured accurately and considerably fast without damaging it. Since the gripper, contact dynamics, and the object properties are not typically known beforehand, an adaptive critic neural network (NN)-based hybrid position/force control scheme is introduced. The feedforward action generating NN in the adaptive critic NN controller compensates the nonlinear gripper and contact dynamics. The learning of the action generating NN is performed on-line based on a critic NN output signal. The controller ensures that a three-finger gripper tracks a desired trajectory while applying desired forces on the object for manipulation. Novel NN weight tuning updates are derived for the action generating and critic NNs so that Lyapunov-based stability analysis can be shown. Simulation results demonstrate that the proposed scheme successfully allows fingers of a gripper to secure objects without the knowledge of the underlying gripper and contact dynamics of the object compared to conventional schemes

Formation control of mobile robots and unmanned aerial vehicles

In this dissertation, the nonlinear control of nonholonomic mobile robot formations and unmanned aerial vehicle (UAV) formations is undertaken and presented in six papers. In the first paper, an asymptotically stable combined kinematic/torque control law is developed for leader-follower based formation control of mobile robots using backstepping. A neural network (NN) is introduced along with robust integral of the sign of the error (RISE) feedback to approximate the dynamics of the follower as well as its leader using online weight tuning. Subsequently, in the second paper, a novel NN observer is designed to estimate the linear and angular velocities of both the follower and its leader robot and a NN output feedback control law is developed. On the other hand, in the third paper, a NN-based output feedback control law is presented for the control of an underactuated quad rotor UAV, and a NN virtual control input scheme is proposed which allows all six degrees of freedom to be controlled using only four control inputs. The results of this paper are extended to include the control of quadrotor UAV formations, and a novel three-dimensional leader-follower framework is proposed in the fourth paper. Next, in the fifth paper, the discrete-time nonlinear optimal control is undertaken using two online approximators (OLA\u27s) to solve the infinite horizon Hamilton-Jacobi-Bellman (HJB) equation forward-in-time to achieve nearly optimal regulation and tracking control. In contrast, paper six utilizes a single OLA to solve the infinite horizon HJB and Hamilton-Jacobi-Isaacs (HJI) equations forward-intime for the near optimal regulation and tracking control of continuous affine nonlinear systems. The effectiveness of the optimal tracking controllers proposed in the fifth and sixth papers are then demonstrated using nonholonomic mobile robot formation control --Abstract, page iv

Adaptive and extendable control of unmanned surface vehicle formations using distributed deep reinforcement learning

Future ocean exploration will be dominated by a large-scale deployment of marine robots such as unmanned surface vehicles (USVs). Without the involvement of human operators, USVs exploit oceans, especially the complex marine environments, in an unprecedented way with an increased mission efficiency. However, current autonomy level of USVs is still limited, and the majority of vessels are being remotely controlled. To address such an issue, artificial intelligence (AI) such as reinforcement learning can effectively equip USVs with high-level intelligence and consequently achieve full autonomous operation. Also, by adopting the concept of multi-agent intelligence, future trend of USV operations is to use them as a formation fleet. Current researches in USV formation control are largely based upon classical control theories such as PID, backstepping and model predictive control methods with the impact by using advanced AI technologies unclear. This paper, therefore, paves the way in this area by proposing a distributed deep reinforcement learning algorithm for USV formations. More importantly, using the proposed algorithm USV formations can learn two critical abilities, i.e. adaptability and extendibility that enable formations to arbitrarily increase the number of USVs or change formation shapes. The effectiveness of algorithms has been verified and validated through a number of computer-based simulations

A Survey and Analysis of Cooperative Multi-Agent Robot Systems: Challenges and Directions

Research in the area of cooperative multi-agent robot systems has received wide attention among researchers in recent years. The main concern is to find the effective coordination among autonomous agents to perform the task in order to achieve a high quality of overall performance. Therefore, this paper reviewed various selected literatures primarily from recent conference proceedings and journals related to cooperation and coordination of multi-agent robot systems (MARS). The problems, issues, and directions of MARS research have been investigated in the literature reviews. Three main elements of MARS which are the type of agents, control architectures, and communications were discussed thoroughly in the beginning of this paper. A series of problems together with the issues were analyzed and reviewed, which included centralized and decentralized control, consensus, containment, formation, task allocation, intelligences, optimization and communications of multi-agent robots. Since the research in the field of multi-agent robot research is expanding, some issues and future challenges in MARS are recalled, discussed and clarified with future directions. Finally, the paper is concluded with some recommendations with respect to multi-agent systems

Chaotic exploration and learning of locomotion behaviours

We present a general and fully dynamic neural system, which exploits intrinsic chaotic dynamics, for the real-time goal-directed exploration and learning of the possible locomotion patterns of an articulated robot of an arbitrary morphology in an unknown environment. The controller is modeled as a network of neural oscillators that are initially coupled only through physical embodiment, and goal-directed exploration of coordinated motor patterns is achieved by chaotic search using adaptive bifurcation. The phase space of the indirectly coupled neural-body-environment system contains multiple transient or permanent self-organized dynamics, each of which is a candidate for a locomotion behavior. The adaptive bifurcation enables the system orbit to wander through various phase-coordinated states, using its intrinsic chaotic dynamics as a driving force, and stabilizes on to one of the states matching the given goal criteria. In order to improve the sustainability of useful transient patterns, sensory homeostasis has been introduced, which results in an increased diversity of motor outputs, thus achieving multiscale exploration. A rhythmic pattern discovered by this process is memorized and sustained by changing the wiring between initially disconnected oscillators using an adaptive synchronization method. Our results show that the novel neurorobotic system is able to create and learn multiple locomotion behaviors for a wide range of body configurations and physical environments and can readapt in realtime after sustaining damage

Sample-Efficient Reinforcement Learning of Robot Control Policies in the Real World

abstract: The goal of reinforcement learning is to enable systems to autonomously solve tasks in the real world, even in the absence of prior data. To succeed in such situations, reinforcement learning algorithms collect new experience through interactions with the environment to further the learning process. The behaviour is optimized by maximizing a reward function, which assigns high numerical values to desired behaviours. Especially in robotics, such interactions with the environment are expensive in terms of the required execution time, human involvement, and mechanical degradation of the system itself. Therefore, this thesis aims to introduce sample-efficient reinforcement learning methods which are applicable to real-world settings and control tasks such as bimanual manipulation and locomotion. Sample efficiency is achieved through directed exploration, either by using dimensionality reduction or trajectory optimization methods. Finally, it is demonstrated how data-efficient reinforcement learning methods can be used to optimize the behaviour and morphology of robots at the same time.Dissertation/ThesisDoctoral Dissertation Computer Science 201