Automating Vehicles by Deep Reinforcement Learning using Task Separation with Hill Climbing

Within the context of autonomous driving a model-based reinforcement learning algorithm is proposed for the design of neural network-parameterized controllers. Classical model-based control methods, which include sampling- and lattice-based algorithms and model predictive control, suffer from the trade-off between model complexity and computational burden required for the online solution of expensive optimization or search problems at every short sampling time. To circumvent this trade-off, a 2-step procedure is motivated: first learning of a controller during offline training based on an arbitrarily complicated mathematical system model, before online fast feedforward evaluation of the trained controller. The contribution of this paper is the proposition of a simple gradient-free and model-based algorithm for deep reinforcement learning using task separation with hill climbing (TSHC). In particular, (i) simultaneous training on separate deterministic tasks with the purpose of encoding many motion primitives in a neural network, and (ii) the employment of maximally sparse rewards in combination with virtual velocity constraints (VVCs) in setpoint proximity are advocated.Comment: 10 pages, 6 figures, 1 tabl

Distributed coordinate tracking control of multiple wheeled mobile robots

In this thesis, distributed coordinate tracking control of multiple wheeled-mobile robots is studied. Control algorithms are proposed for both kinematic and dynamic models. All vehicle agents share the same mechanical structure. The communication topology is leader-follower topology and the reference signal is generated by the virtual leader. We will introduce two common kinematic models of WMR and control algorithms are proposed for both kinematic models with the aid of graph theory. Since it is more realistic that the control inputs are torques so dynamic extension is studied following by the kinematics. Torque controllers are designed with the aid of backstepping method so that the velocities of the mobile robots converge to the desired velocities. Because of the fact that in practice, the inertial parameter of WMR maybe not exactly known or even unknown, so both dynamics with and without inertial uncertainties are considered in this thesis

Self-Learning Variable Structure Control for a Class of Sensor-Actuator Systems

Variable structure strategy is widely used for the control of sensor-actuator systems modeled by Euler-Lagrange equations. However, accurate knowledge on the model structure and model parameters are often required for the control design. In this paper, we consider model-free variable structure control of a class of sensor-actuator systems, where only the online input and output of the system are available while the mathematic model of the system is unknown. The problem is formulated from an optimal control perspective and the implicit form of the control law are analytically obtained by using the principle of optimality. The control law and the optimal cost function are explicitly solved iteratively. Simulations demonstrate the effectiveness and the efficiency of the proposed method

Research on a semiautonomous mobile robot for loosely structured environments focused on transporting mail trolleys

In this thesis is presented a novel approach to model, control, and planning the motion of a nonholonomic wheeled mobile robot that applies stable pushes and pulls to a nonholonomic cart (York mail trolley) in a loosely structured environment. The method is based on grasping and ungrasping the nonholonomic cart, as a result, the robot changes its kinematics properties. In consequence, two robot configurations are produced by the task of grasping and ungrasping the load, they are: the single-robot configuration and the robot-trolley configuration. Furthermore, in order to comply with the general planar motion law of rigid bodies and the kinematic constraints imposed by the robot wheels for each configuration, the robot has been provided with two motorized steerable wheels in order to have a flexible platform able to adapt to these restrictions. [Continues.

Tracking control of redundant mobile manipulator: An RNN based metaheuristic approach

In this paper, we propose a topology of Recurrent Neural Network (RNN) based on a metaheuristic optimization algorithm for the tracking control of mobile-manipulator while enforcing nonholonomic constraints. Traditional approaches for tracking control of mobile robots usually require the computation of Jacobian-inverse or linearization of its mathematical model. The proposed algorithm uses a nature-inspired optimization approach to directly solve the nonlinear optimization problem without any further transformation. First, we formulate the tracking control as a constrained optimization problem. The optimization problem is formulated on position-level to avoid the computationally expensive Jacobian-inversion. The nonholonomic limitation is ensured by adding equality constraints to the formulated optimization problem. We then present the Beetle Antennae Olfactory Recurrent Neural Network (BAORNN) algorithm to solve the optimization problem efficiently using very few mathematical operations. We present a theoretical analysis of the proposed algorithm and show that its computational cost is linear with respect to the degree of freedoms (DOFs), i.e., O(m). Additionally, we also prove its stability and convergence. Extensive simulation results are prepared using a simulated model of IIWA14, a 7-DOF industrial-manipulator, mounted on a differentially driven cart. Comparison results with particle swarm optimization (PSO) algorithm are also presented to prove the accuracy and numerical efficiency of the proposed controller. The results demonstrate that the proposed algorithm is several times (around 75 in the worst case) faster in execution as compared to PSO, and suitable for real-time implementation. The tracking results for three different trajectories; circular, rectangular, and rhodonea paths are presented