Quotient method for controlling the acrobot

This paper describes a two-sweep control design method to stabilize the acrobot, an input-affine under-actuated system, at the upper equilibrium point. In the forward sweep, the system is successively reduced, one dimension at a time, until a two-dimensional system is obtained. At each step of the reduction process, a quotient is taken along one-dimensional integral manifolds of the input vector field. This decomposes the current manifold into classes of equivalence that constitute a quotient manifold of reduced dimension. The input to a given step becomes the representative of the previous-step equivalence class, and a new input vector field can be defined on the tangent of the quotient manifold. The representatives remain undefined throughout the forward sweep. During the backward sweep, the controller is designed recursively, starting with the two- dimensional system. At each step of the recursion, a well-chosen representative of the equivalence class ahead of the current level of recursion is chosen, so as to guarantee stability of the current step. Therefore, this stabilizes the global system once the backward sweep is complete. Although stability can only be guaranteed locally around the upper equilibrium point, the domain of attraction can be enlarged to include the lower equilibrium point, thereby allowing a swing-up implementation. As a result, the controller does not require switching, which is illustrated in simulation. The controller has four tuning parameters, which helps shape the closed-loop behavior

Quotient method for stabilising a ball-on-a-wheel system – Experimental results

This paper extends the quotient method proposed in [1] and applies it to stabilize a “ball-on-a-wheel” system. The quotient method requires a diffeomorphism to obtain the normal form of the input vector field and uses canonical pro- jection to obtain the quotient. However, the whole process can be done without computing the normal form, which requires defining a quotient generating function and a quotient bracket. This paper presents the steps necessary to apply the quotient method without obtaining the normal form. Furthermore, a Lyapunov function is introduced to prove stability. This paper also presents the experimental implementation of the quotient method to stabilize a ball-on-a-wheel system

Reinforcement learning in continuous state- and action-space

Reinforcement learning in the continuous state-space poses the problem of the inability to store the values of all state-action pairs in a lookup table, due to both storage limitations and the inability to visit all states sufficiently often to learn the correct values. This can be overcome with the use of function approximation techniques with generalisation capability, such as artificial neural networks, to store the value function. When this is applied we can select the optimal action by comparing the values of each possible action; however, when the action-space is continuous this is not possible. In this thesis we investigate methods to select the optimal action when artificial neural networks are used to approximate the value function, through the application of numerical optimization techniques. Although it has been stated in the literature that gradient-ascent methods can be applied to the action selection [47], it is also stated that solving this problem would be infeasible, and therefore, is claimed that it is necessary to utilise a second artificial neural network to approximate the policy function [21, 55]. The major contributions of this thesis include the investigation of the applicability of action selection by numerical optimization methods, including gradient-ascent along with other derivative-based and derivative-free numerical optimization methods,and the proposal of two novel algorithms which are based on the application of two alternative action selection methods: NM-SARSA [40] and NelderMead-SARSA. We empirically compare the proposed methods to state-of-the-art methods from the literature on three continuous state- and action-space control benchmark problems from the literature: minimum-time full swing-up of the Acrobot; Cart-Pole balancing problem; and a double pole variant. We also present novel results from the application of the existing direct policy search method genetic programming to the Acrobot benchmark problem [12, 14]

A quotient method for designing nonlinear controllers

An algorithmic method is proposed to design stabilizing control laws for a class of nonlinear systems that comprises single-input feedback-linearizable systems and a particular set of single-input non feedback-linearizable systems. The method proceeds iteratively and consists of two stages; it converts the system into cascade form and reduces the dimension at every step by creating quotient manifold in the forward stage, while it constructs the feedback law iteratively in the backward stage. The paper shows that the construction of these quotient manifolds is well defined for feedback-linearizable system and, furthermore, it can also be applied to a class of non feedback-linearizable systems

Numerical algorithm for feedback linearizable systems

A numerical algorithm that achieves asymptotic stability for feedback linearizable systems is presented. The nonlinear systems can be represented in various forms that include differential equations, simulated physical models or lookup tables. The proposed algorithm is based on a quotient method and proceeds iteratively. At each step, the dynamic system is desensitized with respect to the current input vector field. Control is obtained by tracking a desired value along the input vector field at each step. The numerical algorithm uses the direction on the tangent manifold at a given point and its variation around that point. This enables the algorithm to produce control values simply using a simulator of the nonlinear system

Control Strategy Based on Fourier Transformation and Intelligent Optimization for Planar Pendubot

This paper presents a new control strategy based on Fourier transformation and intelligent optimization for a planar Pendubot with a passive second link, which can be treated as a second-order nonholonomic system whose control has been an open and challenging issue. A controller acting within a time corresponding to the frequency of its fundamental harmonic term is designed to realize the system control objective, which is to move the system from its initial position to the target position. By employing Fourier transformation, a general expression of the controller composed of a constant term and harmonic terms is obtained. Next, the constant term is obtained by the angular momentum theorem, and the particle swarm optimization algorithm is employed to obtain the harmonic terms of the controller. A feedback control strategy based on a nonlinear disturbance observer is then applied to overcome the uncertainties/disturbances in the system. Finally, simulation results prove the validity of this control method

Avoiding Feedback-Linearization Singularity Using a Quotient Method -- The Field-Controlled DC Motor Case

Feedback linearization requires a unique feedback law and a unique diffeomorphism to bring a system to Brunovsk´y normal form. Unfortunately, singularities might arise both in the feedback law and in the diffeomorphism. This paper demonstrates the ability of a quotient method to avoid or mitigate the singularities that typically arise with feedback linearization. The quotient method does it by relaxing the conditions on diffeomorphism, which can be achieved since there is an additional degree of freedom at each step of the iterative procedure. This freedom in choosing quotients and the resulting advantage are demonstrated for a field-controlled DC motor. Using a Lyapunov function, the domain of attraction of the control law obtained with the quotient method is proved to be larger than the domain of attraction of a control law obtained using feedback linearization

Intelligent model-based control of complex three-link mechanisms

The aim of this study is to understand the complexity and control challenges of the locomotion of a three-link mechanism of a robot system. In order to do this a three-link robot gymnast (Robogymnast) has been built in Cardiff University. The Robogymnast is composed of three links (one arm, one torso, one leg) and is powered by two geared DC motors. Currently the robot has three potentiometers to measure the relative angles between adjacent links and only one tachometer to measure the relative angular position of the first link. A mathematical model for the robot is derived using Lagrange equations. Since the model is inherently nonlinear and multivariate, it presents more challenges when modelling the Robogymnast and dealing with control motion problems. The proposed approach for dealing with the design of the control system is based on a discrete-time linear model around the upright position of the Robogymnast. To study the swinging motion of the Robogymnast, a new technique is proposed to manipulate the frequency and the amplitude of the sinusoidal signals as a means of controlling the motors. Due to the many combinations of the frequency and amplitude, an optimisation method is required to find the optimal set. The Bees Algorithm (BA), a novel swarm-based optimisation technique, is used to enhance the performance of the swinging motion through optimisation of the manipulated parameters of the control actions. The time taken to reach the upright position at its best is 128 seconds. Two different control methods are adopted to study the balancing/stablising of the Robogymnast in both the downward and upright configurations. The first is the optimal control algorithm using the Linear Quadratic Regulator (LQR) technique with integrators to help achieve and maintain the set of reference trajectories. The second is a combination of Local Control (LC) and LQR. Each controller is implemented via reduced order state observer to estimate the unmeasured states in terms of their relative angular velocities. From the identified data in the relative angular positions of the upright balancing control, it is reported that the maximum amplitude of the deviation in the relative angles on average are approximately 7.5° for the first link and 18° for the second link. It is noted that the third link deviated approximately by 2.5° using only the LQR controller, and no significant deviation when using the LQR with LC. To explore the combination between swinging and balancing motions, a switching mechanism between swinging and balancing algorithm is proposed. This is achieved by dividing the controller into three stages. The first stage is the swinging control, the next stage is the transition control which is accomplished using the Independent Joint Control (IJC) technique and finally balancing control is achieved by the LQR. The duration time of the transition controller to track the reference trajectory of the Robogymnast at its best is found to be within 0.4 seconds. An external disturbance is applied to each link of the Robogymnast separately in order to study the controller's ability to overcome the disturbance and to study the controller response. The simulation of the Robogymnast and experimental realization of the controllers are implemented using MATLAB® software and the C++ program environment respectively

Regelungstheorie

The workshop “Regelungstheorie” (control theory) covered a broad variety of topics that were either concerned with fundamental mathematical aspects of control or with its strong impact in various fields of engineering