A Practical and Conceptual Framework for Learning in Control

We propose a fully Bayesian approach for efficient reinforcement learning (RL) in Markov decision processes with continuous-valued state and action spaces when no expert knowledge is available. Our framework is based on well-established ideas from statistics and machine learning and learns fast since it carefully models, quantifies, and incorporates available knowledge when making decisions. The key ingredient of our framework is a probabilistic model, which is implemented using a Gaussian process (GP), a distribution over functions. In the context of dynamic systems, the GP models the transition function. By considering all plausible transition functions simultaneously, we reduce model bias, a problem that frequently occurs when deterministic models are used. Due to its generality and efficiency, our RL framework can be considered a conceptual and practical approach to learning models and controllers whe

Intelligent model-based control of complex three-link mechanisms

The aim of this study is to understand the complexity and control challenges of the locomotion of a three-link mechanism of a robot system. In order to do this a three-link robot gymnast (Robogymnast) has been built in Cardiff University. The Robogymnast is composed of three links (one arm, one torso, one leg) and is powered by two geared DC motors. Currently the robot has three potentiometers to measure the relative angles between adjacent links and only one tachometer to measure the relative angular position of the first link. A mathematical model for the robot is derived using Lagrange equations. Since the model is inherently nonlinear and multivariate, it presents more challenges when modelling the Robogymnast and dealing with control motion problems. The proposed approach for dealing with the design of the control system is based on a discrete-time linear model around the upright position of the Robogymnast. To study the swinging motion of the Robogymnast, a new technique is proposed to manipulate the frequency and the amplitude of the sinusoidal signals as a means of controlling the motors. Due to the many combinations of the frequency and amplitude, an optimisation method is required to find the optimal set. The Bees Algorithm (BA), a novel swarm-based optimisation technique, is used to enhance the performance of the swinging motion through optimisation of the manipulated parameters of the control actions. The time taken to reach the upright position at its best is 128 seconds. Two different control methods are adopted to study the balancing/stablising of the Robogymnast in both the downward and upright configurations. The first is the optimal control algorithm using the Linear Quadratic Regulator (LQR) technique with integrators to help achieve and maintain the set of reference trajectories. The second is a combination of Local Control (LC) and LQR. Each controller is implemented via reduced order state observer to estimate the unmeasured states in terms of their relative angular velocities. From the identified data in the relative angular positions of the upright balancing control, it is reported that the maximum amplitude of the deviation in the relative angles on average are approximately 7.5° for the first link and 18° for the second link. It is noted that the third link deviated approximately by 2.5° using only the LQR controller, and no significant deviation when using the LQR with LC. To explore the combination between swinging and balancing motions, a switching mechanism between swinging and balancing algorithm is proposed. This is achieved by dividing the controller into three stages. The first stage is the swinging control, the next stage is the transition control which is accomplished using the Independent Joint Control (IJC) technique and finally balancing control is achieved by the LQR. The duration time of the transition controller to track the reference trajectory of the Robogymnast at its best is found to be within 0.4 seconds. An external disturbance is applied to each link of the Robogymnast separately in order to study the controller's ability to overcome the disturbance and to study the controller response. The simulation of the Robogymnast and experimental realization of the controllers are implemented using MATLAB® software and the C++ program environment respectively

Dynamics and control of a 3D pendulum

Abstract — New pendulum models are introduced and stud-ied. The pendulum consists of a rigid body, supported at a fixed pivot, with three rotational degrees of freedom. The pendulum is acted on by a gravitational force and control forces and moments. Several different pendulum models are developed to analyze properties of the uncontrolled pendulum. Symmetry assumptions are shown to lead to the planar 1D pendulum and to the spherical 2D pendulum models as special cases. The case where the rigid body is asymmetric and the center of mass is distinct from the pivot location leads to the 3D pendulum. Rigid pendulum and multi-body pendulum control problems are proposed. The 3D pendulum models provide a rich source of examples for nonlinear dynamics and control, some of which are similar to simpler pendulum models and some of which are completely new. I

Implementation and Control of an Inverted Pendulum on a Cart

An Inverted Pendulum on a Cart is a common system often used as a benchmark problem for control systems. The system consists of a cart that can move in one direction on the horizontal plane and a pendulum attached to the cart through a hinge point. The pendulum can rotate 360° on the plane made up of the vertical direction and the direction the cart can move. The system is controlled by applying a force to the cart, to make it move. This thesis consists of two goals. The first goal is to build a lab model of the Inverted Pendulum on a Cart system. The second goal is to create a controller that can swing the pendulum from a pendulum down position to a pendulum up position, and balance it in this position. The lab model is built using a track that the cart can move along, a stepper motor for applying force to the cart and a microcontroller for controlling the system. The pendulum angle and the cart position are measured using incremental encoders. A Mathematical model of the system have been derived. This forms the basis for the design of the controller and is also used for simulating and testing the system and controller in MATLAB/Simulink before it is implemented on the real system. The controller consists of three parts. An extended Kalman filter is implemented to estimate the non-measurable state. An energy-based controller is used to swing the pendulum from the down position to the up position. This controller regulates the energy in the pendulum to be close to the energy the pendulum should have when it is balanced in the upright position. When the pendulum is close to the upright position the controller will switch to a linear quadratic regulator to balance the pendulum. This controller is based on a linearized version of the mathematical system model. The lab model and the controllers have been successfully built and implemented

A fast hybrid dual mode NMPC for a parallel double inverted pendulum with experimental validation

This study presents a novel fast non-linear model predictive control approach for a parallel double inverted pendulum. The approach uses dual-mode closed-loop predictions to obtain numerically robust optimal solutions. Moreover, it uses the real-time iteration (RTI) scheme to reduce the computational burden and achieve real-time performance. Furthermore, two main modifications are proposed which significantly improve the performance of the RTI scheme in the presence of large disturbances, namely; additional energy-based costs, and a hybrid switching scheme. Finally, the approach uses a non-standard discretised model combined with an online system identification scheme to address parameter uncertainty, and with an extended Kalman filter for state-estimation. The resulting performance is validated through both simulations and experimental results

Friction compensation in the swing-up control of viscously damped underactuated robotics

A dissertation submitted to the Faculty of Engineering and the Built Environment, University of the Witwatersrand, Johannesburg, in fulfilment of the requirements for the degree of Master of Science in Engineering in the Control Research Group School of Electrical and Information Engineering, Johannesburg, 2017In this research, we observed a torque-related limitation in the swing-up control of underactuated mechanical systems which had been integrated with viscous damping in the unactuated joint. The objective of this research project was thus to develop a practical work-around solution to this limitation. The nth order underactuated robotic system is represented in this research as a collection of compounded pendulums with n-1 actuators placed at each joint with the exception of the first joint. This system is referred to as the PAn-1 robot (Passive first joint, followed by n-1 Active joints), with the Acrobot (PA1 robot) and the PAA robot (or PA2 robot) being among the most well-known examples. A number of friction models exist in literature, which include, and are not exclusive to, the Coulomb and the Stribeck effect models, but the viscous damping model was selected for this research since it is more extensively covered in existing literature. The effectiveness of swing-up control using Lyapunov’s direct method when applied on the undamped PAn-1 robot has been vigorously demonstrated in existing literature, but there is no literature that discusses the swing-up control of viscously damped systems. We show, however, that the application of satisfactory swing-up control using Lyapunov’s direct method is constrained to underactuated systems that are either undamped or actively damped (viscous damping integrated into the actuated joints only). The violation of this constraint results in the derivation of a torque expression that cannot be solved for (invertibility problem, for systems described by n > 2) or a torque expression which contains a conditional singularity (singularity problem, for systems with n = 2). This constraint is formally summarised as the matched damping condition, and highlights a clear limitation in the Lyapunov-related swing-up control of underactuated mechanical systems. This condition has significant implications on the practical realisation of the swing-up control of underactuated mechanical systems, which justifies the investigation into the possibility of a work-around. We thus show that the limitation highlighted by the matched damping condition can be overcome through the implementation of the partial feedback linearisation (PFL) technique. Two key contributions are generated from this research as a result, which iii include the gain selection criterion (for Traditional Collocated PFL), and the convergence algorithm (for noncollocated PFL). The gain selection criterion is an analytical solution that is composed of a set of inequalities that map out a geometric region of appropriate gains in the swing-up gain space. Selecting a gain combination within this region will ensure that the fully-pendent equilibrium point (FPEP) is unstable, which is a necessary condition for swing-up control when the system is initialised near the FPEP. The convergence algorithm is an experimental solution that, once executed, will provide information about the distal pendulum’s angular initial condition that is required to swing-up a robot with a particular angular initial condition for the proximal pendulum, along with the minimum gain that is required to execute the swing-up control in this particular configuration. Significant future contributions on this topic may result from the inclusion of more complex friction models. Additionally, the degree of actuation of the system may be reduced through the implementation of energy storing components, such as torsional springs, at the joint. In summary, we present two contributions in the form of the gain selection criterion and the convergence algorithm which accommodate the circumnavigation of the limitation formalised as the matched damping condition. This condition pertains to the Lyapunov-related swing-up control of underactuated mechanical systems that have been integrated with viscous damping in the unactuated joint.CK201

Hard Real-Time Control of an Inverted Pendulum using RTLinux/Free

The content of this master thesis regards the implementation of a control system in RTLinux/Free, and the use of this for control of a straight track inverted pendulum. A controller developed on the hypothesis that the system is sampled with the sampling time ts will perform differently than expected if sampled with the sampling time t _= ts. If implemented in a regular operating system, the computer will fail to meet demands of hard real-time sampling constraints, i.e. that the desired sampling time must be the one used by the system at all times and in all situations. A computer controlled system in this environment will thus perform di.erently than expected most of the time. Often 'di.erently' is the same as various degrees of worse. The underlying objectives for this project was the desire to be able to control and do research on a hard realtime system connected to an unstable non-linear plant the inverted pendulum. This master thesis covers the work of creating a hard real-time platform using RTLinux/Free on which to add control features. It deals with communication betweenhard real-time and soft real-time, kernel module programming, modelling, linear state feedback, linearization, energy functions, friction compensation, etc

On the robustness of the slotine-Li and the FPT/SVD-based adaptive controllers

A comparative study concerning the robustness of a novel, Fixed Point Transformations/Singular Value Decomposition (FPT/SVD)-based adaptive controller and the Slotine-Li (S&L) approach is given by numerical simulations using a three degree of freedom paradigm of typical Classical Mechanical systems, the cart + double pendulum. The effects of the imprecision of the available dynamical model, presence of dynamic friction at the axles of the drives, and the existence of external disturbance forces unknown and not modeled by the controller are considered. While the Slotine-Li approach tries to identify the parameters of the formally precise, available analytical model of the controlled system with the implicit assumption that the generalized forces are precisely known, the novel one makes do with a very rough, affine form and a formally more precise approximate model of that system, and uses temporal observations of its desired vs. realized responses. Furthermore, it does not assume the lack of unknown perturbations caused either by internal friction and/or external disturbances. Its another advantage is that it needs the execution of the SVD as a relatively time-consuming operation on a grid of a rough system-model only one time, before the commencement of the control cycle within which it works only with simple computations. The simulation examples exemplify the superiority of the FPT/SVD-based control that otherwise has the deficiency that it can get out of the region of its convergence. Therefore its design and use needs preliminary simulation investigations. However, the simulations also exemplify that its convergence can be guaranteed for various practical purposes