Adaptive optimal control of under-actuated robotic systems using a self-regulating nonlinear weight-adjustment scheme: Formulation and experimental verification

This paper formulates an innovative model-free self-organizing weight adaptation that strengthens the robustness of a Linear Quadratic Regulator (LQR) for inverted pendulum-like mechatronic systems against perturbations and parametric uncertainties. The proposed control procedure is devised by using an online adaptation law to dynamically adjust the state weighting factors of LQR's quadratic performance index via pre-calibrated state-error-dependent hyperbolic secant functions (HSFs). The updated state-weighting factors re-compute the optimal control problem to modify the state-compensator gains online. The novelty of the proposed article lies in adaptively adjusting the variation rates of the said HSFs via an auxiliary model-free online self-regulation law that uses dissipative and anti-dissipative terms to flexibly re-calibrate the nonlinear function's waveforms as the state errors vary. This augmentation increases the controller's design flexibility and enhances the system's disturbance rejection capacity while economizing control energy expenditure under every operating condition. The proposed self-organizing LQR is analyzed via customized hardware-in-loop (HIL) experiments conducted on the Quanser's single-link rotational inverted pendulum. As compared to the fixed-gain LQR, the proposed SR-EM-STC delivers an improvement of 52.2%, 16.4%, 55.2%, and 42.7% in the pendulum's position regulation behavior, control energy expenditure, transient recovery duration, and peak overshoot, respectively. The experimental outcomes validate the superior robustness of the proposed scheme against exogenous disturbances

Advanced Strategies for Robot Manipulators

Amongst the robotic systems, robot manipulators have proven themselves to be of increasing importance and are widely adopted to substitute for human in repetitive and/or hazardous tasks. Modern manipulators are designed complicatedly and need to do more precise, crucial and critical tasks. So, the simple traditional control methods cannot be efficient, and advanced control strategies with considering special constraints are needed to establish. In spite of the fact that groundbreaking researches have been carried out in this realm until now, there are still many novel aspects which have to be explored

Markov chain monte carlo algorithm for bayesian policy search

The fundamental intention in Reinforcement Learning (RL) is to seek for optimal parameters of a given parameterized policy. Policy search algorithms have paved the way for making the RL suitable for applying to complex dynamical systems, such as robotics domain, where the environment comprised of high-dimensional state and action spaces. Although many policy search techniques are based on the wide spread policy gradient methods, thanks to their appropriateness to such complex environments, their performance might be a ected by slow convergence or local optima complications. The reason for this is due to the urge for computation of the gradient components of the parameterized policy. In this study, we avail a Bayesian approach for policy search problem pertinent to the RL framework, The problem of interest is to control a discrete time Markov decision process (MDP) with continuous state and action spaces. We contribute to the eld by propounding a Particle Markov Chain Monte Carlo (P-MCMC) algorithm as a method of generating samples for the policy parameters from a posterior distribution, instead of performing gradient approximations. To do so, we adopt a prior density over policy parameters and aim for the posterior distribution where the `likelihood' is assumed to be the expected total reward. In terms of risk-sensitive scenarios, where a multiplicative expected total reward is employed to measure the performance of the policy, rather than its cumulative counterpart, our methodology is t for purpose owing to the fact that by utilizing a reward function in a multiplicative form, one can fully take sequential Monte Carlo (SMC), known as the particle lter within the iterations of the P-MCMC. it is worth mentioning that these methods have widely been used in statistical and engineering applications in recent years. Furthermore, in order to deal with the challenging problem of the policy search in large-dimensional state spaces an Adaptive MCMC algorithm will be proposed. This research is organized as follows: In Chapter 1, we commence with a general introduction and motivation to the current work and highlight the topics that are going to be covered. In Chapter 2ö a literature review pursuant to the context of the thesis will be conducted. In Chapter 3, a brief review of some popular policy gradient based RL methods is provided. We proceed with Bayesian inference notion and present Markov Chain Monte Carlo methods in Chapter 4. The original work of the thesis is formulated in this chapter where a novel SMC algorithm for policy search in RL setting is advocated. In order to exhibit the fruitfulness of the proposed algorithm in learning a parameterized policy, numerical simulations are incorporated in Chapter 5. To validate the applicability of the proposed method in real-time it will be implemented on a control problem of a physical setup of a two degree of freedom (2-DoF) robotic manipulator where its corresponding results appear in Chapter 6. Finally, concluding remarks and future work are expressed in chapter

A brief review of neural networks based learning and control and their applications for robots

As an imitation of the biological nervous systems, neural networks (NN), which are characterized with powerful learning ability, have been employed in a wide range of applications, such as control of complex nonlinear systems, optimization, system identification and patterns recognition etc. This article aims to bring a brief review of the state-of-art NN for the complex nonlinear systems. Recent progresses of NNs in both theoretical developments and practical applications are investigated and surveyed. Specifically, NN based robot learning and control applications were further reviewed, including NN based robot manipulator control, NN based human robot interaction and NN based behavior recognition and generation

Active fault-tolerant control of nonlinear systems with wind turbine application

The thesis concerns the theoretical development of Active Fault-Tolerant Control (AFTC) methods for nonlinear system via T-S multiple-modelling approach. The thesis adopted the estimation and compensation approach to AFTC within a tracking control framework. In this framework, the thesis considers several approaches to robust T-S fuzzy control and T-S fuzzy estimation: T-S fuzzy proportional multiple integral observer (PMIO); T-S fuzzy proportional-proportional integral observer (PPIO); T-S fuzzy virtual sensor (VS) based AFTC; T-S fuzzy Dynamic Output Feedback Control TSDOFC; T-S observer-based feedback control; Sliding Mode Control (SMC). The theoretical concepts have been applied to an offshore wind turbine (OWT) application study. The key developments that present in this thesis are:• The development of three active Fault Tolerant Tracking Control (FTTC) strategies for nonlinear systems described via T-S fuzzy inference modelling. The proposals combine the use of Linear Reference Model Fuzzy Control (LRMFC) with either the estimation and compensation concept or the control reconfiguration concept.• The development of T-S fuzzy observer-based state estimate fuzzy control strategy for nonlinear systems. The developed strategy has the capability to tolerate simultaneous actuator and sensor faults within tracking and regulating control framework. Additionally, a proposal to recover the Separation Principle has also been developed via the use of TSDOFC within the FTTC framework.• The proposals of two FTTC strategies based on the estimation and compensation concept for sustainable OWTs control. The proposals have introduced a significant attribute to the literature of sustainable OWTs control via (1) Obviating the need for Fault Detection and Diagnosis (FDD) unit, (2) Providing useful information to evaluate fault severity via the fault estimation signals.• The development of FTTC architecture for OWTs that combines the use of TSDOFC and a form of cascaded observers (cascaded analytical redundancy). This architecture is proposed in order to ensure the robustness of both the TSDOFC and the EWS estimator against the generator and rotor speed sensor faults.• A sliding mode baseline controller has been proposed within three FTTC strategies for sustainable OWTs control. The proposals utilise the inherent robustness of the SMC to tolerate some matched faults without the need for analytical redundancy. Following this, the combination of SMC and estimation and compensation framework proposed to ensure the close-loop system robustness to various faults.• Within the framework of the developed T-S fuzzy based FTTC strategies, a new perspective to reduce the T-S fuzzy control design conservatism problem has been proposed via the use of different control techniques that demand less design constraints. Moreover, within the SMC based FTTC, an investigation is given to demonstrate the SMC robustness against a wider than usual set of faults is enhanced via designing the sliding surface with minimum dimension of the feedback signals