64 research outputs found

    An Adaptive Multi-Level Quantization-Based Reinforcement Learning Model for Enhancing UAV Landing on Moving Targets

    Get PDF
    The autonomous landing of an unmanned aerial vehicle (UAV) on a moving platform is an essential functionality in various UAV-based applications. It can be added to a teleoperation UAV system or part of an autonomous UAV control system. Various robust and predictive control systems based on the traditional control theory are used for operating a UAV. Recently, some attempts were made to land a UAV on a moving target using reinforcement learning (RL). Vision is used as a typical way of sensing and detecting the moving target. Mainly, the related works have deployed a deep-neural network (DNN) for RL, which takes the image as input and provides the optimal navigation action as output. However, the delay of the multi-layer topology of the deep neural network affects the real-time aspect of such control. This paper proposes an adaptive multi-level quantization-based reinforcement learning (AMLQ) model. The AMLQ model quantizes the continuous actions and states to directly incorporate simple Q-learning to resolve the delay issue. This solution makes the training faster and enables simple knowledge representation without needing the DNN. For evaluation, the AMLQ model was compared with state-of-art approaches and was found to be superior in terms of root mean square error (RMSE), which was 8.7052 compared with the proportional-integral-derivative (PID) controller, which achieved an RMSE of 10.0592

    Reference Governors for MIMO Systems and Preview Control: Theory, Algorithms, and Practical Applications

    Get PDF
    The Reference Governor (RG) is a methodology based on predictive control for constraint management of pre-stablized closed-loop systems. This problem is motivated by the fact that control systems are usually subject to physical restrictions, hardware protection, and safety and efficiency considerations. The goal of RG is to optimize the tracking performance while ensuring that the constraints are satisfied. Due to structural limitations of RG, however, these requirements are difficult to meet for Multi-Input Multi-Output (MIMO) systems or systems with preview information. Hence, in this dissertation, three extensions of RG for constraint management of these classes of systems are developed. The first approach aims to solve constraint management problem for linear MIMO systems based on decoupling the input-output dynamics, followed by the deployment of a bank of RGs for each decoupled channel, namely Decoupled Reference Governor (DRG). This idea was originally developed in my previous work based on transfer function decoupling, namely DRG-tf. This dissertation improves the design of DRG-tf, analyzes the transient performance of DRG-tf, and extends the DRG formula to state space representations. The second scheme, which is called Preview Reference Governor, extends the applicability of RG to systems incorporated with the preview information of the reference and disturbance signals. The third subject focuses on enforcing constraints on nonlinear MIMO systems. To achieve this goal, three different methods are established. In the first approach, which is referred to as the Nonlinear Decoupled Reference Governor (NL-DRG), instead of employing the Maximal Admissible set and using the decoupling methods as the DRG does, numerical simulations are used to compute the constraint-admissible setpoints. Given the extensive numerical simulations required to implement NL-DRG, the second approach, namely Modified RG (M-RG), is proposed to reduce the computational burden of NL-DRG. This solution consists of the sequential application of different RGs based on linear prediction models, each robustified to account for the worst-case linearization error as well as coupling behavior. Due to this robustification, however, M-RG may lead to a conservative response. To lower the computation time of NL-DRG while improving the performance of M-RG, the third approach, which is referred to as Neural Network DRG (NN-DRG), is proposed. The main idea behinds NN-DRG is to approximate the input-output mapping of NL-DRG with a well-trained NN model. Afterwards, a Quadratic Program is solved to augment the results of NN such that the constraints are satisfied at the next timestep. Additionally, motivated by the broad utilization of quadcopter drones and the necessity to impose constraints on the angles and angle rates of drones, the simulation and experimental results of the proposed nonlinear RG-based methods on a real quadcopter are demonstrated

    Embedded system for motion control of an omnidirectional mobile robot

    Get PDF
    In this paper, an embedded system for motion control of omnidirectional mobile robots is presented. An omnidirectional mobile robot is a type of holonomic robots. It can move simultaneously and independently in translation and rotation. The RoboCup small-size league, a robotic soccer competition, is chosen as the research platform in this paper. The first part of this research is to design and implement an embedded system that can communicate with a remote server using a wireless link, and execute received commands. Second, a fuzzy-Tuned proportional-integral (PI) path planner and a related low-level controller are proposed to attain optimal input for driving a linear discrete dynamic model of the omnidirectional mobile robot. To fit the planning requirements and avoid slippage, velocity, and acceleration filters are also employed. In particular, low-level optimal controllers, such as a linear quadratic regulator (LQR) for multiple-input-multiple-output acceleration and deceleration of velocity are investigated, where an LQR controller is running on the robot with feedback from motor encoders or sensors. Simultaneously, a fuzzy adaptive PI is used as a high-level controller for position monitoring, where an appropriate vision system is used as a source of position feedback. A key contribution presented in this research is an improvement in the combined fuzzy-PI LQR controller over a traditional PI controller. Moreover, the efficiency of the proposed approach and PI controller are also discussed. Simulation and experimental evaluations are conducted with and without external disturbance. An optimal result to decrease the variances between the target trajectory and the actual output is delivered by the onboard regulator controller in this paper. The modeling and experimental results confirm the claim that utilizing the new approach in trajectory-planning controllers results in more precise motion of four-wheeled omnidirectional mobile robots. 2018 IEEE.Scopu

    Control of Flexible Manipulator Robots Based on Dynamic Confined Space of Velocities: Dynamic Programming Approach

    Get PDF
    Linear Parameter Varying models-based Model Predictive Control (LPV-MPC) has stood out in manipulator robots because it presents well-rejection to dynamic uncertainties in flexible joints. However, it has become too weak when the MPC's optimization problem does not include kinematic constraints-based conditions. This paper uses dynamic confined space of velocities (DCSV) to include these conditions as a recursive polytopic constraint, guaranteeing optimal dependency on a simplex scheduling parameter. To this end, the local frame's velocities and torque/force preload of joints (related to violation of kinematic constraints) are associated with different time scale dynamics such that DCSV correlates them as a polytope. So, a classical LPV-MPC will be updated using a dynamic programming approach according to the DCSV-based polytope. As a result, one lemma about DCSV-based recursive polytope and a five-step procedure for two decoupled close-loop schemes with different time scales compose the LPV-MPC proposed method. Numerical validation shows that even for relevant flexibility situations, trajectory tracking performance is improved by tuning finite horizons and optimization problem constraints regarding DCSV's behavior

    Channel Assignment in Uplink Wireless Communication using Machine Learning Approach

    Get PDF
    This letter investigates a channel assignment problem in uplink wireless communication systems. Our goal is to maximize the sum rate of all users subject to integer channel assignment constraints. A convex optimization based algorithm is provided to obtain the optimal channel assignment, where the closed-form solution is obtained in each step. Due to high computational complexity in the convex optimization based algorithm, machine learning approaches are employed to obtain computational efficient solutions. More specifically, the data are generated by using convex optimization based algorithm and the original problem is converted to a regression problem which is addressed by the integration of convolutional neural networks (CNNs), feed-forward neural networks (FNNs), random forest and gated recurrent unit networks (GRUs). The results demonstrate that the machine learning method largely reduces the computation time with slightly compromising of prediction accuracy

    Autonomous system control in unknown operating conditions

    Get PDF
    Autonomous systems have become an interconnected part of everyday life with the recent increases in computational power available for both onboard computers and offline data processing. The race by car manufacturers for level 5 (full) autonomy in self-driving cars is well underway and new flying taxi service startups are emerging every week, attracting billions in investments. Two main research communities, Optimal Control and Reinforcement Learning stand out in the field of autonomous systems, each with a vastly different perspective on the control problem. Controllers from the optimal control community are based on models and can be rigorously analyzed to ensure the stability of the system is maintained under certain operating conditions. Learning-based control strategies are often referred to as model-free and typically involve training a neural network to generate the required control actions through direct interactions with the system. This greatly reduces the design effort required to control complex systems. One common problem both learning- and model- based control solutions face is the dependency on a priori knowledge about the system and operating conditions such as possible internal component failures and external environmental disturbances. It is not possible to consider every possible operating scenario an autonomous system can encounter in the real world at design time. Models and simulators are approximations of reality and can only be created for known operating conditions. Autonomous system control in unknown operating conditions, where no a priori knowledge exists, is still an open problem for both communities and no control methods currently exist for such situations. Multiple model adaptive control is a modular control framework that divides the control problem into supervisory and low-level control, which allows for the combination of existing learning- and model-based control methods to overcome the disadvantages of using only one of these. The contributions of this thesis consist of five novel supervisory control architectures, which have been empirically shown to improve a system’s robustness to unknown operating conditions, and a novel low- level controller tuning algorithm that can reduce the number of required controllers compared to traditional tuning approaches. The presented methods apply to any autonomous system that can be controlled using model-based controllers and can be integrated alongside existing fault-tolerant control systems to improve robustness to unknown operating conditions. This impacts autonomous system designers by providing novel control mechanisms to improve a system’s robustness to unknown operating conditions

    A comprehensive study of robot control algorithms

    Get PDF
    The PUMA 560 Industrial Manipulator is presently controlled using a PID control strategy Robot manipulators are highly coupled, nonlinear mechanical systems designed to perform specific tasks. It is the function of any control algorithm to compute the input voltages or torques needed to follow a desired trajectory. The PID controller is detuned, so as to cater for variations in system behaviour. Thus, the performance of such a control algorithm is poor over the entire operating range of the robot and the need for more complex control strategies is clear. The research presented m this thesis derives a third order comprehensive dynamic model for the three primary robot joints, using the Euler-Lagrange formulation for the equations of motion. A simulation package is designed to model this dynamic system. Next, a wide range of different techniques are investigated in a simulation environment, to observe their performance on the computer model. These control algorithms range from Fixed Parameter techniques to Adaptive strategies and Feedforward routines. A set of performance criteria can be used to evaluate these techniques, and the best algonthm from each section is chosen. Using the results of an identification performed on the robot, each of these control methods is applied to the resulting tune varying model. The results here are used to determine the optimal control strategy for manipulator use. Also in this thesis, a new hardware structure is designed and implemented. This structure is capable of implementing complex control routines with adequately low sample periods. The design uses advanced digital signal processors, which can perform arithmetic operations quickly
    corecore