A Learning Based Approach to Control Synthesis of Markov Decision Processes for Linear Temporal Logic Specifications

We propose to synthesize a control policy for a Markov decision process (MDP) such that the resulting traces of the MDP satisfy a linear temporal logic (LTL) property. We construct a product MDP that incorporates a deterministic Rabin automaton generated from the desired LTL property. The reward function of the product MDP is defined from the acceptance condition of the Rabin automaton. This construction allows us to apply techniques from learning theory to the problem of synthesis for LTL specifications even when the transition probabilities are not known a priori. We prove that our method is guaranteed to find a controller that satisfies the LTL property with probability one if such a policy exists, and we suggest empirically with a case study in traffic control that our method produces reasonable control strategies even when the LTL property cannot be satisfied with probability one

Robust nonlinear control of vectored thrust aircraft

An interdisciplinary program in robust control for nonlinear systems with applications to a variety of engineering problems is outlined. Major emphasis will be placed on flight control, with both experimental and analytical studies. This program builds on recent new results in control theory for stability, stabilization, robust stability, robust performance, synthesis, and model reduction in a unified framework using Linear Fractional Transformations (LFT's), Linear Matrix Inequalities (LMI's), and the structured singular value micron. Most of these new advances have been accomplished by the Caltech controls group independently or in collaboration with researchers in other institutions. These recent results offer a new and remarkably unified framework for all aspects of robust control, but what is particularly important for this program is that they also have important implications for system identification and control of nonlinear systems. This combines well with Caltech's expertise in nonlinear control theory, both in geometric methods and methods for systems with constraints and saturations

Improving tracking of trajectories through tracking rate regulation: application to UAVs

The tracking problem (that is, how to follow a previously memorized path) is one of the most important problems in mobile robots. Several methods can be formulated depending on the way the robot state is related to the path. “Trajectory tracking” is the most common method, with the controller aiming to move the robot toward a moving target point, like in a real-time servosystem. In the case of complex systems or systems under perturbations or unmodeled effects, such as UAVs (Unmanned Aerial Vehicles), other tracking methods can offer additional benefits. In this paper, methods that consider the dynamics of the path’s descriptor parameter (which can be called “error adaptive tracking”) are contrasted with trajectory tracking. A formal description of tracking methods is first presented, showing that two types of error adaptive tracking can be used with the same controller in any system. Then, it is shown that the selection of an appropriate tracking rate improves error convergence and robustness for a UAV system, which is illustrated by simulation experiments. It is concluded that error adaptive tracking methods outperform trajectory tracking ones, producing a faster and more robust convergence tracking, while preserving, if required, the same tracking rate when convergence is achieved

A brief review of neural networks based learning and control and their applications for robots

As an imitation of the biological nervous systems, neural networks (NN), which are characterized with powerful learning ability, have been employed in a wide range of applications, such as control of complex nonlinear systems, optimization, system identification and patterns recognition etc. This article aims to bring a brief review of the state-of-art NN for the complex nonlinear systems. Recent progresses of NNs in both theoretical developments and practical applications are investigated and surveyed. Specifically, NN based robot learning and control applications were further reviewed, including NN based robot manipulator control, NN based human robot interaction and NN based behavior recognition and generation

Linear matrix inequality-based nonlinear adaptive robust control with application to unmanned aircraft systems

Unmanned aircraft systems (UASs) are gaining popularity in civil and commercial applications as their lightweight on-board computers become more powerful and affordable, their power storage devices improve, and the Federal Aviation Administration addresses the legal and safety concerns of integrating UASs in the national airspace. Consequently, many researchers are pursuing novel methods to control UASs in order to improve their capabilities, dependability, and safety assurance. The nonlinear control approach is a common choice as it offers several benefits for these highly nonlinear aerospace systems (e.g., the quadrotor). First, the controller design is physically intuitive and is derived from well known dynamic equations. Second, the final control law is valid in a larger region of operation, including far from the equilibrium states. And third, the procedure is largely methodical, requiring less expertise with gain tuning, which can be arduous for a novice engineer. ^ Considering these facts, this thesis proposes a nonlinear controller design method that combines the advantages of adaptive robust control (ARC) with the powerful design tools of linear matrix inequalities (LMI). The ARC-LMI controller is designed with a discontinuous projection-based adaptation law, and guarantees a prescribed transient and steady state tracking performance for uncertain systems in the presence of matched disturbances. The norm of the tracking error is bounded by a known function that depends on the controller design parameters in a known form. Furthermore, the LMI-based part of the controller ensures the stability of the system while overcoming polytopic uncertainties, and minimizes the control effort. This can reduce the number of parameters that require adaptation, and helps to avoid control input saturation. ^ These desirable characteristics make the ARC-LMI control algorithm well suited for the quadrotor UAS, which may have unknown parameters and may encounter external disturbances such as wind gusts and turbulence. This thesis develops the ARC-LMI attitude and position controllers for an X-configuration quadrotor helicopter. The inner-loop of the autopilot controls the attitude and altitude of the quadrotor, and the outer-loop controls its position in the earth-fixed coordinate frame. Furthermore, by intelligently generating a smooth trajectory from the given reference coordinates (waypoints), the transient performance is improved. The simulation results indicate that the ARC-LMI controller design is useful for a variety of quadrotor applications, including precise trajectory tracking, autonomous waypoint navigation in the presence of disturbances, and package delivery without loss of performanc

Deterministic learning enhanced neutral network control of unmanned helicopter

In this article, a neural network-based tracking controller is developed for an unmanned helicopter system with guaranteed global stability in the presence of uncertain system dynamics. Due to the coupling and modeling uncertainties of the helicopter systems, neutral networks approximation techniques are employed to compensate the unknown dynamics of each subsystem. In order to extend the semiglobal stability achieved by conventional neural control to global stability, a switching mechanism is also integrated into the control design, such that the resulted neural controller is always valid without any concern on either initial conditions or range of state variables. In addition, deterministic learning is applied to the neutral network learning control, such that the adaptive neutral networks are able to store the learned knowledge that could be reused to construct neutral network controller with improved control performance. Simulation studies are carried out on a helicopter model to illustrate the effectiveness of the proposed control design