976 research outputs found
Reinforcement learning-based structural control of floating wind turbines
The structural control of floating wind turbines using active tuned mass damper is investigated in this article. To our knowledge, this is for the first time that reinforcement learning-based control approach is employed to this type of application. Specifically, an adaptive dynamic programming (ADP) algorithm is used to derive the optimal control law based on the nonlinear structural dynamics, and the large-scale machine learning platform Tensorflow is employed for the design and implementation of the neural network (NN) structure. Three fully connected NNs, i.e., a plant network, a critic network, and an action network, are included in the proposed NN structure. Their training requires the gradient information flowing through the whole network, which is tackled by automatic differentiation, a popular technique for deriving the gradients of complex networks automatically. While to our knowledge, the network structures in the existing literature are rather simple and the training of the hidden layer is usually ignored. This allows their gradients to be derived analytically, which is infeasible with complex network structures. Thus, automatic differentiation greatly improves the employed ADP algorithm's ability in solving complex problems. The simulation results of structural control of floating wind turbines show that ADP controller performs very well in both normal and extreme conditions, with the standard deviation of the platform pitch displacement being reduced by around 40%. A clear advantage of ADP controllers over the H∞ controller is observed, especially in extreme conditions. Moreover, our design considers the tradeoff between the control performance and power consumption
Hamiltonian-Driven Adaptive Dynamic Programming with Efficient Experience Replay
This article presents a novel efficient experience-replay-based adaptive dynamic programming (ADP) for the optimal control problem of a class of nonlinear dynamical systems within the Hamiltonian-driven framework. The quasi-Hamiltonian is presented for the policy evaluation problem with an admissible policy. With the quasi-Hamiltonian, a novel composite critic learning mechanism is developed to combine the instantaneous data with the historical data. In addition, the pseudo-Hamiltonian is defined to deal with the performance optimization problem. Based on the pseudo-Hamiltonian, the conventional Hamilton–Jacobi–Bellman (HJB) equation can be represented in a filtered form, which can be implemented online. Theoretical analysis is investigated in terms of the convergence of the adaptive critic design and the stability of the closed-loop systems, where parameter convergence can be achieved under a weakened excitation condition. Simulation studies are investigated to verify the efficacy of the presented design scheme
Adaptive Model Predictive Control for Engine-Driven Ducted Fan Lift Systems using an Associated Linear Parameter Varying Model
Ducted fan lift systems (DFLSs) powered by two-stroke aviation piston engines
present a challenging control problem due to their complex multivariable
dynamics. Current controllers for these systems typically rely on
proportional-integral algorithms combined with data tables, which rely on
accurate models and are not adaptive to handle time-varying dynamics or system
uncertainties. This paper proposes a novel adaptive model predictive control
(AMPC) strategy with an associated linear parameter varying (LPV) model for
controlling the engine-driven DFLS. This LPV model is derived from a global
network model, which is trained off-line with data obtained from a general mean
value engine model for two-stroke aviation engines. Different network models,
including multi-layer perceptron, Elman, and radial basis function (RBF), are
evaluated and compared in this study. The results demonstrate that the RBF
model exhibits higher prediction accuracy and robustness in the DFLS
application. Based on the trained RBF model, the proposed AMPC approach
constructs an associated network that directly outputs the LPV model parameters
as an adaptive, robust, and efficient prediction model. The efficiency of the
proposed approach is demonstrated through numerical simulations of a vertical
take-off thrust preparation process for the DFLS. The simulation results
indicate that the proposed AMPC method can effectively control the DFLS thrust
with a relative error below 3.5%
Beyond Basins of Attraction: Quantifying Robustness of Natural Dynamics
Properly designing a system to exhibit favorable natural dynamics can greatly
simplify designing or learning the control policy. However, it is still unclear
what constitutes favorable natural dynamics and how to quantify its effect.
Most studies of simple walking and running models have focused on the basins of
attraction of passive limit-cycles and the notion of self-stability. We instead
emphasize the importance of stepping beyond basins of attraction. We show an
approach based on viability theory to quantify robust sets in state-action
space. These sets are valid for the family of all robust control policies,
which allows us to quantify the robustness inherent to the natural dynamics
before designing the control policy or specifying a control objective. We
illustrate our formulation using spring-mass models, simple low dimensional
models of running systems. We then show an example application by optimizing
robustness of a simulated planar monoped, using a gradient-free optimization
scheme. Both case studies result in a nonlinear effective stiffness providing
more robustness.Comment: 15 pages. This work has been accepted to IEEE Transactions on
Robotics (2019
SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence
Spiking neural networks (SNNs) aim to realize brain-inspired intelligence on
neuromorphic chips with high energy efficiency by introducing neural dynamics
and spike properties. As the emerging spiking deep learning paradigm attracts
increasing interest, traditional programming frameworks cannot meet the demands
of the automatic differentiation, parallel computation acceleration, and high
integration of processing neuromorphic datasets and deployment. In this work,
we present the SpikingJelly framework to address the aforementioned dilemma. We
contribute a full-stack toolkit for pre-processing neuromorphic datasets,
building deep SNNs, optimizing their parameters, and deploying SNNs on
neuromorphic chips. Compared to existing methods, the training of deep SNNs can
be accelerated , and the superior extensibility and flexibility of
SpikingJelly enable users to accelerate custom models at low costs through
multilevel inheritance and semiautomatic code generation. SpikingJelly paves
the way for synthesizing truly energy-efficient SNN-based machine intelligence
systems, which will enrich the ecology of neuromorphic computing.Comment: Accepted in Science Advances
(https://www.science.org/doi/10.1126/sciadv.adi1480
- …