Issues on Stability of ADP Feedback Controllers for Dynamical Systems

This paper traces the development of neural-network (NN)-based feedback controllers that are derived from the principle of adaptive/approximate dynamic programming (ADP) and discusses their closed-loop stability. Different versions of NN structures in the literature, which embed mathematical mappings related to solutions of the ADP-formulated problems called “adaptive critics” or “action-critic” networks, are discussed. Distinction between the two classes of ADP applications is pointed out. Furthermore, papers in “model-free” development and model-based neurocontrollers are reviewed in terms of their contributions to stability issues. Recent literature suggests that work in ADP-based feedback controllers with assured stability is growing in diverse forms

A review of variable-pitch propellers and their control strategies in aerospace systems

The relentless pursuit of aircraft flight efficiency has thrust variable-pitch propeller technology into the forefront of aviation innovation. This technology, rooted in the ancient power unit of propellers, has found renewed significance, particularly in the realms of unmanned aerial vehicles and urban air mobility. This underscores the profound interplay between visionary aviation concepts and the enduring utility of propellers. Variable-pitch propellers are poised to be pivotal in shaping the future of human aviation, offering benefits such as extended endurance, enhanced maneuverability, improved fuel economy, and prolonged engine life. However, with additional capabilities come new technical challenges. The development of an online adaptive control of variable-pitch propellers that does not depend on an accurate dynamic model stands as a critical imperative. Therefore, a comprehensive review and forward-looking analysis of this technology is warranted. This paper introduces the development background of variable-pitch aviation propeller technology, encompassing diverse pitch angle adjustment schemes and their integration with various engine types. It places a central focus on the latest research frontiers and emerging directions in pitch control strategies. Lastly, it delves into the research domain of constant speed pitch control, articulating the three main challenges confronting this technology: inadequacies in system modeling, the intricacies of propeller-engine compatibility, and the impact of external, time-varying factors. By shedding light on these multifaceted aspects of variable-pitch propeller technology, this paper serves as a resource for aviation professionals and researchers navigating the intricate landscape of future aircraft development

Machine Learning in Aerodynamic Shape Optimization

Machine learning (ML) has been increasingly used to aid aerodynamic shape optimization (ASO), thanks to the availability of aerodynamic data and continued developments in deep learning. We review the applications of ML in ASO to date and provide a perspective on the state-of-the-art and future directions. We first introduce conventional ASO and current challenges. Next, we introduce ML fundamentals and detail ML algorithms that have been successful in ASO. Then, we review ML applications to ASO addressing three aspects: compact geometric design space, fast aerodynamic analysis, and efficient optimization architecture. In addition to providing a comprehensive summary of the research, we comment on the practicality and effectiveness of the developed methods. We show how cutting-edge ML approaches can benefit ASO and address challenging demands, such as interactive design optimization. Practical large-scale design optimizations remain a challenge because of the high cost of ML training. Further research on coupling ML model construction with prior experience and knowledge, such as physics-informed ML, is recommended to solve large-scale ASO problems

Quantum inspired algorithms for learning and control of stochastic systems

Motivated by the limitations of the current reinforcement learning and optimal control techniques, this dissertation proposes quantum theory inspired algorithms for learning and control of both single-agent and multi-agent stochastic systems. A common problem encountered in traditional reinforcement learning techniques is the exploration-exploitation trade-off. To address the above issue an action selection procedure inspired by a quantum search algorithm called Grover\u27s iteration is developed. This procedure does not require an explicit design parameter to specify the relative frequency of explorative/exploitative actions. The second part of this dissertation extends the powerful adaptive critic design methodology to solve finite horizon stochastic optimal control problems. To numerically solve the stochastic Hamilton Jacobi Bellman equation, which characterizes the optimal expected cost function, large number of trajectory samples are required. The proposed methodology overcomes the above difficulty by using the path integral control formulation to adaptively sample trajectories of importance. The third part of this dissertation presents two quantum inspired coordination models to dynamically assign targets to agents operating in a stochastic environment. The first approach uses a quantum decision theory model that explains irrational action choices in human decision making. The second approach uses a quantum game theory model that exploits the quantum mechanical phenomena \u27entanglement\u27 to increase individual pay-off in multi-player games. The efficiency and scalability of the proposed coordination models are demonstrated through simulations of a large scale multi-agent system --Abstract, page iii

Benchmarking Cerebellar Control

Cerebellar models have long been advocated as viable models for robot dynamics control. Building on an increasing insight in and knowledge of the biological cerebellum, many models have been greatly refined, of which some computational models have emerged with useful properties with respect to robot dynamics control. Looking at the application side, however, there is a totally different picture. Not only is there not one robot on the market which uses anything remotely connected with cerebellar control, but even in research labs most testbeds for cerebellar models are restricted to toy problems. Such applications hardly ever exceed the complexity of a 2 DoF simulated robot arm; a task which is hardly representative for the field of robotics, or relates to realistic applications. In order to bring the amalgamation of the two fields forwards, we advocate the use of a set of robotics benchmarks, on which existing and new computational cerebellar models can be comparatively tested. It is clear that the traditional approach to solve robotics dynamics loses ground with the advancing complexity of robotic structures; there is a desire for adaptive methods which can compete as traditional control methods do for traditional robots. In this paper we try to lay down the successes and problems in the fields of cerebellar modelling as well as robot dynamics control. By analyzing the common ground, a set of benchmarks is suggested which may serve as typical robot applications for cerebellar models