Automatic control of a multirotor

Objective of this thesis is to describe the design and realisation phases of a multirotor to be used for low risk and cost aerial observation. Starting point of this activity was a wide literature study related to the technological evolution of multirotors design and to the state of the art. Firstly the most common multirotor configurations were defined and, according to a size and performance based evaluation, the most suitable one was chosen. A detailed computer aided design model was drawn as basis for the realisation of two prototypes. The realised multirotors were “X-shaped” octorotors with eight coaxially coupled motors. The mathematical model of the multirotor dynamics was studied. “Proportional Integral Derivative” and “Linear Quadratic” algorithms were chosen as techniques to regulate the attitude dynamics of the multirotor. These methods were tested with a nonlinear model simulation developed in the Matlab Simulink environment. In the meanwhile the Arduino board was selected as the best compromise between costs and performance and the above mentioned algorithms were implemented using this platform thanks to its main characteristic of being completely “open source”. Indeed the multirotor was conceived to be a serviceable tool for the public utility and, at the same time, to be an accessible device for research and studies. The behaviour of the physical multirotor was evaluated with a test bench designed to isolate the rotation about one single body axis at a time. The data of the experimental tests were gathered in real time using a custom Matlab code and several indoor tests allowed the “fine tuning” of the controllers gains. Afterwards a portable “ground station” was conceived and realised in adherence with the real scenarios users needs. Several outdoor experimental flights were executed with successful results and the data gathered during the outdoor tests were used to evaluate some key performance indicators as the endurance and the maximum allowable payload mass. Then the fault tolerance of the control system was evaluated simulating and experimenting the loss of one motor; even in this critical condition the system exhibited an acceptable behaviour. The reached project readiness allowed to meet some potential users as the “Turin Fire Department” and to cooperate with them in a simulated emergency. During this event the multirotor was used to gather and transmit real time aerial images for an improved “situation awareness”. Finally the study was extended to more innovative control techniques like the neural networks based ones. Simulations results demonstrated their effectiveness; nevertheless the inherent complexity and the unreliability outside the training ranges could have a catastrophic impact on the airworthiness. This is a factor that cannot be neglected especially in the applications related to flying platforms. Summarising, this research work was addressed mainly to the operating procedures for implementing automatic control algorithms to real platforms. All the design aspects, from the preliminary multirotor configuration choice to the tests in possible real scenarios, were covered obtaining performances comparable with other commercial of-the-shelf platforms

Global sensitivity analysis based on DIRECT-KG-HDMR and thermal optimization of pin-fin heat sink for the platform inertial navigation system

In this study, in order to reduce the local high temperature of the platform in inertial navigation system (PINS), a pin-fin heat sink with staggered arrangement is designed. To reduce the dimension of the inputs and improve the efficiency of optimization, a feasible global sensitivity analysis (GSA) based on Kriging-High Dimensional Model Representation with DIviding RECTangles sampling strategy (DIRECT-KG-HDMR) is proposed. Compared with other GSA methods, the proposed method can indicate the effects of the structural and the material parameters on the maximum temperature at the bottom of the heat sink by using both sensitivity and coupling coefficients. From the results of GSA, it can be found that the structural parameters have greater effects on thermal performance than the material ones. Moreover, the coupling intensities between the structural and material parameters are weak. Therefore, the structural parameters are selected to optimize the thermal performance of the heat sink, and several popular optimization algorithms such as GA, DE, TLBO, PSO and EGO are used for the optimization. Moreover, steady thermal response of the PINS with the optimized heat sink is also studied, and its result shows that the maximum temperature of high temperature region of the platform is reduced by 1.09 degree Celsius compared with the PINS without the heat sink.Comment: 34 pages, 18 figures, 5 table

Flight controller synthesis via deep reinforcement learning

Traditional control methods are inadequate in many deployment settings involving autonomous control of Cyber-Physical Systems (CPS). In such settings, CPS controllers must operate and respond to unpredictable interactions, conditions, or failure modes. Dealing with such unpredictability requires the use of executive and cognitive control functions that allow for planning and reasoning. Motivated by the sport of drone racing, this dissertation addresses these concerns for state-of-the-art flight control by investigating the use of deep artificial neural networks to bring essential elements of higher-level cognition to bear on the design, implementation, deployment, and evaluation of low level (attitude) flight controllers. First, this thesis presents a feasibility analyses and results which confirm that neural networks, trained via reinforcement learning, are more accurate than traditional control methods used by commercial uncrewed aerial vehicles (UAVs) for attitude control. Second, armed with these results, this thesis reports on the development and release of an open source, full solution stack for building neuro-flight controllers. This stack consists of a tuning framework for implementing training environments (GymFC) and firmware for the world’s first neural network supported flight controller (Neuroflight). GymFC’s novel approach fuses together the digital twinning paradigm with flight control training to provide seamless transfer to hardware. Third, to transfer models synthesized by GymFC to hardware, this thesis reports on the toolchain that has been released for compiling neural networks into Neuroflight, which can be flashed to off-the-shelf microcontrollers. This toolchain includes detailed procedures for constructing a multicopter digital twin to allow the research and development community to synthesize flight controllers unique to their own aircraft. Finally, this thesis examines alternative reward system functions as well as changes to the software environment to bridge the gap between simulation and real world deployment environments. The design, evaluation, and experimental work summarized in this thesis demonstrates that deep reinforcement learning is able to be leveraged for the design and implementation of neural network controllers capable not only of maintaining stable flight, but also precision aerobatic maneuvers in real world settings. As such, this work provides a foundation for developing the next generation of flight control systems

강화학습을 이용한 KSTAR 토카막 운전 궤적 설계

학위논문(박사) -- 서울대학교대학원 : 공과대학 에너지시스템공학부, 2022. 8. 나용수.토카막에서 정밀한 물리 실험을 하기 위해서는 먼저 특정한 내부 플라즈마 상태를 달성하고 유지하는 것이 필요하다. 특히 상용 핵융합로 운전을 위해서는 자기유체역학적으로 안정적인 플라즈마 영역 내에서의 제어가 필수적이며, 고출력의 핵융합 반응을 일으킬 수 있는 플라즈마 상태를 유지할 수 있어야 한다. 기존에는 실험에서 목표로 하는 플라즈마 상태를 달성하기 위해, 다양한 토카막 운전 조건에서의 사전 시뮬레이션과 실험에서의 추가적인 시행착오가 필요하였다. 이 경우 많은 인적 노동력과 시간이 소요되었으며, 새로운 목표 상태들에 대해 매번 동일한 수준의 시행착오가 요구된다는 문제가 있다. 본 논문은 목표 플라즈마 상태를 달성하기 위한 토카막의 운전 경로를 설계하는 기계학습 기반의 알고리즘 개발을 다룬다. 해당 알고리즘은 기존의 상당한 시뮬레이션 및 시행착오를 수행하는 작업들을 대체할 수 있으며, 이를 통해 보다 빠르고 효율적으로 가능성 높은 운전 조건을 도출할 수 있다. 첫 번째로, 토카막 운전 설계 인공지능 모델의 훈련 환경에 해당하는 토카막 시뮬레이션 기술이 연구되었다. KSTAR 실험 데이터의 패턴을 학습하여 시간에 따른 플라즈마 상태를 순차적으로 예측하는 LSTM 기반의 인공신경망 모델을 개발하였다. 데이터 학습 과정에서 과적합 및 오차 누적 등의 문제를 해결하기 위해 다양한 수치적 기법들이 적용되었다. 학습된 모델은 KSTAR의 다양한 운전 시나리오 방전들에 대해 우수한 예측 정확도를 보여주었으며, 신뢰도 분석을 통해 모델이 과적합되지 않음을 확인하였다. 또한 해당 모델을 기반으로, 실시간 상호작용을 통한 가상 토카막 실험이 가능하도록 그래픽 사용자 인터페이스 (GUI)를 개발하였다. 해당 GUI 상에서 사용자가 토카막 운전 변수들을 조정함에 따라 플라즈마의 변화를 실시간으로 시각적으로 확인할 수 있기 때문에 물리 연구 뿐 아니라 전문가 교육용으로서의 의의가 있다. 두 번째로, 개발된 시뮬레이터 상에서 스스로 운전 변수들을 조정하여 목표로 하는 플라즈마 상태를 달성하는 인공지능 모델을 강화학습 기법을 이용하여 훈련하였다. 이를 통해 목표 플라즈마 상태를 달성하기 위한 적절한 토카막 운전 경로를 설계하는 알고리즘을 개발할 수 있다. 먼저 목표 β_N 달성을 위해 플라즈마 전류, 플라즈마 형태 및 가열 파워를 결정하는 모델을 훈련하였다. 훈련된 모델이 설계한 운전 경로를 이용하여 실험을 수행해본 결과 오차범위 내에서 목표 β_N이 도출됨을 검증하였다. 특히 한정된 가열 조건에서 높은 성능을 달성하기 위해 플라즈마 형태를 적절히 조정하여 가둠 성능을 향상시키는 것을 확인하였다. 이후 보다 더 구체적인 플라즈마 상태를 달성하기 위해, 플라즈마 압력 (β_p) 뿐 아니라 자기장 구조 (q_95) 및 내부 인덕턴스 (l_i)의 다중 파라미터들의 목표값을 동시에 달성케 하는 인공지능 모델 또한 훈련하였다. 해당 모델이 설계한 운전 경로를 실제 실험에 적용해본 결과, 다중 플라즈마 파라미터들이 성공적으로 목표값으로 제어됨을 확인하였다. 본 논문에서 개발된 기계학습 기반 알고리즘은 추후 고성능 운전 시나리오 연구에 도움을 줄 수 있으며, 정밀한 물리 조건을 요구하는 실험에서 초기 조건 달성을 위한 기술로 적용될 수 있을 것으로 기대된다. 추후 실시간 피드백 제어에 적용됨으로써 다양한 상황에서 자율적으로 제어되는 핵융합로 기술 개발을 위한 초석이 될 수 있을 것으로 전망한다.In order to conduct a sophisticated physics experiment in a tokamak, it is necessary to achieve and sustain a specific target plasma state first. Especially, the commercial fusion reactor requires controlling plasmas within a stable parametric range and maintaining a favorable plasma state for high fusion power generation. Conventionally, we had to conduct numerous simulations with various tokamak operating conditions and experiment with trials and errors for achieving a target plasma state. This takes lots of labor and time and requires the same level of trial and error for different targets each time. This thesis addresses the development of a reinforcement learning (RL)-based algorithm that designs the tokamak operation trajectory to achieve a given target plasma state. This algorithm replaces the conventional manual tasks of numerous simulative experiments and provides a probable tokamak operation condition faster and more efficiently. First, the tokamak simulator, corresponding to the training environment of the RL agent that designs the operation trajectory, was developed. An LSTM-based neural network was trained that sequentially predicts the plasma state over time by learning the patterns of the KSTAR experimental data. Various numerical techniques were applied to prevent overfitting and error accumulation during the training process. The trained model showed reasonable prediction accuracy for various operation scenarios in KSTAR, and reliability analyses verified that the model was not significantly overfitted. Furthermore, based on the trained model, we developed a graphical user interface (GUI) to enable virtual tokamak experiments through real-time interaction. By adjusting the tokamak operation parameters on the GUI, the user can visually check the plasma evolution in real time, which can be useful not only for physics research but also for expert education. Second, an artificial agent was trained using a reinforcement learning technique, that adjusts the operation parameters to achieve a target plasma state in the developed simulator. This agent can design a plausible tokamak operation trajectory to achieve a given target after training. First, the agent was trained to determine the plasma current, the plasma shape, and the heating power to achieve the target β_N. We conducted a KSTAR experiment with the operation trajectory designed by the trained agent, and it was verified that the target performance was achieved within the tolerance range. In particular, it was observed that the confinement enhancement factor was improved by adjusting the plasma shape to achieve high performance under limited heating conditions. Moreover, in order to achieve a more specific plasma state, another RL agent was trained to achieve multiple targets of β_p, q_95, and l_i simultaneously. The KSTAR experiment with the RL operation design showed that multiple plasma parameters were successfully controlled to the target values. The RL-based algorithm addressed in this thesis can provide clues for the research of advanced operation scenarios and can be applied to achieve initial plasma states in experiments that require sophisticated physical conditions. By applying this algorithm to real-time feedback control in the future, it will become a basis for developing a self-operating fusion reactor that can be autonomously controlled to achieve high power generation.1. Introduction １ 1.1. Advanced operation scenario in tokamak ３ 1.2. Machine learning in fusion research ４ 1.2.1. Precedent research ４ 1.2.2. AI operation trajectory design and control ６ 1.2.3. Simulation of tokamak plasmas １０ 1.3. Objective and outline of this dissertation １５ 2. Data-driven tokamak simulator １７ 2.1. Predictive modeling with DNN １８ 2.1.1. Construction and training of the LSTM-based model １８ 2.1.2. Demonstration and validation ２９ 2.2. Analysis for model reliability ３２ 2.2.1. Uncertainties in dataset ３２ 2.2.2. Consistency with prior knowledge ３４ 3. Operation trajectory design algorithm ３６ 3.1. Environment of the RL training ３７ 3.2. Control of normalized beta ３９ 3.2.1. The action, observation, and reward for RL ３９ 3.2.2. RL training ４４ 3.3. Simultaneous control of multiple parameters ４５ 3.3.1. The action, observation, and reward for RL ４５ 3.3.2. RL training ４９ 4. Validation in KSTAR ５３ 4.1. Control of normalized beta ５４ 4.1.1. Case 1: β_N control to 2.4 and 1.8 ５４ 4.1.2. Case 2: β_N control to 2.7 ５７ 4.1.3. Case 3: β_N control to 3.5 ６０ 4.2. Simultaneous control of multiple 0D parameters ６２ 4.2.1. Experiment with RL-designed trajectory ６２ 4.2.2. Comparison with other shots in dataset ６６ 5. Conclusion ６９ Bibliography ７１ Abstract in Korean ７７박

Theoretical and experimental application of neural networks in spaceflight control systems

“Spaceflight systems can enable advanced mission concepts that can help expand our understanding of the universe. To achieve the objectives of these missions, spaceflight systems typically leverage guidance and control systems to maintain some desired path and/or orientation of their scientific instrumentation. A deep understanding of the natural dynamics of the environment in which these spaceflight systems operate is required to design control systems capable of achieving the desired scientific objectives. However, mitigating strategies are critically important when these dynamics are unknown or poorly understood and/or modelled. This research introduces two neural network methodologies to control the translation and rotation dynamics of spaceflight systems. The first method uses a neural network to perform nonlinear estimation in the control space for both translational and attitude control. The second method uses an observer with a neural network to perform estimation outside the control space, and input-output feedback linearization using the estimated dynamics for both translational and attitude control. The methods are demonstrated for attitude control through simulation and hardware testing on the Wallops Arc-Second Pointer, a high-altitude balloon-borne spaceflight system. Results show that the two new methodologies can provide improved attitude control performance over the heritage control system. The methods are also demonstrated for translational and attitude control of two small spacecraft in a deep space environment, where they provide improved position and attitude control performance as compared to a traditional control method. This work demonstrates, through simulation and hardware testing, that the two neural network methods presented can offer improved translational and attitude control performance of spaceflight systems where the dynamic environment may be unknown or poorly understood and/or modeled”--Abstract, page iv

A Predictive Fuzzy-Neural Autopilot for the Guidance of Small Motorised Marine Craft

This thesis investigates the design and evaluation of a control system, that is able to adapt quickly to changes in environment and steering characteristics. This type of controller is particularly suited for applications with wide-ranging working conditions such as those experienced by small motorised craft. A small motorised craft is assumed to be highly agile and prone to disturbances, being thrown off-course very easily when travelling at high speed 'but rather heavy and sluggish at low speeds. Unlike large vessels, the steering characteristics of the craft will change tremendously with a change in forward speed. Any new design of autopilot needs to be to compensate for these changes in dynamic characteristics to maintain near optimal levels of performance. This study identities the problems that need to be overcome and the variables involved. A self-organising fuzzy logic controller is developed and tested in simulation. This type of controller learns on-line but has certain performance limitations. The major original contribution of this research investigation is the development of an improved self-adaptive and predictive control concept, the Predictive Self-organising Fuzzy Logic Controller (PSoFLC). The novel feature of the control algorithm is that is uses a neural network as a predictive simulator of the boat's future response and this network is then incorporated into the control loop to improve the course changing, as well as course keeping capabilities of the autopilot investigated. The autopilot is tested in simulation to validate the working principle of the concept and to demonstrate the self-tuning of the control parameters. Further work is required to establish the suitability of the proposed novel concept to other control

Ice Inspection Robot

The purpose of this project was to design, create, and test a robotic mobile platform capable of housing and interfacing with an ice thickness measuring sensor. This robot was designed to: drive across natural snow, ice, and water surfaces; follow a user-defined path; report live position and heading information to a user. The unique auger-drive system of this robot was designed to provide efficient movement across ice, as well as buoyancy and aquatic propulsion, in the case of broken ice. A user interface was also designed and implemented as part of this project. This interface was designed to: display the live information sent by the robot; allow the user to send instructions to the robot; prompt the user for input; inform the user of the program\u27s progress and state

Biologically Inspired Vision and Control for an Autonomous Flying Vehicle

This thesis makes a number of new contributions to control and sensing for unmanned vehicles. I begin by developing a non-linear simulation of a small unmanned helicopter and then proceed to develop new algorithms for control and sensing using the simulation. The work is field-tested in successful flight trials of biologically inspired vision and neural network control for an unstable rotorcraft. The techniques are more robust and more easily implemented on a small flying vehicle than previously attempted methods. ¶ ..

AAS/GSFC 13th International Symposium on Space Flight Dynamics

This conference proceedings preprint includes papers and abstracts presented at the 13th International Symposium on Space Flight Dynamics. Cosponsored by American Astronautical Society and the Guidance, Navigation and Control Center of the Goddard Space Flight Center, this symposium featured technical papers on a wide range of issues related to orbit-attitude prediction, determination, and control; attitude sensor calibration; attitude dynamics; and mission design