Reinforcement Learning for Attitude Control of a Spacecraft with Flexible Appendages

This study explores the reinforcement learning (RL) approach to constructing attitude control strategies for a LEOsatellite with flexible appendages. Attitude control system actuated by a set of three reaction wheels is considered.The satellite is assumed to move in a circular low Earth orbit under the action of gravity-gradient torque, randomdisturbance torque, and oscillations excited in flexible appendages. The control policy for rest-to-rest slew maneuversis learned via the Proximal Policy Optimization (PPO) technique. The robustness of the obtained control policy isanalyzed and compared to that of conventional controllers. The first part of the study is focused on problem formulationin terms of Markov Decision Processes, analysis of different reward-shaping techniques, and finally training the RL-agent and comparing the obtained results with the state-of-the-art RL-controllers as well as with the performance ofa commonly used quaternion feedback regulator (Lyapunov-based PD controller). We then proceed to consider thesame spacecraft with flexible appendages added to its structure. Equations of excitable oscillations are appended tothe system and coupling terms are added describing the interactions between the main rigid body and the flexiblestructures. The dynamics of the rigid spacecraft thus becomes coupled with that of its flexible appendages and thecontrol strategy should change accordingly in order to prevent actions that entail excitation of oscillation modes.Again PPO is used to learn the control policy for rest-to-rest slew maneuvers in the extended system. All in all,the proposed reinforcement learning strategy is shown to converge to a policy that matches the performance of thequaternion feedback regulator for a rigid spacecraft. It is also shown that a policy can be trained to take into accountthe highly nonlinear dynamics caused by the presence of flexible elements that need to be brought to rest in the requiredattitude. We also discuss the advantages of the reinforcement learning approach such as robustness and ability of onlinelearning pertaining to the systems that require a high level of autonom

Sequences close to periodic

The paper is a survey of notions and results related to classical and new generalizations of the notion of a periodic sequence. The topics related to almost periodicity in combinatorics on words, symbolic dynamics, expressibility in logical theories, algorithmic computability, Kolmogorov complexity, number theory, are discussed.Comment: In Russian. 76 pages, 6 figure

Attitude control algorithms in a swarm of cubesats: Kriging interpolation and coordinated data exchange

This study is a part of the Skoltech University project to deploy a swarm of four identical 3U CubeSats in LEO. The CubeSats are to be equipped with gamma-ray sensors and their collective behavior will be exhibited in detecting gamma-ray bursts and coordinated attitude control. We consider a fully magnetic attitude control system, comprising a magnetometer as a part of attitude determination routine and three orthogonal magnetorquers as actuators. Having implemented and tested the conventional three-axis magnetic attitude determination and control algorithms, we proceed to study how the performance of such ADCS may be enhanced by using measurements and state vectors exchange. We interpolate the exchanged data, using the Kriging algorithm in conjunction with Extended Kalman filter and Lyapunov-based controller, since it provides the auto correlation and variance information about the environment of the magnetic field, which is of utmost importance for heterogeneous and noisy fields. In our simulations we compare the performance of the controller for a single satellite to that of the satellite in the swarm of CubeSats, which maintains the form of a regular tetrahedron and carries out distributed measurements with interpolation. Improved attitude stabilization for the latter scenario is demonstrated by mean squared errors