Reinforcement learning-based approximate optimal control for attitude reorientation under state constraints

This paper addresses the attitude reorientation problems of rigid bodies under multiple state constraints. A novel reinforcement learning (RL)-based approximate optimal control method is proposed to make the trade-off between control cost and performance. The novelty lies in that it guarantees constraint handling abilities on attitude forbidden zones and angular-velocity limits. To achieve this, barrier functions are employed to encode the constraint information into the cost function. Then an RL-based learning strategy is developed to approximate the optimal cost function and control policy. A simplified critic-only neural network (NN) is employed to replace the conventional actor-critic structure once adequate data is collected online. This design guarantees the uniform boundedness of reorientation errors and NN weight estimation errors subject to the satisfaction of a finite excitation condition, which is a relaxation compared with the persistent excitation condition that is typically required for this class of problems. More importantly, all underlying state constraints are strictly obeyed during the online learning process. The effectiveness and advantages of the proposed controller are verified by both numerical simulations and experimental tests based on a comprehensive hardware-in-loop testbed

Applied Reachability Analysis for Time-Optimal Spacecraft Attitude Reorientations

Satellite attitude reorientation has been of significant interest in astronautical engineering, and being able to reorient in a time-optimal manner has been of exceeding interest since the 1970s. Ensuring a spacecraft mission set can be conducted within a certain amount of time begs the question of whether or not a certain maneuver can be completed with a bounded control. This thesis answers that question by using the concept of reachability to provide reachable sets for different spacecraft reorientation scenarios. The reachable sets generated provide a range of initial states that guarantee a satellite reach a desired end orientation given a certain time constraint. Prior research providing a formal approach of applying reachability to spacecraft attitude maneuvers has not been found. It was found that using Modified Rodriguez Parameters (MRPs) to generate reachable sets is more efficient than other attitude parameterizations. It was also found that the linearized MRP dynamics provide a valid time optimal solution for the true, nonlinear dynamics of medium angle attitude maneuvers. This linearized version of the dynamics was used to formulate an optimal control policy for spacecraft reorientations with bounded controls

Spacecraft nonlinear attitude control with bounded control input

The research in this thesis deals with nonlinear control of spacecraft attitude stabilization and tracking manoeuvres and addresses the issue of control toque saturation on a priori basis. The cascaded structure of spacecraft attitude kinematics and dynamics makes the method of integrator backstepping preferred scheme for the spacecraft nonlinear attitude control. However, the conventional backstepping control design method may result in excessive control torque beyond the saturation bound of the actuators. While remaining within the framework of conventional backstepping control design, the present work proposes the formulation of analytical bounds for the control torque components as functions of the initial attitude and angular velocity errors and the gains involved in the control design procedure. The said analytical bounds have been shown to be useful for tuning the gains in a way that the guaranteed maximum torque upper bound lies within the capability of the actuator and, hence, addressing the issue of control input saturation. Conditions have also been developed as well as the generalization of the said analytical bounds which allow for the tuning of the control gains to guarantee prescribed stability with the additional aim that the control action avoids reaching saturation while anticipating the presence of bounded external disturbance torque and uncertainties in the spacecraft moments of inertia. Moreover, the work has also been extended blending it with the artificial potential function method for achieving autonomous capability of avoiding pointing constraints for the case of spacecraft large angle slew manoeuvres. The idea of undergoing such manoeuvres using control moment gyros to track commanded angular momentum rather than a torque command has also been studied. In this context, a gimbal position command generation algorithm has been proposed for a pyramid-type cluster of four single gimbal control moment gyros. The proposed algorithm not only avoids the saturation of the angular momentum input from the control moment gyro cluster but also exploits its maximum value deliverable by the cluster along the direction of the commanded angular momentum for the major part of the manoeuvre. In this way, it results in rapid spacecraft slew manoeuvres. The ideas proposed in the thesis have also been validated using numerical simulations and compared with results already existing in the literature

ADP-based spacecraft attitude control under actuator misalignment and pointing constraints

This paper is devoted to real-time optimal attitude reorientation control of rigid spacecraft control. Particularly, two typical practical problems - actuator misalignment and forbidden pointing constraints are considered. Within the framework of adaptive dynamic programming (ADP), a novel constrained optimal attitude control scheme is proposed. In this design, a special reward function is developed to characterize the environment feedback and deal with the pointing constraints. Notably, a novel argument term is introduced to the reward function for overcoming the inevitable difficulty in actuator misalignment. By virtue of the Lyapunov stability theory, the ultimate boundedness of state error and the optimality of the proposed method can be guaranteed. Finally, the effectiveness and performance of the developed ADP-based controller are evaluated by not only numerical simulations but also experimental tests with a hardware-in-loop platform

Optimal control problems solved via swarm intelligence

Questa tesi descrive come risolvere problemi di controllo ottimo tramite swarm in telligence. Grande enfasi viene posta circa la formulazione del problema di controllo ottimo, in particolare riguardo a punti fondamentali come l’identificazione delle incognite, la trascrizione numerica e la scelta del risolutore per la programmazione non lineare. L’algoritmo Particle Swarm Optimization viene preso in considerazione e la maggior parte dei problemi proposti sono risolti utilizzando una formulazione differential flatness. Quando viene usato l’approccio di dinamica inversa, il problema di ottimo relativo ai parametri di trascrizione è risolto assumendo che le traiettorie da identificare siano approssimate con curve B-splines. La tecnica Inverse-dynamics Particle Swarm Optimization, che viene impiegata nella maggior parte delle applicazioni numeriche di questa tesi, è una combinazione del Particle Swarm e della formulazione differential flatness. La tesi investiga anche altre opportunità di risolvere problemi di controllo ottimo tramite swarm intelligence, per esempio usando un approccio di dinamica diretta e imponendo a priori le condizioni necessarie di ottimalitá alla legge di controllo. Per tutti i problemi proposti, i risultati sono analizzati e confrontati con altri lavori in letteratura. Questa tesi mostra quindi the algoritmi metaeuristici possono essere usati per risolvere problemi di controllo ottimo, ma soluzioni ottime o quasi-ottime possono essere ottenute al variare della formulazione del problema.This thesis deals with solving optimal control problems via swarm intelligence. Great emphasis is given to the formulation of the optimal control problem regarding fundamental issues such as unknowns identification, numerical transcription and choice of the nonlinear programming solver. The Particle Swarm Optimization is taken into account, and most of the proposed problems are solved using a differential flatness formulation. When the inverse-dynamics approach is used, the transcribed parameter optimization problem is solved assuming that the unknown trajectories are approximated with B-spline curves. The Inverse-dynamics Particle Swarm Optimization technique, which is employed in the majority of the numerical applications in this work, is a combination of Particle Swarm and differential flatness formulation. This thesis also investigates other opportunities to solve optimal control problems with swarm intelligence, for instance using a direct dynamics approach and imposing a-priori the necessary optimality conditions to the control policy. For all the proposed problems, results are analyzed and compared with other works in the literature. This thesis shows that metaheuristic algorithms can be used to solve optimal control problems, but near-optimal or optimal solutions can be attained depending on the problem formulation

Construction of control barrier function and C2C^2 reference trajectory for constrained attitude maneuvers

Constrained attitude maneuvers have numerous applications in robotics and aerospace. In our previous work, a general framework to this problem was proposed with resolution completeness guarantee. However, a smooth reference trajectory and a low-level safety-critical controller were lacking. In this work, we propose a novel construction of a $C^2$ continuous reference trajectory based on B\'ezier curves on $SO(3)$ that evolves within predetermined cells and eliminates previous stop-and-go behavior. Moreover, we propose a novel zeroing control barrier function on $SO(3)$ that provides a safety certificate over a set of overlapping cells on $SO(3)$ while avoiding nonsmooth analysis. The safety certificate is given as a linear constraint on the control input and implemented in real-time. A remedy is proposed to handle the states where the coefficient of the control input in the linear constraint vanishes. Numerical simulations are given to verify the advantages of the proposed method.Comment: Extended version of an accepted paper at IEEE CDC 202

Technology for large space systems: A bibliography with indexes (supplement 12)

A bibliography listing 516 reports, articles and other documents introduced into the NASA scientific and technical information system between July 1, 1984 and December 31, 1984 is presented. Its purpose is to provide helpful information to the researcher, manager, and designer in technology development and mission design in the area of Large Space System Technology. Subject matter is grouped according to system, interactive analysis and design, structural and thermal analysis and design, structural concepts and control systems, electronics, advanced materials, assembly concepts, propulsion, and solar power satellite systems

Developing an Integrated Guidance and Control System for Reactive Free-Flyer Maneuvering

The use of highly autonomous free-flying spacecraft has been investigated for potential utility in future human spaceflight endeavors. In general, ’free-flyer’ robots are small, self-sufficient spacecraft that operate near the exterior of larger space structures, such as the International Space Station, and are designed to provide support during various operations. Free-flyer designs and concepts often include a large degree of autonomy to provide mission support with little operational overhead. One of the building blocks of autonomous free-flyer behavior is safe and reliable point-to-point maneuvering during proximity operations. This thesis explores the development and simulation of an integrated guidance and control (G&C) system to enable safe free-flyer point-to-point maneuvering in proximity to larger space structures, including the avoidance of collision and jet plume impingement. The foundation for this system is an existing trajectory planning method introduced by Roger [1]. This method represents the free-space as a discrete harmonic potential field and uses the resulting field gradient as a condition for generating collision-free trajectories that efficiently account for natural dynamics. A real-time guidance process is built around this method that can manage an internal model of the environment based upon obstacle mapping data and react quickly to dynamic obstacles. A linear-programming jet selection technique is implemented to fulfill six DOF velocity impulse commands using a free-flyer’s RCS propulsion system, and additionally is augmented to include jet plume impingement avoidance functionality. Finally, an attitude con-troller process was developed and implemented to enable the free-flyer to reach and track a desired attitude during translational maneuvers. To verify the system’s capabilities, a test-bed simulation was developed using the SpaceCRAFT platform, specifically utilizing it’s modular, asynchronous architecture. In a set of four maneuver-ing tests set in distinct obstacle environments, the G&C system demonstrated the ability to maneuver the free-flyer to the goal state along collision-free trajectories. In three of the test cases, the plume avoidance strategy results in a large reduction in the accumulated plume cost (36-54%). Overall, these simulation results demonstrate that the system enabled point-to-point maneuvering for a reference free-flyer design, and support its feasibility and practicality. This work represents a somewhat unique approach to free-flyer autonomous maneuvering in that it departs from a trajectory planning, or offline-online paradigm. Instead, this approach relies on the refinement of an internal model of the environment and the resulting potential field to per-form reactive path-finding. This approach results in better overall flexibility when the free-flyer lacks knowledge of the position, geometry, and motion of nearby obstacles. In the future, the integration of Simultaneous Localization and Mapping (SLAM) and 3D mapping algorithms will help to further verify the feasibility of this approach. Other future improvements include integrating more robust methods of dynamic obstacle avoidance and automated positioning and scaling of the potential field grid