ALGORITHMS AND OPTIMAL CONTROL FOR SPACECRAFT MAGNETIC ATTITUDE MANEUVERS

This study focused on providing applicable control solutions for spacecraft magnetic attitude control system. Basically, two main lines are pursued; first, developing detumbling control laws and second, an improvement in the three-axis attitude control schemes by extending magnetic rods activation time. Spacecraft, after separation from the launching mechanism, experiences a tumbling phase due to an undesired angular momentum. In this study, we present a new efficient variant of the B-dot detumbling law by introducing a substitute of the spacecraft angular velocity, based on the ambient magnetic field data. This B-dot law preserves the orthogonality, among the applied torque, dipole moment and magnetic field vectors. Most of the existing variants of the B-dot law in the literature don\u27t preserve this orthogonality. Furthermore, the problem of minimum-time spacecraft magnetic detumbling is revisited within the context of optimal control theory. Two formulations are presented; the first one assumes the availability of the angular velocity measurements for feedback. The second formulation assumes the availability of only the ambient magnetic field measurements in the feedback; the latter is considered another optimal-based B-dot law. A reduction in detumbling time is fulfilled by the proposed laws along with less power consumption for the proposed B-dot laws. In magnetic attitude maneuvers, magnetic rods and magnetometers usually operate alternatively, to avoid the magnetic rods\u27 noise effect on magnetometers measurements. Because of that, there will be no control authority over the spacecraft during the magnetometer measurement period. Hence longer maneuver times are usually experienced. In this study, a control scheme that enables the extension of the magnetic rods’ activation time is developed, regardless of the attitude control law. The key concept is replacing the real magnetic field measurement by a pseudo measurement, which is computed based on other sensors measurements. By applying a known command to the spacecraft and measuring the spacecraft response, it is possible to compute the ambient magnetic field around the spacecraft. The system mathematical singularity is solved using the Tikhonov regularization approach. Another developed approach estimates the magnetic field, using a relatively simple and fast dynamic model inside a Multiplicative Extended Kalman Filter. A less maneuver time with less power consumption are fulfilled. These control approaches are further validated using real telemetry data from CASSIOPE mission. This dissertation develops a stability analysis for the spacecraft magnetic attitude control, taking into consideration the alternate operation between the magnetic rods and the magnetometers. It is shown that the system stability degrades because of this alternate operation, supporting the proposed approach of extending the operation time of the magnetic rods

Combined state and parameter estimation for Hammerstein systems with time-delay using the Kalman filtering

This paper discusses the state and parameter estimation problem for a class of Hammerstein state space systems with time-delay. Both the process noise and the measurement noise are considered in the system. Based on the observable canonical state space form and the key term separation, a pseudo-linear regressive identification model is obtained. For the unknown states in the information vector, the Kalman filter is used to search for the optimal state estimates. A Kalman-filter based least squares iterative and a recursive least squares algorithms are proposed. Extending the information vector to include the latest information terms which are missed for the time-delay, the Kalman-filter based recursive extended least squares algorithm is derived to obtain the estimates of the unknown time-delay, parameters and states. The numerical simulation results are given to illustrate the effectiveness of the proposed algorithms

Delta Hedging of Financial Options Using Reinforcement Learning and an Impossibility Hypothesis

In this thesis we take a fresh perspective on delta hedging of financial options as undertaken by market makers. The current industry standard of delta hedging relies on the famous Black Scholes formulation that prescribes continuous time hedging in a way that allows the market maker to remain risk neutral at all times. But the Black Scholes formulation is a deterministic model that comes with several strict assumptions such as zero transaction costs, log normal distribution of the underlying stock prices, etc. In this paper we employ Reinforcement Learning to redesign the delta hedging problem in way that allows us to relax the strict assumption of risk neutrality and allows us to embed market realities such as transaction costs right at the outset. Our main argument is that by taking a controlled amount of risk and encouraging some uncertainty (referred to as exploration) in the hedged position, the market maker is able to generate incremental profit in the entire operation. Our model does not assume any parametric distribution for the underlying stock prices and is fundamentally online in nature i.e. learns on the go

Application of artificial neural networks to weighted interval Kalman filtering

The interval Kalman filter is a variant of the traditional Kalman filter for systems with bounded parametric uncertainty. For such systems, modelled in terms of intervals, the interval Kalman filter provides estimates of the system state also in the form of intervals, guaranteed to contain the Kalman filter estimates of all point-valued systems contained in the interval model. However, for practical purposes, a single, point-valued estimate of the system state is often required. This point value can be seen as a weighted average of the interval bounds provided by the interval Kalman filter. This article proposes a methodology based on the application of artificial neural networks by which an adequate weight can be computed at each time step, whereby the weighted average of the interval bounds approximates the optimal estimate or estimate which would be obtained using a Kalman filter if no parametric uncertainty was present in the system model, even when this is not the case. The practical applicability and robustness of the method are demonstrated through its application to the navigation of an uninhabited surface vehicle. © IMechE 2014

Combined state and parameter estimation for on-line applications,

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Aeronautics and Astronautics, 1972.Bibliography: leaves 354-361.by Peter S. Maybeck.Ph.D

An Investigation of State-Space Model Fidelity for SSME Data

In previous studies, a variety of unsupervised anomaly detection techniques for anomaly detection were applied to SSME (Space Shuttle Main Engine) data. The observed results indicated that the identification of certain anomalies were specific to the algorithmic method under consideration. This is the reason why one of the follow-on goals of these previous investigations was to build an architecture to support the best capabilities of all algorithms. We appeal to that goal here by investigating a cascade, serial architecture for the best performing and most suitable candidates from previous studies. As a precursor to a formal ROC (Receiver Operating Characteristic) curve analysis for validation of resulting anomaly detection algorithms, our primary focus here is to investigate the model fidelity as measured by variants of the AIC (Akaike Information Criterion) for state-space based models. We show that placing constraints on a state-space model during or after the training of the model introduces a modest level of suboptimality. Furthermore, we compare the fidelity of all candidate models including those embodying the cascade, serial architecture. We make recommendations on the most suitable candidates for application to subsequent anomaly detection studies as measured by AIC-based criteria

Approximate Dynamic Programming with Parallel Stochastic Planning Operators

This thesis presents an approximate dynamic programming (ADP) technique for environment modelling agents. The agent learns a set of parallel stochastic planning operators (P-SPOs) by evaluating changes in its environment in response to actions, using an association rule mining approach. An approximate policy is then derived by iteratively improving state value aggregation estimates attached to the operators using the P-SPOs as a model in a Dyna-Q-like architecture. Reinforcement learning and dynamic programming are powerful techniques for automated agent decision making in stochastic environments. Dynamic programming is effective when there is a known environment model, while reinforcement learning is effective when a model is not available. The techniques derive a policy: a mapping from each environment state to an action which optimizes the long term reward the agent receives. The standard methods become less effective as the state space for the environment increases because they require values to be associated with each state, the storage and processing of which is exponential to the number of state variables. Resolving this “curse of dimensionality” is an important topic of research amongst all communities working on this problem. Two key methods are to: (i) derive an estimate of the value (approximate dynamic programming) using function approximation or state aggregation; or (ii) build a model of the environment from experience. This thesis presents a method of combining these approaches by exploiting structure in the state transition and value functions captured in a set of planning operators which are learnt through experience in the environment. Standard planning operators define the deterministic changes that occur in an environment in response to an action. This work presents Parallel Stochastic Planning Operators (P-SPOs), a novel form of planning operator providing a structured model of the state transition function in environments which are both non-deterministic and for which changes can occur outside the influence of actions. Next, an automated method for extracting P-SPOs from observations in an environment is explored using an adaptation of association rule mining. Finally, methods of relating the state transition structure encapsulated in the P-SPOs to state values, using the operators to store state value aggregation estimates, are evaluated. The framework described provides a method by which approximate dynamic programming can be applied by designers of AI agents and AI planning systems for which they have minimal prior knowledge. The framework and P-SPO based implementations are tested against standard techniques in two bench-mark stochastic environments: a “slippery gripper” block painting robot; and a “predator-prey” agent environment. Experimental results show that an agent using a P-SPO-based approach is able to learn an accurate model of its environment if successor state variables exhibit conditional independence, and an approximate model in the non-independent case. Results also demonstrate that the agent’s ability to generalise to previously unseen states using the model allow it to form an improved policy over an agent employing a standard Dyna-Q based technique. Finally, an approximate policy stored in state aggregation estimates attached to operators is shown to be optimal in experiments for which the P-SPO set contains sufficient information for effective aggregations to be formed

Bias analysis in mode-based Kalman filters for stochastic hybrid systems

Doctor of PhilosophyDepartment of Electrical and Computer EngineeringBalasubramaniam NatarajanStochastic hybrid system (SHS) is a class of dynamical systems that experience interaction of both discrete mode and continuous dynamics with uncertainty. State estimation for SHS has attracted research interests for decades with Kalman filter based solutions dominating the area. Mode-based Kalman filter is an extended version of the traditional Kalman filter for SHS. In general, as Kalman filter is unbiased for non-hybrid system estimation, prior research efforts primarily focus on the behavior of error covariance. In SHS state estimate, mode mismatch errors could result in a bias in the mode-based Kalman filter and have impacts on the continuous state estimation quality. The relationship between mode mismatch errors and estimation stability is an open problem that this dissertation attempts to address. Specifically, the probabilistic model of mode mismatch errors can be independent and identically distributed (i.i.d.), correlated across different modes and correlated across time. The proposed approach builds on the idea of modeling the bias evolution as a transformed system. The statistical convergence of the bias dynamics is then mapped to the stability of the transformed system. For each specific model of the mode mismatch error, the system matrix of the transformed system varies which results in challenges for the stability analysis. For the first time, the dissertation derives convergence conditions that provide tolerance regions for the mode mismatch error for three mode mismatch situations. The convergence conditions are derived based on generalized spectral radius theorem, Lyapunov theorem, Schur stability of a matrix polytope and interval matrix method. This research is fundamental in nature and its application is widespread. For example, the spatially and timely correlated mode mismatch errors can effectively capture cyber-attacks and communication link impairments in a cyber-physical system. Therefore, the theory and techniques developed in this dissertation can be used to analyze topology errors in any networked system such as smart grid, smart home, transportation, flight management system etc. The main results provide new insights on the fidelity in discrete state knowledge needed to maintain the performance of a mode-based Kalman filter and provide guidance on design of estimation strategies for SHS