Chance-Constrained Trajectory Optimization for Safe Exploration and Learning of Nonlinear Systems

Learning-based control algorithms require data collection with abundant supervision for training. Safe exploration algorithms ensure the safety of this data collection process even when only partial knowledge is available. We present a new approach for optimal motion planning with safe exploration that integrates chance-constrained stochastic optimal control with dynamics learning and feedback control. We derive an iterative convex optimization algorithm that solves an \underline{Info}rmation-cost \underline{S}tochastic \underline{N}onlinear \underline{O}ptimal \underline{C}ontrol problem (Info-SNOC). The optimization objective encodes both optimal performance and exploration for learning, and the safety is incorporated as distributionally robust chance constraints. The dynamics are predicted from a robust regression model that is learned from data. The Info-SNOC algorithm is used to compute a sub-optimal pool of safe motion plans that aid in exploration for learning unknown residual dynamics under safety constraints. A stable feedback controller is used to execute the motion plan and collect data for model learning. We prove the safety of rollout from our exploration method and reduction in uncertainty over epochs, thereby guaranteeing the consistency of our learning method. We validate the effectiveness of Info-SNOC by designing and implementing a pool of safe trajectories for a planar robot. We demonstrate that our approach has higher success rate in ensuring safety when compared to a deterministic trajectory optimization approach.Comment: Submitted to RA-L 2020, review-

Vertical Take-Off and Landing Control via Dual-Quaternions and Sliding Mode

The landing and reusability of space vehicles is one of the driving forces into renewed interest in space utilization. For missions to planetary surfaces, this soft landing has been most commonly accomplished with parachutes. However, in spite of their simplicity, they are susceptible to parachute drift. This parachute drift makes it very difficult to predict where the vehicle will land, especially in a dense and windy atmosphere such as Earth. Instead, recent focus has been put into developing a powered landing through gimbaled thrust. This gimbaled thrust output is dependent on robust path planning and controls algorithms. Being able to have a powered landing with on-board real-time control algorithms is absolutely essential to exploring the solar system as it is the only effective way to bring heavy equipment or people to a planetary surface. A robust, efficient, and easy-to-use controls algorithm will be formulated to solve this controls problem known as the \emph{soft landing problem}. Through representing rigid body motion with dual-quaternions, translation and rotation can be represented in a single compact form that is free of singularities and provides the shortest path interpolation compared to any other formulation. These rigid bodies will be shown to follow a desired time-dependent orientation and position through one of the most powerful methods of modern control known for its accuracy, robustness, and easy tuning and implementation -- sliding mode control

Safe and accurate MAV Control, navigation and manipulation

This work focuses on the problem of precise, aggressive and safe Micro Aerial Vehicle (MAV) navigation as well as deployment in applications which require physical interaction with the environment. To address these issues, we propose three different MAV model based control algorithms that rely on the concept of receding horizon control. As a starting point, we present a computationally cheap algorithm which utilizes an approximate linear model of the system around hover and is thus maximally accurate for slow reference maneuvers. Aiming at overcoming the limitations of the linear model parameterisation, we present an extension to the first controller which relies on the true nonlinear dynamics of the system. This approach, even though computationally more intense, ensures that the control model is always valid and allows tracking of full state aggressive trajectories. The last controller addresses the topic of aerial manipulation in which the versatility of aerial vehicles is combined with the manipulation capabilities of robotic arms. The proposed method relies on the formulation of a hybrid nonlinear MAV-arm model which also takes into account the effects of contact with the environment. Finally, in order to enable safe operation despite the potential loss of an actuator, we propose a supervisory algorithm which estimates the health status of each motor. We further showcase how this can be used in conjunction with the nonlinear controllers described above for fault tolerant MAV flight. While all the developed algorithms are formulated and tested using our specific MAV platforms (consisting of underactuated hexacopters for the free flight experiments, hexacopter-delta arm system for the manipulation experiments), we further discuss how these can be applied to other underactuated/overactuated MAVs and robotic arm platforms. The same applies to the fault tolerant control where we discuss different stabilisation techniques depending on the capabilities of the available hardware. Even though the primary focus of this work is on feedback control, we thoroughly describe the custom hardware platforms used for the experimental evaluation, the state estimation algorithms which provide the basis for control as well as the parameter identification required for the formulation of the various control models. We showcase all the developed algorithms in experimental scenarios designed to highlight the corresponding strengths and weaknesses as well as show that the proposed methods can run in realtime on commercially available hardware.Open Acces

Space craft reliable trajectory tracking and landing using model predictive control with chance constraints

This work considers the study of chance constrained Model Predictive Control (MPC) for reliable spacecraft trajectory tracking and landing. Objectives of the master thesis: • To identify and study mathematical dynamic models of a spacecraft. • To study the trajectory design and landing schemes for a given mission. • To study the source of uncertainty in the model parameters and external disturbances. • To study the chance constrained MPC scheme for the reliable and optimal trajectory tracking and landing. • To testing the new analytic approximation approaches, Inner and Outer, for chance constraints. • To study appropriate MPC algorithms and implement on case-studies. In the first part of the thesis considers deterministic dynamical models of spacecraft are discussed. The first example is about the tracking of trajectory and soft landing on the surface of an asteroid EROS433, this model uses Cartesian coordinates. In the second example, in a similar way to the first example, the trajectory and soft landing is performed on the surface of a celestial body. It is assumed that the celestial body is a perfect sphere, something that does not happen in the first example. Thus, the second example uses a Spherical coordinate system. The third example is about a Lander that enters the Martian atmosphere. This Lander follows a designed trajectory until reaching a certain altitude over the Martian surface. At this altitude the Lander deploys a parachute to make the landing. To solve the deterministic examples described above, the following sequence of steps are: • pose the deterministic Nonlinear Optimal Control Problem (NOCP), • convert the infinite Optimal Control Problem (OCP) to a finite Nonlinear Programming Problem (NLP), applying the Runge-Kutta 4th order discretization method, • apply the Quasi-sequential method to the deterministic NLP obtained from the previous step, • solution of the reduced NLP obtained from the previous step using IpOpt software. The steps outlined above are also part of the Nonlinear Model Predictive Control (NMPC) approach. In the second part of the thesis, the same examples of the first part are used but now with stochastic variables. To find the control law in each model, the stochastic NMPC was used. The above mentioned approach begins with a chance constrained OCP. The latter is discretized obtaining an NLP. The problem with this NLP, with chance constraints, is that is very difficult to solve in analytic form. So these chance constraints are approached by a different method that exist in the state of the art. This thesis work is focused on approaching the chance constraints through Analytic Approximation Strategies, specifically by the recent: Inner and Outer Approximation methods. The chance constrained MPC is expensive from a computational point of view, but it allows to find a control law for a more reliable trajectory-tracking and soft landing . That is suitable for applications with random disturbances, model inaccuracies, and measurement errors.Tesi

Assessing plant design with regard to MPC performance

Model Predictive Control is ubiquitous in the chemical industry and offers great advantages over traditional controllers. Notwithstanding, new plants are being projected without taking into account how design choices affect the MPC’s ability to deliver better control and optimization. Thus a methodology to determine if a certain design option favours or hinders MPC performance would be desirable. This paper presents the economic MPC optimization index whose intended use is to provide a procedure to compare different designs for a given process, assessing how well they can be controlled and optimised by a zone constrained MPC. The index quantifies the economic benefits available and how well the plant performs under MPC control given the plant’s controllability properties, requirements and restrictions. The index provides a monetization measure of expected control performance. This approach assumes the availability of a linear state-space model valid within the control zone defined by the upper and lower bounds of each controlled and manipulated variable. We have used a model derived from simulation step tests as a practical way to use the method. The impact of model uncertainty on the methodology is discussed. An analysis of the effects of disturbances on the index illustrates how they may reduce profitability by restricting the ability of a MPC to reach dynamic equilibrium near process restrictions, which in turn increases product quality giveaway and costs. A case of study consisting of four alternative designs for a realistically sized crude oil atmospheric distillation plant is provided in order to demonstrate the applicability of the index

Enabling technologies for precise aerial manufacturing with unmanned aerial vehicles

The construction industry is currently experiencing a revolution with automation techniques such as additive manufacturing and robot-enabled construction. Additive Manufacturing (AM) is a key technology that can o er productivity improvement in the construction industry by means of o -site prefabrication and on-site construction with automated systems. The key bene t is that building elements can be fabricated with less materials and higher design freedom compared to traditional manual methods. O -site prefabrication with AM has been investigated for some time already, but it has limitations in terms of logistical issues of components transportation and due to its lack of design exibility on-site. On-site construction with automated systems, such as static gantry systems and mobile ground robots performing AM tasks, can o er additional bene ts over o -site prefabrication, but it needs further research before it will become practical and economical. Ground-based automated construction systems also have the limitation that they cannot extend the construction envelope beyond their physical size. The solution of using aerial robots to liberate the process from the constrained construction envelope has been suggested, albeit with technological challenges including precision of operation, uncertainty in environmental interaction and energy e ciency. This thesis investigates methods of precise manufacturing with aerial robots. In particular, this work focuses on stabilisation mechanisms and origami-based structural elements that allow aerial robots to operate in challenging environments. An integrated aerial self-aligning delta manipulator has been utilised to increase the positioning accuracy of the aerial robots, and a Material Extrusion (ME) process has been developed for Aerial Additive Manufacturing (AAM). A 28-layer tower has been additively manufactured by aerial robots to demonstrate the feasibility of AAM. Rotorigami and a bioinspired landing mechanism demonstrate their abilities to overcome uncertainty in environmental interaction with impact protection capabilities and improved robustness for UAV. Design principles using tensile anchoring methods have been explored, enabling low-power operation and explores possibility of low-power aerial stabilisation. The results demonstrate that precise aerial manufacturing needs to consider not only just the robotic aspects, such as ight control algorithms and mechatronics, but also material behaviour and environmental interaction as factors for its success.Open Acces

Towards real-time reinforcement learning control of a wave energy converter

The levellised cost of energy of wave energy converters (WECs) is not competitive with fossil fuel-powered stations yet. To improve the feasibility of wave energy, it is necessary to develop effective control strategies that maximise energy absorption in mild sea states, whilst limiting motions in high waves. Due to their model-based nature, state-of-the-art control schemes struggle to deal with model uncertainties, adapt to changes in the system dynamics with time, and provide real-time centralised control for large arrays of WECs. Here, an alternative solution is introduced to address these challenges, applying deep reinforcement learning (DRL) to the control of WECs for the first time. A DRL agent is initialised from data collected in multiple sea states under linear model predictive control in a linear simulation environment. The agent outperforms model predictive control for high wave heights and periods, but suffers close to the resonant period of the WEC. The computational cost at deployment time of DRL is also much lower by diverting the computational effort from deployment time to training. This provides confidence in the application of DRL to large arrays of WECs, enabling economies of scale. Additionally, model-free reinforcement learning can autonomously adapt to changes in the system dynamics, enabling fault-tolerant control

Chance-Constrained Trajectory Optimization for Safe Exploration and Learning of Nonlinear Systems

Learning-based control algorithms require data collection with abundant supervision for training. Safe exploration algorithms ensure the safety of this data collection process even when only partial knowledge is available. We present a new approach for optimal motion planning with safe exploration that integrates chance-constrained stochastic optimal control with dynamics learning and feedback control. We derive an iterative convex optimization algorithm that solves an Information-cost Stochastic Nonlinear Optimal Control problem (Info-SNOC). The optimization objective encodes control cost for performance and exploration cost for learning, and the safety is incorporated as distributionally robust chance constraints. The dynamics are predicted from a robust regression model that is learned from data. The Info-SNOC algorithm is used to compute a sub-optimal pool of safe motion plans that aid in exploration for learning unknown residual dynamics under safety constraints. A stable feedback controller is used to execute the motion plan and collect data for model learning. We prove the safety of rollout from our exploration method and reduction in uncertainty over epochs, thereby guaranteeing the consistency of our learning method. We validate the effectiveness of Info-SNOC by designing and implementing a pool of safe trajectories for a planar robot. We demonstrate that our approach has higher success rate in ensuring safety when compared to a deterministic trajectory optimization approach