173 research outputs found
Data-Driven Control with Learned Dynamics
This research focuses on studying data-driven control with dynamics that are actively learned from machine learning algorithms. With system dynamics being identified using neural networks either explicitly or implicitly, we can apply control following either a model-based approach or a model-free approach. In this thesis, the two different methods are explained in detail and finally compared to shed light on the emerging data-driven control research field.
In the first part of the thesis, we first introduce state-of-art Reinforcement Learning (RL) algorithm representing data-driven control using a model-free learning approach. We discuss the advantages and shortcomings of the current RL algorithms and motivate our study to search for a model-based control which is physics-based and also provides better model interpretability. We then propose a novel data-driven, model-based approach for the optimal control of the dynamical system. The proposed approach relies on the Deep Neural Network (DNN) based learning of Koopman operator and therefore is named as Deep Learning of Koopman Representation for Control (DKRC). In particular, DNN is employed for the data-driven identification of basis function used in the linear lifting of nonlinear control system dynamics. One a linear representation of system dynamics is learned, we can implement classic control algorithms such as iterative Linear Quadratic Regulator (iLQR) and Model Predictive Control (MPC) for optimal control design. The controller synthesis is purely data-driven and does not rely on prior domain knowledge. The OpenAI Gym environment is used for simulations of various control problems. The method is applied to three classic dynamical systems on OpenAI Gym environment to demonstrate the capability.
In the second part, we compare the proposed method with a state-of-art model-free control method based on an actor-critic architecture – Deep Deterministic Policy Gradient (DDPG), which has been proved to be effective in various dynamical systems. Two examples are provided for comparison, i.e., classic Inverted Pendulum and Lunar Lander Continuous Control. We compare these two methods in terms of control strategies and the effectiveness under various initialization conditions from the results of the experiments. We also examine the learned dynamic model from DKRC with the analytical model derived from the Euler-Lagrange Linearization method, demonstrating the accuracy in the learned model for unknown dynamics from a data-driven sample-efficient approach
Deep Reinforcement Learning with semi-expert distillation for autonomous UAV cinematography
Unmanned Aerial Vehicles (UAVs, or drones) have revolutionized modern media production. Being rapidly deployable “flying cameras”, they can easily capture aesthetically pleasing aerial footage of static or moving filming targets/subjects. Current approaches rely either on manual UAV/gimbal control by human experts or on a combination of complex computer vision algorithms and hardware configurations for automating the flight+flying process. This paper explores an efficient Deep Reinforcement Learning (DRL) alternative, which implicitly merges the target detection and path planning steps into a single algorithm. To achieve this, a baseline DRL approach is augmented with a novel policy distillation component, which transfers knowledge from a suitable, semi-expert Model Predictive Control (MPC) controller into the DRL agent. Thus, the latter is able to autonomously execute a specific UAV cinematography task with purely visual input. Unlike the MPC controller, the proposed DRL agent does not need to know the 3D world position of the filming target during inference. Experiments conducted in a photorealistic simulator showcase superior performance and training speed compared to the baseline agent while surpassing the MPC controller in terms of visual occlusion avoidance
A Survey of Offline and Online Learning-Based Algorithms for Multirotor UAVs
Multirotor UAVs are used for a wide spectrum of civilian and public domain
applications. Navigation controllers endowed with different attributes and
onboard sensor suites enable multirotor autonomous or semi-autonomous, safe
flight, operation, and functionality under nominal and detrimental conditions
and external disturbances, even when flying in uncertain and dynamically
changing environments. During the last decade, given the
faster-than-exponential increase of available computational power, different
learning-based algorithms have been derived, implemented, and tested to
navigate and control, among other systems, multirotor UAVs. Learning algorithms
have been, and are used to derive data-driven based models, to identify
parameters, to track objects, to develop navigation controllers, and to learn
the environment in which multirotors operate. Learning algorithms combined with
model-based control techniques have been proven beneficial when applied to
multirotors. This survey summarizes published research since 2015, dividing
algorithms, techniques, and methodologies into offline and online learning
categories, and then, further classifying them into machine learning, deep
learning, and reinforcement learning sub-categories. An integral part and focus
of this survey are on online learning algorithms as applied to multirotors with
the aim to register the type of learning techniques that are either hard or
almost hard real-time implementable, as well as to understand what information
is learned, why, and how, and how fast. The outcome of the survey offers a
clear understanding of the recent state-of-the-art and of the type and kind of
learning-based algorithms that may be implemented, tested, and executed in
real-time.Comment: 26 pages, 6 figures, 4 tables, Survey Pape
Discussion on Different Controllers Used for the Navigation of Mobile Robot
Robots that can comprehend and navigate their surroundings independently on their own are considered intelligent mobile robots (MR). Using a sophisticated set of controllers, artificial intelligence (AI), deep learning (DL), machine learning (ML), sensors, and computation for navigation, MR\u27s can understand and navigate around their environments without even being connected to a cabled source of power. Mobility and intelligence are fundamental drivers of autonomous robots that are intended for their planned operations. They are becoming popular in a variety of fields, including business, industry, healthcare, education, government, agriculture, military operations, and even domestic settings, to optimize everyday activities. We describe different controllers, including proportional integral derivative (PID) controllers, model predictive controllers (MPCs), fuzzy logic controllers (FLCs), and reinforcement learning controllers used in robotics science. The main objective of this article is to demonstrate a comprehensive idea and basic working principle of controllers utilized by mobile robots (MR) for navigation. This work thoroughly investigates several available books and literature to provide a better understanding of the navigation strategies taken by MR. Future research trends and possible challenges to optimizing the MR navigation system are also discussed
How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review
Context: Machine Learning (ML) has been at the heart of many innovations over
the past years. However, including it in so-called 'safety-critical' systems
such as automotive or aeronautic has proven to be very challenging, since the
shift in paradigm that ML brings completely changes traditional certification
approaches.
Objective: This paper aims to elucidate challenges related to the
certification of ML-based safety-critical systems, as well as the solutions
that are proposed in the literature to tackle them, answering the question 'How
to Certify Machine Learning Based Safety-critical Systems?'.
Method: We conduct a Systematic Literature Review (SLR) of research papers
published between 2015 to 2020, covering topics related to the certification of
ML systems. In total, we identified 217 papers covering topics considered to be
the main pillars of ML certification: Robustness, Uncertainty, Explainability,
Verification, Safe Reinforcement Learning, and Direct Certification. We
analyzed the main trends and problems of each sub-field and provided summaries
of the papers extracted.
Results: The SLR results highlighted the enthusiasm of the community for this
subject, as well as the lack of diversity in terms of datasets and type of
models. It also emphasized the need to further develop connections between
academia and industries to deepen the domain study. Finally, it also
illustrated the necessity to build connections between the above mention main
pillars that are for now mainly studied separately.
Conclusion: We highlighted current efforts deployed to enable the
certification of ML based software systems, and discuss some future research
directions.Comment: 60 pages (92 pages with references and complements), submitted to a
journal (Automated Software Engineering). Changes: Emphasizing difference
traditional software engineering / ML approach. Adding Related Works, Threats
to Validity and Complementary Materials. Adding a table listing papers
reference for each section/subsection
- …