1,751 research outputs found

    Hierarchical Decomposition of Nonlinear Dynamics and Control for System Identification and Policy Distillation

    Full text link
    The control of nonlinear dynamical systems remains a major challenge for autonomous agents. Current trends in reinforcement learning (RL) focus on complex representations of dynamics and policies, which have yielded impressive results in solving a variety of hard control tasks. However, this new sophistication and extremely over-parameterized models have come with the cost of an overall reduction in our ability to interpret the resulting policies. In this paper, we take inspiration from the control community and apply the principles of hybrid switching systems in order to break down complex dynamics into simpler components. We exploit the rich representational power of probabilistic graphical models and derive an expectation-maximization (EM) algorithm for learning a sequence model to capture the temporal structure of the data and automatically decompose nonlinear dynamics into stochastic switching linear dynamical systems. Moreover, we show how this framework of switching models enables extracting hierarchies of Markovian and auto-regressive locally linear controllers from nonlinear experts in an imitation learning scenario.Comment: 2nd Annual Conference on Learning for Dynamics and Contro

    The past, present, and future of macroeconomic forecasting

    Get PDF
    Broadly defined, macroeconomic forecasting is alive and well. Nonstructural forecasting, which is based largely on reduced-form correlations, has always been well and continues to improve. Structural forecasting, which aligns itself with economic theory and, hence, rises and falls with theory, receded following the decline of Keynesian theory. In recent years, however, powerful new dynamic stochastic general equilibrium theory has been developed, and structural macroeconomic forecasting is poised for resurgence.Forecasting

    Statistical Machine Learning for Modeling and Control of Stochastic Structured Systems

    Get PDF
    Machine learning and its various applications have driven innovation in robotics, synthetic perception, and data analytics. The last decade especially has experienced an explosion in interest in the research and development of artificial intelligence with successful adoption and deployment in some domains. A significant force behind these advances has been an abundance of data and the evolution of simple computational models and tools with a capacity to scale up to massive learning automata. Monolithic neural networks with billions of parameters that rely on automatic differentiation are a prime example of the significant role efficient computation has had on supercharging the ability of well-established representations to extract intelligent patterns from unstructured data. Nonetheless, despite the strides taken in the digital domains of vision and natural language processing, applications of optimal control and robotics significantly trail behind and have not been able to capitalize as much on the latest trends of machine learning. This discrepancy can be explained by the limited transferability of learning concepts that rely on full differentiability to the heavily structured physical and human interaction environments, not to mention the substantial cost of data generation on real physical systems. Therefore, these factors severely limit the application scope of loosely-structured over-parameterized data-crunching machines in the mechanical realm of robot learning and control. This thesis investigates modeling paradigms of hierarchical and switching systems to tackle some of the previously highlighted issues. This research direction is motivated by insights into universal function approximation via local cooperating units and the promise of inherently regularized representations through explicit structural design. Moreover, we explore ideas from robust optimization that address model mismatch issues in statistical models and outline how related methods may be used to improve the tractability of state filtering in stochastic hybrid systems. In Chapter 2, we consider hierarchical modeling for general regression problems. The presented approach is a generative probabilistic interpretation of local regression techniques that approximate nonlinear functions through a set of local linear or polynomial units. The number of available units is crucial in such models, as it directly balances representational power with the parametric complexity. This ambiguity is addressed by using principles from Bayesian nonparametrics to formulate flexible models that adapt their complexity to the data and can potentially encompass an infinite number of components. To learn these representations, we present two efficient variational inference techniques that scale well with data and highlight the advantages of hierarchical infinite local regression models, such as dealing with non-smooth functions, mitigating catastrophic forgetting, and enabling parameter sharing and fast predictions. Finally, we validate this approach on a set of large inverse dynamics datasets and test the learned models in real-world control scenarios. Chapter 3 addresses discrete-continuous hybrid modeling and control for stochastic dynamical systems, which implies dealing with time-series data. In this scenario, we develop an automatic system identification technique that decomposes nonlinear systems into hybrid automata and leverages the resulting structure to learn switching feedback control via hierarchical reinforcement learning. In the process, we rely on an augmented closed-loop hidden Markov model architecture that captures time correlations over long horizons and provides a principled Bayesian inference framework for learning hybrid representations and filtering the hidden discrete states to apply control accordingly. Finally, we embed this structure explicitly into a novel hybrid relative entropy policy search algorithm that optimizes a set of local polynomial feedback controllers and value functions. We validate the overall switching-system perspective by benchmarking the open-loop predictive performance against popular black-box representations. We also provide qualitative empirical results for hybrid reinforcement learning on common nonlinear control tasks. In Chapter 4, we attend to a general and fundamental problem in learning for control, namely robustness in data-driven stochastic optimization. The question of sensitivity has a strong priority, given the rising popularity of embedding statistical models into stochastic control frameworks. However, data from dynamical, especially mechanical, systems is often scarce due to a high extraction cost and limited coverage of the state-action space. The result is usually poor models with narrow validity and brittle control laws, particularly in an ill-posed over-parameterized learning example. We propose to robustify stochastic control by finding the worst-case distribution over the dynamics and optimizing a corresponding robust policy that minimizes the probability of catastrophic failures. We achieve this goal by formulating a two-stage iterative minimax optimization problem that finds the most pessimistic adversary in a trust region around a nominal model and uses it to optimize a robust optimal controller. We test this approach on a set of linear and nonlinear stochastic systems and supply empirical evidence of its practicality. Finally, we provide an outlook on how similar multi-stage distributional optimization techniques can be applied in approximate filtering of stochastic switching systems in order to tackle the issue of exponential explosion in state mixture components. In summation, the individual contributions of this thesis are a collection of interconnected principles for structured and robust learning for control. Although many challenges remain ahead, this research lays a foundation for reflecting on future structured learning questions that strive to combine optimal control and statistical machine learning perspectives for the automatic decomposition and optimization of hierarchical models

    Nonlinear Controller Design for UAVs with Time-Varying Aerodynamic Uncertainties

    Get PDF
    Unmanned Aerial Vehicles (UAVs) are here and they are here to stay. Unmanned Aviation has expanded significantly in recent years and research and development in the field of navigation and control have advanced beyond expectations. UAVs are currently being used for defense programs around the world but the range of applications is expected to grow in the near future, with civilian applications such as environmental and aerial monitoring, aerial surveillance and homeland security being some representative examples. Conventional and commercially available small-scale UAVs have limited utilization and applicability to executing specific short-duration missions because of limitations in size, payload, power supply and endurance. This fact has already marked the dawn of a new era of more powerful and versatile UAVs (e.g. morphing aircraft), able to perform a variety of missions. This dissertation presents a novel, comprehensive, step-by-step, nonlinear controller design framework for new generation, non-conventional UAVs with time-varying aerodynamic characteristics during flight. Controller design for such UAVs is a challenging task mainly due to uncertain aerodynamic parameters in the UAV mathematical model. This challenge is tackled by using and implementing μ-analysis and additive uncertainty weighting functions. The technique described herein can be generalized and applied to the class of non-conventional UAVs, seeking to address uncertainty challenges regarding the aircraft\u27s aerodynamic coefficients

    Building Indoor Air Temperature and Humidity Control via Innovative Techniques

    Get PDF
    In this thesis a novel Model Reference Adaptive Control approach, developed to tame the thermohygrometric behaviour of buildings and to guarantee the indoor comfort, is presented. The main advantages of the proposed method are: i) robustness with respect to a large class of perturbations, external disturbances, nonlinear unmodelled dynamics or parameters uncertainty; ii) ability to impose some given profiles to the thermohygrometric variables, iii) accurate regulation and fast tracking of indoor air temperature and humidity in the case of stringent requirements in special building spaces. In order to analyse the effectiveness and robustness of the proposed control strategy, several case studies have been carried out. They refer to some reference buildings with different geometry, use and construction materials (also including phase change materials integrated into the building envelope) simulated in different weather conditions. In addition, the control of multi-zone thermal systems have been also considered in the relevant and innovative case of thermal zones completely included in others (e.g., an expo indoor space of a museum building including a display/case, special indoor hospitals spaces including multiple infant-incubators, etc.). Finally, model based control solutions have been designed for the control of the thermohygrometric dynamics in order to better point out the robustness of the adaptive solution to the computation of the required sensible loads. Results confirm the ability of the developed approach to achieve the selected indoor air temperature and humidity conditions in order to guarantee indoor comfort in uncertain conditions. In order to devise model based controllers, the design of a procedure for generating low order building models from detailed ones is proposed. The mismatch between the outputs of the simplified models with respect to those of the detailed ones have been measured quantitatively via a set of performance indexes. Results confirm that reduced order models of buildings can effectively predict i) indoor air temperature, ii) energy consumptions, iii) sensible heat demands, as well as iv) comfort of occupants (e.g., predicted mean vote, predicted percentage dissatisfied, and mean radiant temperature). Hence, they can be used not only for the design of advanced model based controllers, but also for i) reducing drastically the computation time to get an insight into building energy performance, especially when a large set of simulations are required, ii) deriving mathematical models of building dynamics via reverse engineering methods, when experimental data are available and iii) simulate cluster of buildings

    Benelux meeting on systems and control, 23rd, March 17-19, 2004, Helvoirt, The Netherlands

    Get PDF
    Book of abstract

    Corticonic models of brain mechanisms underlying cognition and intelligence

    Get PDF
    The concern of this review is brain theory or more specifically, in its first part, a model of the cerebral cortex and the way it:(a) interacts with subcortical regions like the thalamus and the hippocampus to provide higher-level-brain functions that underlie cognition and intelligence, (b) handles and represents dynamical sensory patterns imposed by a constantly changing environment, (c) copes with the enormous number of such patterns encountered in a lifetime bymeans of dynamic memory that offers an immense number of stimulus-specific attractors for input patterns (stimuli) to select from, (d) selects an attractor through a process of “conjugation” of the input pattern with the dynamics of the thalamo–cortical loop, (e) distinguishes between redundant (structured)and non-redundant (random) inputs that are void of information, (f) can do categorical perception when there is access to vast associative memory laid out in the association cortex with the help of the hippocampus, and (g) makes use of “computation” at the edge of chaos and information driven annealing to achieve all this. Other features and implications of the concepts presented for the design of computational algorithms and machines with brain-like intelligence are also discussed. The material and results presented suggest, that a Parametrically Coupled Logistic Map network (PCLMN) is a minimal model of the thalamo–cortical complex and that marrying such a network to a suitable associative memory with re-entry or feedback forms a useful, albeit, abstract model of a cortical module of the brain that could facilitate building a simple artificial brain. In the second part of the review, the results of numerical simulations and drawn conclusions in the first part are linked to the most directly relevant works and views of other workers. What emerges is a picture of brain dynamics on the mesoscopic and macroscopic scales that gives a glimpse of the nature of the long sought after brain code underlying intelligence and other higher level brain functions. Physics of Life Reviews 4 (2007) 223–252 © 2007 Elsevier B.V. All rights reserved
    corecore