Learning effective state-feedback controllers through efficient multilevel importance samplers

Monte Carlo sampling can be used to estimate the solution of path integral control problems, which are a restricted class of nonlinear control problems with arbitrary dynamics and state cost, but with a linear dependence of the control on the dynamics and quadratic control cost. Although importance sampling is used to improve numerical computations, the effective sample size may still be low or many samples could be required. In this work, we propose a method to learn effective state-feedback controllers for nonlinear stochastic control problems based on multilevel importance samplers. In particular, we focus on the question of how to compute effective importance samplers considering a multigrid scenario. We test our algorithm in finite horizon control problems based on Lorenz-96 model with chaotic and non-chaotic behaviour, showing, in all cases, that our multigrid implementation reduces the computational time and improves the effective sample size.Fil: Menchón, Silvia Adriana. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Instituto de Física Enrique Gaviola. Universidad Nacional de Córdoba. Instituto de Física Enrique Gaviola; Argentina. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina. Radboud Universiteit Nijmegen. Donders Instituto Brain Cognition and Behavior. SNN Machine Learning Group; Países BajosFil: Kappen, Hilbert Johan. Radboud Universiteit Nijmegen. Donders Instituto Brain Cognition and Behavior. SNN Machine Learning Group; Países Bajo

A Multilevel Approach for Stochastic Nonlinear Optimal Control

We consider a class of finite time horizon nonlinear stochastic optimal control problem, where the control acts additively on the dynamics and the control cost is quadratic. This framework is flexible and has found applications in many domains. Although the optimal control admits a path integral representation for this class of control problems, efficient computation of the associated path integrals remains a challenging Monte Carlo task. The focus of this article is to propose a new Monte Carlo approach that significantly improves upon existing methodology. Our proposed methodology first tackles the issue of exponential growth in variance with the time horizon by casting optimal control estimation as a smoothing problem for a state space model associated with the control problem, and applying smoothing algorithms based on particle Markov chain Monte Carlo. To further reduce computational cost, we then develop a multilevel Monte Carlo method which allows us to obtain an estimator of the optimal control with $\mathcal{O}(\epsilon^2)$ mean squared error with a computational cost of $\mathcal{O}(\epsilon^{-2}\log(\epsilon)^2)$. In contrast, a computational cost of $\mathcal{O}(\epsilon^{-3})$ is required for existing methodology to achieve the same mean squared error. Our approach is illustrated on two numerical examples, which validate our theory

Control analysis and design of medium voltage converter with multirate techniques

This work aims to unify the current knowledge about multirate controllers with design techniques for grid-tied converters, in this occasion, connected to Medium Voltage AC grid. Therefore, the multirate contributions, that have been given so far, are studied, as well as everything related to modulation techniques for power converters. The temporal implications of the DSPWM actuator will be correlated to multirate analysis, in addition to possible alternatives for applications with a lower sampling frequency than modulation one. Finalizing with explanations and result demonstrations of controllers working between two frequencies or rates, by means of the available power converter in laboratory.Este trabajo pretende unir el conocimiento actual sobre controladores multitasa o multifrecuencia (multirate) con técnicas de diseño para convertidores conectados a la red, en este caso concreto, a la red alterna (AC) de Media Tensión. Por tanto, se estudian las contribuciones multirate realizadas hasta la fecha, así como todo lo relacionado con la modulación de la señal de control para los convertidores. Las implicaciones temporales del actuador DSPWM se relacionarán con el análisis multitasa, así como se explicarán posibles alternativas para aplicaciones con una frecuencia de muestreo menor que la de modulación. Finalizando con la explicación y presentación de resultados de controladores trabajando entre dos frecuencias o tasas, mediante simulaciones del convertidor disponible en laboratorio.Máster Universitario en Ingeniería Industrial (M141

Computational Methods for Bayesian Inference in Complex Systems

Bayesian methods are critical for the complete understanding of complex systems. In this approach, we capture all of our uncertainty about a system’s properties using a probability distribution and update this understanding as new information becomes available. By taking the Bayesian perspective, we are able to effectively incorporate our prior knowledge about a model and to rigorously assess the plausibility of candidate models based upon observed data from the system. We can then make probabilistic predictions that incorporate uncertainties, which allows for better decision making and design. However, while these Bayesian methods are critical, they are often computationally intensive, thus necessitating the development of new approaches and algorithms. In this work, we discuss two approaches to Markov Chain Monte Carlo (MCMC). For many statistical inference and system identification problems, the development of MCMC made the Bayesian approach possible. However, as the size and complexity of inference problems has dramatically increased, improved MCMC methods are required. First, we present Second-Order Langevin MCMC (SOL-MC), a stochastic dynamical system-based MCMC algorithm that uses the damped second-order Langevin stochastic differential equation (SDE) to sample a desired posterior distribution. Since this method is based on an underlying dynamical system, we can utilize existing work in the theory for dynamical systems to develop, implement, and optimize the sampler's performance. Second, we present advances and theoretical results for Sequential Tempered MCMC (ST-MCMC) algorithms. Sequential Tempered MCMC is a family of parallelizable algorithms, based upon Transitional MCMC and Sequential Monte Carlo, that gradually transform a population of samples from the prior to the posterior through a series of intermediate distributions. Since the method is population-based, it can easily be parallelized. In this work, we derive theoretical results to help tune parameters within the algorithm. We also introduce a new sampling algorithm for ST-MCMC called the Rank-One Modified Metropolis Algorithm (ROMMA). This algorithm improves sampling efficiency for inference problems where the prior distribution constrains the posterior. In particular, this is shown to be relevant for problems in geophysics. We also discuss the application of Bayesian methods to state estimation, disturbance detection, and system identification problems in complex systems. We introduce a Bayesian perspective on learning models and properties of physical systems based upon a layered architecture that can learn quickly and flexibly. We then apply this architecture to detecting and characterizing changes in physical systems with applications to power systems and biology. In power systems, we develop a new formulation of the Extended Kalman Filter for estimating dynamic states described by differential algebraic equations. This filter is then used as the basis for sub-second fault detection and classification. In synthetic biology, we use a Bayesian approach to detect and identify unknown chemical inputs in a biosensor system implemented in a cell population. This approach uses the tools of Bayesian model selection.</p

Smart Urban Water Networks

This book presents the paper form of the Special Issue (SI) on Smart Urban Water Networks. The number and topics of the papers in the SI confirm the growing interest of operators and researchers for the new paradigm of smart networks, as part of the more general smart city. The SI showed that digital information and communication technology (ICT), with the implementation of smart meters and other digital devices, can significantly improve the modelling and the management of urban water networks, contributing to a radical transformation of the traditional paradigm of water utilities. The paper collection in this SI includes different crucial topics such as the reliability, resilience, and performance of water networks, innovative demand management, and the novel challenge of real-time control and operation, along with their implications for cyber-security. The SI collected fourteen papers that provide a wide perspective of solutions, trends, and challenges in the contest of smart urban water networks. Some solutions have already been implemented in pilot sites (i.e., for water network partitioning, cyber-security, and water demand disaggregation and forecasting), while further investigations are required for other methods, e.g., the data-driven approaches for real time control. In all cases, a new deal between academia, industry, and governments must be embraced to start the new era of smart urban water systems

NASA Tech Briefs, April 1990

Topics: New Product Ideas; NASA TU Services; Electronic Components and Circuits; Electronic Systems; Physical Sciences; Materials; Computer Programs; Mechanics; Machinery; Fabrication Technology; Mathematics and Information Sciences

Accelerated Risk Assessment And Domain Adaptation For Autonomous Vehicles

Autonomous vehicles (AVs) are already driving on public roads around the US; however, their rate of deployment far outpaces quality assurance and regulatory efforts. Consequently, even the most elementary tasks, such as automated lane keeping, have not been certified for safety, and operations are constrained to narrow domains. First, due to the limitations of worst-case analysis techniques, we hypothesize that new methods must be developed to quantify and bound the risk of AVs. Counterintuitively, the better the performance of the AV under consideration, the harder it is to accurately estimate its risk as failures become rare and difficult to sample. This thesis presents a new estimation procedure and framework that can efficiently evaluate and AV\u27s risk even in the rare event regime. We demonstrate the approach\u27s performance on a variety of AV software stacks. Second, given a framework for AV evaluation, we turn to a related question: how can AV software be efficiently adapted for new or expanded operating conditions? We hypothesize that stochastic search techniques can improve the naive trial-and-error approach commonly used today. One of the most challenging aspects of this task is that proficient driving requires making tradeoffs between performance and safety. Moreover, for novel scenarios or operational domains there may be little data that can be used to understand the behavior of other drivers. To study these challenges we create a low-cost scale platform, simulator, benchmarks, and baseline solutions. Using this testbed, we develop a new population-based self-play method for creating dynamic actors and detail both offline and online procedures for adapting AV components to these conditions. Taken as a whole, this work represents a rigorous approach to the evaluation and improvement of AV software

2022 Review of Data-Driven Plasma Science

Data-driven science and technology offer transformative tools and methods to science. This review article highlights the latest development and progress in the interdisciplinary field of data-driven plasma science (DDPS), i.e., plasma science whose progress is driven strongly by data and data analyses. Plasma is considered to be the most ubiquitous form of observable matter in the universe. Data associated with plasmas can, therefore, cover extremely large spatial and temporal scales, and often provide essential information for other scientific disciplines. Thanks to the latest technological developments, plasma experiments, observations, and computation now produce a large amount of data that can no longer be analyzed or interpreted manually. This trend now necessitates a highly sophisticated use of high-performance computers for data analyses, making artificial intelligence and machine learning vital components of DDPS. This article contains seven primary sections, in addition to the introduction and summary. Following an overview of fundamental data-driven science, five other sections cover widely studied topics of plasma science and technologies, i.e., basic plasma physics and laboratory experiments, magnetic confinement fusion, inertial confinement fusion and high-energy-density physics, space and astronomical plasmas, and plasma technologies for industrial and other applications. The final section before the summary discusses plasma-related databases that could significantly contribute to DDPS. Each primary section starts with a brief introduction to the topic, discusses the state-of-the-art developments in the use of data and/or data-scientific approaches, and presents the summary and outlook. Despite the recent impressive signs of progress, the DDPS is still in its infancy. This article attempts to offer a broad perspective on the development of this field and identify where further innovations are required

Social work with airports passengers

Social work at the airport is in to offer to passengers social services. The main methodological position is that people are under stress, which characterized by a particular set of characteristics in appearance and behavior. In such circumstances passenger attracts in his actions some attention. Only person whom he trusts can help him with the documents or psychologically

On the hunt for feedback: Vibrotactile feedback in interactive electronic music performances

The expressivity of musical performance is highly dependent on the feedback relationship between the performer and the instrument. Despite current advances in music technology, performers still struggle to retain the same expressive nuances of acoustic instruments. The capacity of performative musical expression in technologically-driven music is mitigated by the limitations of controllers and other sensor-based devices used in the performance of such music. Due to their physical properties, such devices and components are unable to provide mainly the haptic and vibrotactile experience between the instrument and the user, thus breaking the link with traditional musical performance. Such limitations are apparent to performers, suggesting often the existence of an unnatural barrier between the technology and the performer. The thesis proposes the use of vibrotactile feedback as means to enhance performer’s expressivity and creativity in technology mediated performances and situate vibrotactile feedback as part of the tradition of instrumental musical playing. Achieved through the use of small controllable electric motors, vibrotactile feedback can nourish communicative pathways between the performer and technology, a relationship that is otherwise limited or non-existing. The ability to experience an instrument's communicative response can significantly improve the performer-instrument relationship, and in turn the music performed. Through a series of case studies, compositions and performances, the dissertation suggests ways in which vibrotactile feedback may be applied to enhance the experience between the technology and the performer. As a result performers are able to develop expressive nuances and have better control of the technology during performance