Estimation and stability of nonlinear control systems under intermittent information with applications to multi-agent robotics

This dissertation investigates the role of intermittent information in estimation and control problems and applies the obtained results to multi-agent tasks in robotics. First, we develop a stochastic hybrid model of mobile networks able to capture a large variety of heterogeneous multi-agent problems and phenomena. This model is applied to a case study where a heterogeneous mobile sensor network cooperatively detects and tracks mobile targets based on intermittent observations. When these observations form a satisfactory target trajectory, a mobile sensor is switched to the pursuit mode and deployed to capture the target. The cost of operating the sensors is determined from the geometric properties of the network, environment and probability of target detection. The above case study is motivated by the Marco Polo game played by children in swimming pools. Second, we develop adaptive sampling of targets positions in order to minimize energy consumption, while satisfying performance guarantees such as increased probability of detection over time, and no-escape conditions. A parsimonious predictor-corrector tracking filter, that uses geometrical properties of targets\u27 tracks to estimate their positions using imperfect and intermittent measurements, is presented. It is shown that this filter requires substantially less information and processing power than the Unscented Kalman Filter and Sampling Importance Resampling Particle Filter, while providing comparable estimation performance in the presence of intermittent information. Third, we investigate stability of nonlinear control systems under intermittent information. We replace the traditional periodic paradigm, where the up-to-date information is transmitted and control laws are executed in a periodic fashion, with the event-triggered paradigm. Building on the small gain theorem, we develop input-output triggered control algorithms yielding stable closed-loop systems. In other words, based on the currently available (but outdated) measurements of the outputs and external inputs of a plant, a mechanism triggering when to obtain new measurements and update the control inputs is provided. Depending on the noise environment, the developed algorithm yields stable, asymptotically stable, and Lp-stable (with bias) closed-loop systems. Control loops are modeled as interconnections of hybrid systems for which novel results on Lp-stability are presented. Prediction of a triggering event is achieved by employing Lp-gains over a finite horizon in the small gain theorem. By resorting to convex programming, a method to compute Lp-gains over a finite horizon is devised. Next, we investigate optimal intermittent feedback for nonlinear control systems. Using the currently available measurements from a plant, we develop a methodology that outputs when to update the control law with new measurements such that a given cost function is minimized. Our cost function captures trade-offs between the performance and energy consumption of the control system. The optimization problem is formulated as a Dynamic Programming problem, and Approximate Dynamic Programming is employed to solve it. Instead of advocating a particular approximation architecture for Approximate Dynamic Programming, we formulate properties that successful approximation architectures satisfy. In addition, we consider problems with partially observable states, and propose Particle Filtering to deal with partially observable states and intermittent feedback. Finally, we investigate a decentralized output synchronization problem of heterogeneous linear systems. We develop a self-triggered output broadcasting policy for the interconnected systems. Broadcasting time instants adapt to the current communication topology. For a fixed topology, our broadcasting policy yields global exponential output synchronization, and Lp-stable output synchronization in the presence of disturbances. Employing a converse Lyapunov theorem for impulsive systems, we provide an average dwell time condition that yields disturbance-to-state stable output synchronization in case of switching topology. Our approach is applicable to directed and unbalanced communication topologies.\u2

Urban load optimization based on agent-based model representation

Tese de mestrado integrado em Engenharia da Energia e do Ambiente, apresentada à Universidade de Lisboa, através da Faculdade de Ciências, em 2018O sistema energético atravessará uma profunda transformação nos próximos anos à medida que a produção renovável distribuída, a flexibilidade no lado do consumo e as funcionalidades de SmartGrid são implementadas. Este processo, conduzido em grande parte pelas imposições causadas pelos efeitos das alterações climáticas, implica profundas transformações na produção e consumo de energia e torna a transição energética extremamente urgente. Simultaneamente, novos players, entidades e modelos de negócio têm emergido em quase todos os níveis da cadeia energética desde a produção, a transmissão, distribuição e comercialização até à gestão da rede elétrica, num movimento conduzido pelo processo de particionamento (unbundling) do sistema elétrico e pela exigência de um sistema mais descentralizado e horizontal. O efeito combinado desta nova paisagem energética torna possíveis novas funcionalidades e arquitecturas de sistema na mesma medida em que coloca enormes problemas de natureza física e matemática mas também enormes questões económicas, sociais e políticas que terão, necessariamente, de ser abordadas e resolvidas. A Gestão do Consumo é um termo abrangente que representa tanto os mecanismos de Resposta na Procura (Demand Response) ou a Gestão no Lado da Procura (Demand-Side Management) e que se impõe como um dos problemas actuais mais importantes em sistemas energéticos inteligentes caracterizados por altas penetrações renováveis e mecanismos de mercado. Para resolver estes problemas, um conjunto de métodos matemáticos e computacionais têm sido propostos nos últimos anos. Otimização distribuída e sistemas inteligentes, sistemas baseados em agentes de software e teoria de jogos encontram-se entre algumas das ferramentas usadas para otimizar o consumo de energia e determinar o agendamento e a alocação ótima de equipamentos e máquinas para consumidores residenciais, comerciais e industriais. Na sequência de trabalhos prévios disponíveis na literatura da especialidade, o presente trabalho propõe um modelo geral para abordar o problema da otimização de cargas através de arquitecturas e métodos baseados no paradigma dos Agentes. O trabalho começa por definir agentes em pontos críticos da rede elétrica e os seus processos internos de raciocínio representados por modelos de otimização matemática. Seguidamente as interações entre agentes são modeladas como um jogo de dois níveis (bi-level game) entre uma entidade gestora da rede e consumidores de energia tipificados de forma a coordenar o carregamento de diversos equipamentos, incluindo veículos elétricos, e determinar uma solução admissível para o sistema global. A funcionalidade geral do modelo proposto é demonstrada através da sua implementação em software proprietário e recorrendo a um conjunto de dados específicos. Está, então, pronto para ser complementado e refinado no futuro de forma a ser aplicado em problemas do mundo real, de grandes dimensões, mas também novas implementações em software open source de forma a ficar acessível a novos utilizadores.The energy system is expected to go through a phase change in coming years as distributed generation, demand flexibility and SmartGrid features gets implemented. The main driver for this process, climate change, imposes constraints on energy production and consumption making energy transition extremely urgent. Simultaneously, new players, entities and business models have emerged at almost all levels of the energy chain from production, transmission, distribution and commercialization down to power grid management driven by the unbundling process and the call for a more decentralized and horizontal energy system. The combined effect of this new energy landscape makes new system’s architectures and functionalities desirable and possible, but poses huge physical, mathematical, engineering, economic and political questions and problems that need to be tackled. Load Management is one broad term depicting Demand-Side Management and Demand Response mechanisms and is one of the pressing problems on smart energy systems. To solve them, a plethora of computational and mathematical methods have been proposed in recent years. Distributed optimization and intelligence, software agents, agent-based systems and game theory are among the tools used to optimize load consumption and determine optimal device scheduling for residential, commercial and industrial power consumers Following previous work found in literature, the present work proposes a general framework to treat the load optimization problem using agent-based architectures and models. We start by defining agents at critical points within the power grid as well as their internal reasoning process depicted by mathematical optimization models. We then proceed to model the cooperative interactions between agents as a Bi-level game between a grid entity and typified power consumers in order to coordinate the charging of several appliances and electrical vehicles and determine a feasible solution for the global system. We show the general functionality of the framework by implementing it in software and applying it to specific datasets. The framework is suitable for further refinement and development when applied to real world problems

Reinforcement Learning and Planning for Preference Balancing Tasks

Robots are often highly non-linear dynamical systems with many degrees of freedom, making solving motion problems computationally challenging. One solution has been reinforcement learning (RL), which learns through experimentation to automatically perform the near-optimal motions that complete a task. However, high-dimensional problems and task formulation often prove challenging for RL. We address these problems with PrEference Appraisal Reinforcement Learning (PEARL), which solves Preference Balancing Tasks (PBTs). PBTs define a problem as a set of preferences that the system must balance to achieve a goal. The method is appropriate for acceleration-controlled systems with continuous state-space and either discrete or continuous action spaces with unknown system dynamics. We show that PEARL learns a sub-optimal policy on a subset of states and actions, and transfers the policy to the expanded domain to produce a more refined plan on a class of robotic problems. We establish convergence to task goal conditions, and even when preconditions are not verifiable, show that this is a valuable method to use before other more expensive approaches. Evaluation is done on several robotic problems, such as Aerial Cargo Delivery, Multi-Agent Pursuit, Rendezvous, and Inverted Flying Pendulum both in simulation and experimentally. Additionally, PEARL is leveraged outside of robotics as an array sorting agent. The results demonstrate high accuracy and fast learning times on a large set of practical applications

Action in Mind: Neural Models for Action and Intention Perception

To notice, recognize, and ultimately perceive the others’ actions and to discern the intention behind those observed actions is an essential skill for social communications and improves markedly the chances of survival. Encountering dangerous behavior, for instance, from a person or an animal requires an immediate and suitable reaction. In addition, as social creatures, we need to perceive, interpret, and judge correctly the other individual’s actions as a fundamental skill for our social life. In other words, our survival and success in adaptive social behavior and nonverbal communication depends heavily on our ability to thrive in complex social situations. However, it has been shown that humans spontaneously can decode animacy and social interactions even from strongly impoverished stimuli and this is a fundamental part of human experience that develops early in infancy and is shared with other primates. In addition, it is well established that perceptual and motor representations of actions are tightly coupled and both share common mechanisms. This coupling between action perception and action execution plays a critical role in action understanding as postulated in various studies and they are potentially important for our social cognition. This interaction likely is mediated by action-selective neurons in the superior temporal sulcus (STS), premotor and parietal cortex. STS and TPJ have been identified also as coarse neural substrate for the processing of social interactions stimuli. Despite this localization, the underlying exact neural circuits of this processing remain unclear. The aim of this thesis is to understand the neural mechanisms behind the action perception coupling and to investigate further how human brain perceive different classes of social interactions. To achieve this goal, first we introduce a neural model that provides a unifying account for multiple experiments on the interaction between action execution and action perception. The model reproduces correctly the interactions between action observation and execution in several experiments and provides a link towards electrophysiological detailed models of relevant circuits. This model might thus provide a starting point for the detailed quantitative investigation how motor plans interact with perceptual action representations at the level of single-cell mechanisms. Second we present a simple neural model that reproduces some of the key observations in psychophysical experiments about the perception of animacy and social interactions from stimuli. Even in its simple form the model proves that animacy and social interaction judgments partly might be derived by very elementary operations in hierarchical neural vision systems, without a need of sophisticated or accurate probabilistic inference

Multi-agent control and operation of electric power distribution systems

This dissertation presents operation and control strategies for electric power distribution systems containing distributed generators. First, models of microturbines and fuel cells are developed. These dynamic models are incorporated in a power system analysis package. Second, operation of these generators in a distribution system is addressed and load following schemes are designed. The penetration of distributed generators (DGs) into the power distribution system stability becomes an issue and so the control of those DGs becomes necessary. A decentralized control structure based on conventional controllers is designed for distributed generators using a new developed optimization technique called Guided Particle Swarm Optimization. However, the limitations of the conventional controllers do not satisfy the stability requirement of a power distribution system that has a high DG penetration level, which imposes the necessity of developing a new control structure able to overcome the limitations imposed by the fixed structure conventional controllers and limit the penetration of DGs in the overall transient stability of the distribution system. Third, a novel multi-agent based control architecture is proposed for transient stability enhancement for distribution systems with microturbines. The proposed control architecture is hierarchical with one supervisory global control agent and a distributed number of local control agents in the lower layer. Specifically, a central control center supervises and optimizes the overall process, while each microturbine is equipped with its own local control agent.;The control of naval shipboard electric power system is another application of distributed control with multi-agent based structure. In this proposal, the focus is to introduce the concept of multi-agent based control architecture to improve the stability of the shipboard power system during faulty conditions. The effectiveness of the proposed methods is illustrated using a 37-bus IEEE benchmark system and an all-electric naval ship

Risk-Averse Model Predictive Operation Control of Islanded Microgrids

In this paper we present a risk-averse model predictive control (MPC) scheme for the operation of islanded microgrids with very high share of renewable energy sources. The proposed scheme mitigates the effect of errors in the determination of the probability distribution of renewable infeed and load. This allows to use less complex and less accurate forecasting methods and to formulate low-dimensional scenario-based optimisation problems which are suitable for control applications. Additionally, the designer may trade performance for safety by interpolating between the conventional stochastic and worst-case MPC formulations. The presented risk-averse MPC problem is formulated as a mixed-integer quadratically-constrained quadratic problem and its favourable characteristics are demonstrated in a case study. This includes a sensitivity analysis that illustrates the robustness to load and renewable power prediction errors

Green transition, investment horizon, and dynamic portfolio decisions

This paper analyzes the implications of investors’ short-term oriented asset holding and portfolio decisions (or short-termism), and its consequences on green investments. We adopt a dynamic portfolio model, which contrary to conventional static mean-variance models, allows us to study optimal portfolios for different decision horizons. Our baseline model contains two assets, one asset with fluctuating returns and another asset with a constant risk-free return. The asset with fluctuating returns can arise from fossil-fuel based sectors or from clean energy related sectors. We consider different drivers of short-termism: the discount rate, the nature of discounting (exponential vs. hyperbolic), and the decision horizon of investors itself. We study first the implications of these determinants of short-termism on the portfolio wealth dynamics of the baseline model. We find that portfolio wealth declines faster with a higher discount rate, with hyperbolic discounting, and with shorter decision horizon. We extend our model to include a portfolio of two assets with fluctuating returns. For both model variants, we explore the cases where innovation efforts are spent on fossil fuel or clean energy sources. Detailing dynamic portfolio decisions in such a way may allow us for better pathways to empirical tests and may provide guidance to some online financial decision making

Study of Cooperative Control System for Multiple Mobile Robots Using Particle Swarm Optimization

The idea of using multiple mobile robots for tracking targets in an unknown environment can be realized with Particle Swarm Optimization proposed by Kennedy and Eberhart in 1995. The actual implementation of an efficient algorithm like Particle Swarm Optimization (PSO) is required when robots need to avoid the randomly placed obstacles in unknown environment and reach the target point. However, ordinary methods of obstacle avoidance have not proven good results in route planning. PSO is a self-adaptive population-based method in which behavior of the swarm is iteratively generated from the combination of social and cognitive behaviors and is an effective technique for collective robotic search problem. When PSO is used for exploration, this algorithm enables robots to travel on trajectories that lead to total swarm convergence on some target