Distributed Management of Resources in a Smart City using Multi-Agent Systems (MAS)

This document describes study of distributed management of resources in the context of Smart City with support of multi-agent systems. The investigated points include theoretical concepts of Smart City and application of multi-agent systems, decentralized and centralized designs for agent-based solutions and aspects of interactions between different self-interested agents. The document explores design of an agent-based solutions for the set of proposed problems related to Smart City environment with the emphasis on sharing common good, models of agent interactions within the modeled environments and possibilities of multi-agent approaches in terms of collective problem-solving, adaptability and learning proficiency

A Decoupling Principle for Simultaneous Localization and Planning Under Uncertainty in Multi-Agent Dynamic Environments

Simultaneous localization and planning for nonlinear stochastic systems under process and measurement uncertainties is a challenging problem. In its most general form, it is formulated as a stochastic optimal control problem in the space of feedback policies. The Hamilton-Jacobi-Bellman equation provides the theoretical solution of the optimal problem; but, as is typical of almost all nonlinear stochastic systems, optimally solving the problem is intractable. Moreover, even if an optimal solution was obtained, it would require centralized control, while multi-agent mobile robotic systems under dynamic environments require decentralized solutions. In this study, we aim for a theoretically sound solution for various modes of this problem, including the single-agent and multi-agent variations with perfect and imperfect state information, where the underlying state, control and observation spaces are continuous with discrete-time models. We introduce a decoupling principle for planning and control of multi-agent nonlinear stochastic systems based on a small noise asymptotics. Through this decoupling principle, under small noise, the design of the real-time feedback law can be decoupled from the off-line design of the nominal trajectory of the system. Further, for a multi-agent problem, the design of the feedback laws for different agents can be decoupled from each other, reducing the centralized problem to a decentralized problem requiring no communication during execution. The resulting solution is quantifiably near-optimal. We establish this result for all the above-mentioned variations, which results in the following variants: Trajectory-optimized Linear Quadratic Regulator (T-LQR), Multi-agent T-LQR (MT-LQR), Trajectory-optimized Linear Quadratic Gaussian (T-LQG), and Multi-agent T-LQG (MT-LQG). The decoupling principle provides the conditions under which a decentralized linear Gaussian system with a quadratic approximation of the cost, obtained by linearization around an optimally designed nominal trajectory can be utilized to control the nonlinear system. The resulting decentralized feedback solution at runtime, being decoupled with respect to the mobile agents, requires no communication between the agents during the execution phase. Moreover, the complexity of the solution vis-a-vis the computation of the nominal trajectory as well as the closed-loop gains is tractable with low polynomial orders of computation. Experimental implementation of the solution shows that the results hold for moderate levels of noise with high probability. Further optimizing the performance of this approach we show how to design a special cost function for the problem with imperfect state measurement that takes advantage of the fact that the estimation covariance of a linear Gaussian system is deterministic and not dependent on the observations. This design, which corresponds in our overall design to “belief space planning”, incorporates the consequently deterministic cost of the stochastic feedback system into the deterministic design of the nominal trajectory to obtain an optimal nominal trajectory with the best estimation performance. Then, it utilizes the T-LQG approach to design an optimal feedback law to track the designed nominal trajectory. This iterative approach can be used to further tune both the open loop as well as the decentralized feedback gain portions of the overall design. We also provide the multi-agent variant of this approach based on the MT-LQG method. Based on the near-optimality guarantees of the decoupling principle and the TLQG approach, we analyze the performance and correctness of a well-known heuristic in robotic path planning. We show that optimizing measures of the observability Gramian as a surrogate for estimation performance may provide irrelevant or misleading trajectories for planning under observation uncertainty. We then consider systems with non-Gaussian perturbations. An alternative heuristic method is proposed that aims for fast planning in belief space under non- Gaussian uncertainty. We provide a special design approach based on particle filters that results in a convex planning problem implemented via a model predictive control strategy in convex environments, and a locally convex problem in non-convex environments. The environment here refers to the complement of the region in Euclidean space that contains the obstacles or “no fly zones”. For non-convex dynamic environments, where the no-go regions change dynamically with time, we design a special form of an obstacle penalty function that incorporates non-convex time-varying constraints into the cost function, so that the decoupling principle still applies to these problems. However, similar to any constrained problem, the quality of the optimal nominal trajectory is dependent on the quality of the solution obtainable for the nonlinear optimization problem. We simulate our algorithms for each of the problems on various challenging situations, including for several nonlinear robotic models and common measurement models. In particular, we consider 2D and 3D dynamic environments for heterogeneous holonomic and non-holonomic robots, and range and bearing sensing models. Future research can potentially extend the results to more general situations including continuous-time models

A Decoupling Principle for Simultaneous Localization and Planning Under Uncertainty in Multi-Agent Dynamic Environments

Simultaneous localization and planning for nonlinear stochastic systems under process and measurement uncertainties is a challenging problem. In its most general form, it is formulated as a stochastic optimal control problem in the space of feedback policies. The Hamilton-Jacobi-Bellman equation provides the theoretical solution of the optimal problem; but, as is typical of almost all nonlinear stochastic systems, optimally solving the problem is intractable. Moreover, even if an optimal solution was obtained, it would require centralized control, while multi-agent mobile robotic systems under dynamic environments require decentralized solutions. In this study, we aim for a theoretically sound solution for various modes of this problem, including the single-agent and multi-agent variations with perfect and imperfect state information, where the underlying state, control and observation spaces are continuous with discrete-time models. We introduce a decoupling principle for planning and control of multi-agent nonlinear stochastic systems based on a small noise asymptotics. Through this decoupling principle, under small noise, the design of the real-time feedback law can be decoupled from the off-line design of the nominal trajectory of the system. Further, for a multi-agent problem, the design of the feedback laws for different agents can be decoupled from each other, reducing the centralized problem to a decentralized problem requiring no communication during execution. The resulting solution is quantifiably near-optimal. We establish this result for all the above-mentioned variations, which results in the following variants: Trajectory-optimized Linear Quadratic Regulator (T-LQR), Multi-agent T-LQR (MT-LQR), Trajectory-optimized Linear Quadratic Gaussian (T-LQG), and Multi-agent T-LQG (MT-LQG). The decoupling principle provides the conditions under which a decentralized linear Gaussian system with a quadratic approximation of the cost, obtained by linearization around an optimally designed nominal trajectory can be utilized to control the nonlinear system. The resulting decentralized feedback solution at runtime, being decoupled with respect to the mobile agents, requires no communication between the agents during the execution phase. Moreover, the complexity of the solution vis-a-vis the computation of the nominal trajectory as well as the closed-loop gains is tractable with low polynomial orders of computation. Experimental implementation of the solution shows that the results hold for moderate levels of noise with high probability. Further optimizing the performance of this approach we show how to design a special cost function for the problem with imperfect state measurement that takes advantage of the fact that the estimation covariance of a linear Gaussian system is deterministic and not dependent on the observations. This design, which corresponds in our overall design to “belief space planning”, incorporates the consequently deterministic cost of the stochastic feedback system into the deterministic design of the nominal trajectory to obtain an optimal nominal trajectory with the best estimation performance. Then, it utilizes the T-LQG approach to design an optimal feedback law to track the designed nominal trajectory. This iterative approach can be used to further tune both the open loop as well as the decentralized feedback gain portions of the overall design. We also provide the multi-agent variant of this approach based on the MT-LQG method. Based on the near-optimality guarantees of the decoupling principle and the TLQG approach, we analyze the performance and correctness of a well-known heuristic in robotic path planning. We show that optimizing measures of the observability Gramian as a surrogate for estimation performance may provide irrelevant or misleading trajectories for planning under observation uncertainty. We then consider systems with non-Gaussian perturbations. An alternative heuristic method is proposed that aims for fast planning in belief space under non- Gaussian uncertainty. We provide a special design approach based on particle filters that results in a convex planning problem implemented via a model predictive control strategy in convex environments, and a locally convex problem in non-convex environments. The environment here refers to the complement of the region in Euclidean space that contains the obstacles or “no fly zones”. For non-convex dynamic environments, where the no-go regions change dynamically with time, we design a special form of an obstacle penalty function that incorporates non-convex time-varying constraints into the cost function, so that the decoupling principle still applies to these problems. However, similar to any constrained problem, the quality of the optimal nominal trajectory is dependent on the quality of the solution obtainable for the nonlinear optimization problem. We simulate our algorithms for each of the problems on various challenging situations, including for several nonlinear robotic models and common measurement models. In particular, we consider 2D and 3D dynamic environments for heterogeneous holonomic and non-holonomic robots, and range and bearing sensing models. Future research can potentially extend the results to more general situations including continuous-time models

To Trust or Not: The Effects of Monitoring Intensity on Discretionary Effort, Honesty, and Problem Solving Ability

Managerial accounting researchers and practitioners are increasingly concerned with the effects of formal organizational controls on agent behavior. This three-paper dissertation extends this line of research by experimentally examining the effects of monitoring intensity on three important work behaviors which, generally, are not directly observable by the organizational control system: discretionary effort, problem solving ability, and honesty. Together, these studies help fill a gap in the managerial accounting literature by examining the relationship between the monitoring environment and agent behavior. The principal-agent theory of the firm suggests that tighter monitoring by the principal will increase the agent’s work effort at best, and have no effect at worst. However, the psychology literature suggests that monitoring may actually reduce effort by “crowding out” an individual’s intrinsic motivation to perform unmeasured or unrewarded work related tasks. In Paper 1, I test for the crowding out effect of monitoring and find mixed results. In Paper 2, I investigate the effects of monitoring intensity on various aspects of problem solving ability and creativity. Past research suggests that strict environmental controls can have detrimental effects on creative thinking. I extend this line of literature by investigating how monitoring affects an individual’s problem solving ability. In general, I find that monitoring intensity is negatively associated with problem solving ability. In Paper 3, I investigate how monitoring intensity affects an individual’s propensity toward dishonesty using a 3x2 experimental design where the participants are given a simple task, with a monetary reward based on performance, in one of the three monitoring treatments—trust, human monitoring, or electronic monitoring—and in one of two outcome reporting regimes—self-report or verified. I find an inverted-U shape relationship between monitoring intensity and dishonesty, where dishonesty is highest under human monitoring. Organizations are increasing their use of all types of surveillance and controls, and, in general, trust is increasingly discouraged within organizations. These papers add to the managerial accounting literature by shedding light on how different monitoring environments can change human behavior. This line of research can only increase in importance as regulation increases and monitoring technology becomes more advanced, reliable, and accessible

Design thinking support: information systems versus reasoning

Numerous attempts have been made to conceive and implement appropriate information systems to support architectural designers in their creative design thinking processes. These information systems aim at providing support in very diverse ways: enabling designers to make diverse kinds of visual representations of a design, enabling them to make complex calculations and simulations which take into account numerous relevant parameters in the design context, providing them with loads of information and knowledge from all over the world, and so forth. Notwithstanding the continued efforts to develop these information systems, they still fail to provide essential support in the core creative activities of architectural designers. In order to understand why an appropriately effective support from information systems is so hard to realize, we started to look into the nature of design thinking and on how reasoning processes are at play in this design thinking. This investigation suggests that creative designing rests on a cyclic combination of abductive, deductive and inductive reasoning processes. Because traditional information systems typically target only one of these reasoning processes at a time, this could explain the limited applicability and usefulness of these systems. As research in information technology is increasingly targeting the combination of these reasoning modes, improvements may be within reach for design thinking support by information systems

Agent Assistance: From Problem Solving to Music Teaching

We report on our research on agents that act and behave in a web learning environment. This research is part of a general approach to agents acting and behaving in virtual environments where they are involved in providing information, performing transactions, demonstrating products and, more generally, assisting users or visitors of the web environment in doing what they want or have been asked to do. While initially we hardly provided our agents with 'teaching knowledge', we now are in the process of making such knowledge explicit, especially in models that take into account that assisting and teaching takes place in a visualized and information-rich environment. Our main (embodied) tutor-agent is called Jacob; it knows about the Towers of Hanoi, a well-known problem that is offered to CS students to learn about recursion. Other agents we are working on assist a visitor in navigating in a virtual world or help the visitor in getting information. We are now designing a music teacher - using knowledge of software engineering and how to design multi-modal interactions, from previous projects

An Agent-based approach to modelling integrated product teams undertaking a design activity.

The interactions between individual designers, within integrated product teams, and the nature of design tasks, all have a significant impact upon how well a design task can be performed, and hence the quality of the resultant product and the time in which it can be delivered. In this paper we describe an ongoing research project which aims to model integrated product teams through the use of multi-agent systems. We first describe the background and rationale for our work, and then present our initial computational model and results from the simulation of an integrated product team. The paper concludes with a discussion of how the model will evolve to improve the accuracy of the simulation