Optimal Control of Logically Constrained Partially Observable and Multi-Agent Markov Decision Processes

Autonomous systems often have logical constraints arising, for example, from safety, operational, or regulatory requirements. Such constraints can be expressed using temporal logic specifications. The system state is often partially observable. Moreover, it could encompass a team of multiple agents with a common objective but disparate information structures and constraints. In this paper, we first introduce an optimal control theory for partially observable Markov decision processes (POMDPs) with finite linear temporal logic constraints. We provide a structured methodology for synthesizing policies that maximize a cumulative reward while ensuring that the probability of satisfying a temporal logic constraint is sufficiently high. Our approach comes with guarantees on approximate reward optimality and constraint satisfaction. We then build on this approach to design an optimal control framework for logically constrained multi-agent settings with information asymmetry. We illustrate the effectiveness of our approach by implementing it on several case studies.Comment: arXiv admin note: substantial text overlap with arXiv:2203.0903

Distributed Optimization

We demonstrate a new framework for analyzing and controlling distributed systems, by solving constrained optimization problems with an algorithm based on that framework. The framework is ar. information-theoretic extension of conventional full-rationality game theory to allow bounded rational agents. The associated optimization algorithm is a game in which agents control the variables of the optimization problem. They do this by jointly minimizing a Lagrangian of (the probability distribution of) their joint state. The updating of the Lagrange parameters in that Lagrangian is a form of automated annealing, one that focuses the multi-agent system on the optimal pure strategy. We present computer experiments for the k-sat constraint satisfaction problem and for unconstrained minimization of NK functions

On tractability and congruence distributivity

Constraint languages that arise from finite algebras have recently been the object of study, especially in connection with the Dichotomy Conjecture of Feder and Vardi. An important class of algebras are those that generate congruence distributive varieties and included among this class are lattices, and more generally, those algebras that have near-unanimity term operations. An algebra will generate a congruence distributive variety if and only if it has a sequence of ternary term operations, called Jonsson terms, that satisfy certain equations. We prove that constraint languages consisting of relations that are invariant under a short sequence of Jonsson terms are tractable by showing that such languages have bounded relational width

KEMNAD: A Knowledge Engineering Methodology for Negotiating Agent Development

Automated negotiation is widely applied in various domains. However, the development of such systems is a complex knowledge and software engineering task. So, a methodology there will be helpful. Unfortunately, none of existing methodologies can offer sufficient, detailed support for such system development. To remove this limitation, this paper develops a new methodology made up of: (1) a generic framework (architectural pattern) for the main task, and (2) a library of modular and reusable design pattern (templates) of subtasks. Thus, it is much easier to build a negotiating agent by assembling these standardised components rather than reinventing the wheel each time. Moreover, since these patterns are identified from a wide variety of existing negotiating agents(especially high impact ones), they can also improve the quality of the final systems developed. In addition, our methodology reveals what types of domain knowledge need to be input into the negotiating agents. This in turn provides a basis for developing techniques to acquire the domain knowledge from human users. This is important because negotiation agents act faithfully on the behalf of their human users and thus the relevant domain knowledge must be acquired from the human users. Finally, our methodology is validated with one high impact system

Learning an Approximate Model Predictive Controller with Guarantees

A supervised learning framework is proposed to approximate a model predictive controller (MPC) with reduced computational complexity and guarantees on stability and constraint satisfaction. The framework can be used for a wide class of nonlinear systems. Any standard supervised learning technique (e.g. neural networks) can be employed to approximate the MPC from samples. In order to obtain closed-loop guarantees for the learned MPC, a robust MPC design is combined with statistical learning bounds. The MPC design ensures robustness to inaccurate inputs within given bounds, and Hoeffding's Inequality is used to validate that the learned MPC satisfies these bounds with high confidence. The result is a closed-loop statistical guarantee on stability and constraint satisfaction for the learned MPC. The proposed learning-based MPC framework is illustrated on a nonlinear benchmark problem, for which we learn a neural network controller with guarantees.Comment: 6 pages, 3 figures, to appear in IEEE Control Systems Letter

Two-Stage Subspace Constrained Precoding in Massive MIMO Cellular Systems

We propose a subspace constrained precoding scheme that exploits the spatial channel correlation structure in massive MIMO cellular systems to fully unleash the tremendous gain provided by massive antenna array with reduced channel state information (CSI) signaling overhead. The MIMO precoder at each base station (BS) is partitioned into an inner precoder and a Transmit (Tx) subspace control matrix. The inner precoder is adaptive to the local CSI at each BS for spatial multiplexing gain. The Tx subspace control is adaptive to the channel statistics for inter-cell interference mitigation and Quality of Service (QoS) optimization. Specifically, the Tx subspace control is formulated as a QoS optimization problem which involves an SINR chance constraint where the probability of each user's SINR not satisfying a service requirement must not exceed a given outage probability. Such chance constraint cannot be handled by the existing methods due to the two stage precoding structure. To tackle this, we propose a bi-convex approximation approach, which consists of three key ingredients: random matrix theory, chance constrained optimization and semidefinite relaxation. Then we propose an efficient algorithm to find the optimal solution of the resulting bi-convex approximation problem. Simulations show that the proposed design has significant gain over various baselines.Comment: 13 pages, accepted by IEEE Transactions on Wireless Communication