A new model for solution of complex distributed constrained problems

In this paper we describe an original computational model for solving different types of Distributed Constraint Satisfaction Problems (DCSP). The proposed model is called Controller-Agents for Constraints Solving (CACS). This model is intended to be used which is an emerged field from the integration between two paradigms of different nature: Multi-Agent Systems (MAS) and the Constraint Satisfaction Problem paradigm (CSP) where all constraints are treated in central manner as a black-box. This model allows grouping constraints to form a subset that will be treated together as a local problem inside the controller. Using this model allows also handling non-binary constraints easily and directly so that no translating of constraints into binary ones is needed. This paper presents the implementation outlines of a prototype of DCSP solver, its usage methodology and overview of the CACS application for timetabling problems

DISCO: Distributed Multi-domain SDN Controllers

Modern multi-domain networks now span over datacenter networks, enterprise networks, customer sites and mobile entities. Such networks are critical and, thus, must be resilient, scalable and easily extensible. The emergence of Software-Defined Networking (SDN) protocols, which enables to decouple the data plane from the control plane and dynamically program the network, opens up new ways to architect such networks. In this paper, we propose DISCO, an open and extensible DIstributed SDN COntrol plane able to cope with the distributed and heterogeneous nature of modern overlay networks and wide area networks. DISCO controllers manage their own network domain and communicate with each others to provide end-to-end network services. This communication is based on a unique lightweight and highly manageable control channel used by agents to self-adaptively share aggregated network-wide information. We implemented DISCO on top of the Floodlight OpenFlow controller and the AMQP protocol. We demonstrated how DISCO's control plane dynamically adapts to heterogeneous network topologies while being resilient enough to survive to disruptions and attacks and providing classic functionalities such as end-point migration and network-wide traffic engineering. The experimentation results we present are organized around three use cases: inter-domain topology disruption, end-to-end priority service request and virtual machine migration

Sensorimotor coordination and metastability in a situated HKB model

Oscillatory phenomena are ubiquitous in nature and have become particularly relevant for the study of brain and behaviour. One of the simplest, yet explanatorily powerful, models of oscillatory Coordination Dynamics is the Haken–Kelso–Bunz (HKB) model. The metastable regime described by the HKB equation has been hypothesised to be the signature of brain oscillatory dynamics underlying sensorimotor coordination. Despite evidence supporting such a hypothesis, to our knowledge, there are still very few models (if any) where the HKB equation generates spatially situated behaviour and, at the same time, has its dynamics modulated by the behaviour it generates (by means of the sensory feedback resulting from body movement). This work presents a computational model where the HKB equation controls an agent performing a simple gradient climbing task and shows (i) how different metastable dynamical patterns in the HKB equation are generated and sustained by the continuous interaction between the agent and its environment; and (ii) how the emergence of functional metastable patterns in the HKB equation – i.e. patterns that generate gradient climbing behaviour – depends not only on the structure of the agent's sensory input but also on the coordinated coupling of the agent's motor–sensory dynamics. This work contributes to Kelso's theoretical framework and also to the understanding of neural oscillations and sensorimotor coordination

QDQD-Learning: A Collaborative Distributed Strategy for Multi-Agent Reinforcement Learning Through Consensus + Innovations

The paper considers a class of multi-agent Markov decision processes (MDPs), in which the network agents respond differently (as manifested by the instantaneous one-stage random costs) to a global controlled state and the control actions of a remote controller. The paper investigates a distributed reinforcement learning setup with no prior information on the global state transition and local agent cost statistics. Specifically, with the agents' objective consisting of minimizing a network-averaged infinite horizon discounted cost, the paper proposes a distributed version of $Q$-learning, $\mathcal{QD}$-learning, in which the network agents collaborate by means of local processing and mutual information exchange over a sparse (possibly stochastic) communication network to achieve the network goal. Under the assumption that each agent is only aware of its local online cost data and the inter-agent communication network is \emph{weakly} connected, the proposed distributed scheme is almost surely (a.s.) shown to yield asymptotically the desired value function and the optimal stationary control policy at each network agent. The analytical techniques developed in the paper to address the mixed time-scale stochastic dynamics of the \emph{consensus + innovations} form, which arise as a result of the proposed interactive distributed scheme, are of independent interest.Comment: Submitted to the IEEE Transactions on Signal Processing, 33 page

FAST : a fault detection and identification software tool

The aim of this work is to improve the reliability and safety of complex critical control systems by contributing to the systematic application of fault diagnosis. In order to ease the utilization of fault detection and isolation (FDI) tools in the industry, a systematic approach is required to allow the process engineers to analyze a system from this perspective. In this way, it should be possible to analyze this system to find if it provides the required fault diagnosis and redundancy according to the process criticality. In addition, it should be possible to evaluate what-if scenarios by slightly modifying the process (f.i. adding sensors or changing their placement) and evaluating the impact in terms of the fault diagnosis and redundancy possibilities. Hence, this work proposes an approach to analyze a process from the FDI perspective and for this purpose provides the tool FAST which covers from the analysis and design phase until the final FDI supervisor implementation in a real process. To synthesize the process information, a very simple format has been defined based on XML. This format provides the needed information to systematically perform the Structural Analysis of that process. Any process can be analyzed, the only restriction is that the models of the process components need to be available in the FAST tool. The processes are described in FAST in terms of process variables, components and relations and the tool performs the structural analysis of the process obtaining: (i) the structural matrix, (ii) the perfect matching, (iii) the analytical redundancy relations (if any) and (iv) the fault signature matrix. To aid in the analysis process, FAST can operate stand alone in simulation mode allowing the process engineer to evaluate the faults, its detectability and implement changes in the process components and topology to improve the diagnosis and redundancy capabilities. On the other hand, FAST can operate on-line connected to the process plant through an OPC interface. The OPC interface enables the possibility to connect to almost any process which features a SCADA system for supervisory control. When running in on-line mode, the process is monitored by a software agent known as the Supervisor Agent. FAST has also the capability of implementing distributed FDI using its multi-agent architecture. The tool is able to partition complex industrial processes into subsystems, identify which process variables need to be shared by each subsystem and instantiate a Supervision Agent for each of the partitioned subsystems. The Supervision Agents once instantiated will start diagnosing their local components and handle the requests to provide the variable values which FAST has identified as shared with other agents to support the distributed FDI process.Per tal de facilitar la utilització d'eines per la detecció i identificació de fallades (FDI) en la indústria, es requereix un enfocament sistemàtic per permetre als enginyers de processos analitzar un sistema des d'aquesta perspectiva. D'aquesta forma, hauria de ser possible analitzar aquest sistema per determinar si proporciona el diagnosi de fallades i la redundància d'acord amb la seva criticitat. A més, hauria de ser possible avaluar escenaris de casos modificant lleugerament el procés (per exemple afegint sensors o canviant la seva localització) i avaluant l'impacte en quant a les possibilitats de diagnosi de fallades i redundància. Per tant, aquest projecte proposa un enfocament per analitzar un procés des de la perspectiva FDI i per tal d'implementar-ho proporciona l'eina FAST la qual cobreix des de la fase d'anàlisi i disseny fins a la implementació final d'un supervisor FDI en un procés real. Per sintetitzar la informació del procés s'ha definit un format simple basat en XML. Aquest format proporciona la informació necessària per realitzar de forma sistemàtica l'Anàlisi Estructural del procés. Qualsevol procés pot ser analitzat, només hi ha la restricció de que els models dels components han d'estar disponibles en l'eina FAST. Els processos es descriuen en termes de variables de procés, components i relacions i l'eina realitza l'anàlisi estructural obtenint: (i) la matriu estructural, (ii) el Perfect Matching, (iii) les relacions de redundància analítica, si n'hi ha, i (iv) la matriu signatura de fallades. Per ajudar durant el procés d'anàlisi, FAST pot operar aïlladament en mode de simulació permetent a l'enginyer de procés avaluar fallades, la seva detectabilitat i implementar canvis en els components del procés i la topologia per tal de millorar les capacitats de diagnosi i redundància. Per altra banda, FAST pot operar en línia connectat al procés de la planta per mitjà d'una interfície OPC. La interfície OPC permet la possibilitat de connectar gairebé a qualsevol procés que inclogui un sistema SCADA per la seva supervisió. Quan funciona en mode en línia, el procés està monitoritzat per un agent software anomenat l'Agent Supervisor. Addicionalment, FAST té la capacitat d'implementar FDI de forma distribuïda utilitzant la seva arquitectura multi-agent. L'eina permet dividir sistemes industrials complexes en subsistemes, identificar quines variables de procés han de ser compartides per cada subsistema i generar una instància d'Agent Supervisor per cadascun dels subsistemes identificats. Els Agents Supervisor un cop activats, començaran diagnosticant els components locals i despatxant les peticions de valors per les variables que FAST ha identificat com compartides amb altres agents, per tal d'implementar el procés FDI de forma distribuïda.Postprint (published version

Distributed formation control of multiple unmanned aerial vehicles over time-varying graphs using population games

© 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.This paper presents a control technique based on distributed population dynamics under time-varying communication graphs for a multi-agent system structured in a leader-follower fashion. Here, the leader agent follows a particular trajectory and the follower agents should track it in a certain organized formation manner. The tracking of the leader can be performed in the position coordinates x; y; and z, and in the yaw angle phi. Additional features are performed with this method: each agent has only partial knowledge of the position of other agents and not necessarily all agents should communicate to the leader. Moreover, it is possible to integrate a new agent into the formation (or for an agent to leave the formation task) in a dynamical manner. In addition, the formation configuration can be changed along the time, and the distributed population-games-based controller achieves the new organization goal accommodating conveniently the information-sharing graph in function of the communication range capabilities of each UAV. Finally, several simulations are presented to illustrate different scenarios, e.g., formation with time-varying communication network, and time-varying formationPeer ReviewedPostprint (author's final draft

Consensus-based control for a network of diffusion PDEs with boundary local interaction

In this paper the problem of driving the state of a network of identical agents, modeled by boundary-controlled heat equations, towards a common steady-state profile is addressed. Decentralized consensus protocols are proposed to address two distinct problems. The first problem is that of steering the states of all agents towards the same constant steady-state profile which corresponds to the spatial average of the agents initial condition. A linear local interaction rule addressing this requirement is given. The second problem deals with the case where the controlled boundaries of the agents dynamics are corrupted by additive persistent disturbances. To achieve synchronization between agents, while completely rejecting the effect of the boundary disturbances, a nonlinear sliding-mode based consensus protocol is proposed. Performance of the proposed local interaction rules are analyzed by applying a Lyapunov-based approach. Simulation results are presented to support the effectiveness of the proposed algorithms