Decentralized Constraint Satisfaction

We show that several important resource allocation problems in wireless networks fit within the common framework of Constraint Satisfaction Problems (CSPs). Inspired by the requirements of these applications, where variables are located at distinct network devices that may not be able to communicate but may interfere, we define natural criteria that a CSP solver must possess in order to be practical. We term these algorithms decentralized CSP solvers. The best known CSP solvers were designed for centralized problems and do not meet these criteria. We introduce a stochastic decentralized CSP solver and prove that it will find a solution in almost surely finite time, should one exist, also showing it has many practically desirable properties. We benchmark the algorithm's performance on a well-studied class of CSPs, random k-SAT, illustrating that the time the algorithm takes to find a satisfying assignment is competitive with stochastic centralized solvers on problems with order a thousand variables despite its decentralized nature. We demonstrate the solver's practical utility for the problems that motivated its introduction by using it to find a non-interfering channel allocation for a network formed from data from downtown Manhattan

Decentralization of Multiagent Policies by Learning What to Communicate

Effective communication is required for teams of robots to solve sophisticated collaborative tasks. In practice it is typical for both the encoding and semantics of communication to be manually defined by an expert; this is true regardless of whether the behaviors themselves are bespoke, optimization based, or learned. We present an agent architecture and training methodology using neural networks to learn task-oriented communication semantics based on the example of a communication-unaware expert policy. A perimeter defense game illustrates the system's ability to handle dynamically changing numbers of agents and its graceful degradation in performance as communication constraints are tightened or the expert's observability assumptions are broken.Comment: 7 page

Smart Contracts for Multiagent Plan Execution in Untrusted Cyber-physical Systems

Intelligent Cyber-physical systems can be modelled as multi-agent systems with planning capability to impart adaptivity for changing contexts. In such multi-agent systems, the protocol for plan execution must result in the proper completion and ordering of actions in spite of their distributed execution. However, in untrusted scenarios, there is a possibility of agents not respecting the protocol either due to faults or due to malicious reasons thereby resulting in plan failure. In order to prevent such situations, we propose to implement the execution of agents through smart contracts. This points to a generic architecture seamlessly integrating intelligent planning-based CPS and smart-contracts.Comment: Planning, Artificial intelligence, Blockchain, Smart Contrac

Iroko: A Framework to Prototype Reinforcement Learning for Data Center Traffic Control

Recent networking research has identified that data-driven congestion control (CC) can be more efficient than traditional CC in TCP. Deep reinforcement learning (RL), in particular, has the potential to learn optimal network policies. However, RL suffers from instability and over-fitting, deficiencies which so far render it unacceptable for use in datacenter networks. In this paper, we analyze the requirements for RL to succeed in the datacenter context. We present a new emulator, Iroko, which we developed to support different network topologies, congestion control algorithms, and deployment scenarios. Iroko interfaces with the OpenAI gym toolkit, which allows for fast and fair evaluation of different RL and traditional CC algorithms under the same conditions. We present initial benchmarks on three deep RL algorithms compared to TCP New Vegas and DCTCP. Our results show that these algorithms are able to learn a CC policy which exceeds the performance of TCP New Vegas on a dumbbell and fat-tree topology. We make our emulator open-source and publicly available: https://github.com/dcgym/irokoComment: 5 figures, 1 Table, 11 pages, Accepted to http://mlforsystems.org/accepted_papers.html (ML for Systems) worksho

A Survey and Critique of Multiagent Deep Reinforcement Learning

Deep reinforcement learning (RL) has achieved outstanding results in recent years. This has led to a dramatic increase in the number of applications and methods. Recent works have explored learning beyond single-agent scenarios and have considered multiagent learning (MAL) scenarios. Initial results report successes in complex multiagent domains, although there are several challenges to be addressed. The primary goal of this article is to provide a clear overview of current multiagent deep reinforcement learning (MDRL) literature. Additionally, we complement the overview with a broader analysis: (i) we revisit previous key components, originally presented in MAL and RL, and highlight how they have been adapted to multiagent deep reinforcement learning settings. (ii) We provide general guidelines to new practitioners in the area: describing lessons learned from MDRL works, pointing to recent benchmarks, and outlining open avenues of research. (iii) We take a more critical tone raising practical challenges of MDRL (e.g., implementation and computational demands). We expect this article will help unify and motivate future research to take advantage of the abundant literature that exists (e.g., RL and MAL) in a joint effort to promote fruitful research in the multiagent community.Comment: Under review since Oct 2018. Earlier versions of this work had the title: "Is multiagent deep reinforcement learning the answer or the question? A brief survey

A System Complexity Approach to Swarm Electrification

The study investigates a bottom-up concept for microgrids. Financial analysis is performed through a business model approach to test for viability when replacing a researched energy expenditure baseline in Bangladesh. A literature review compares the approach to current trends in microgrids. A case study of Bangladesh illustrates the potential for building on the existing infrastructure base of solar home systems. Opportunities are identified to improve access to reliable energy through a microgrid approach that aims at community-driven economic and infrastructure development by building on network effects generated through the inclusion of localized economies with strong producer-consumer linkages embedded within larger systems of trade and exchange. The analysed approach involves the linking together of individual stand-alone energy systems to form a microgrid that can eventually interconnect with present legacy infrastructure consisting of national or regional grids. The approach is likened to the concept of swarm intelligence, where each individual node brings independent input to create a conglomerate of value greater than the sum of its parts

Blockchain And The Future of the Internet: A Comprehensive Review

Blockchain is challenging the status quo of the central trust infrastructure currently prevalent in the Internet towards a design principle that is underscored by decentralization, transparency, and trusted auditability. In ideal terms, blockchain advocates a decentralized, transparent, and more democratic version of the Internet. Essentially being a trusted and decentralized database, blockchain finds its applications in fields as varied as the energy sector, forestry, fisheries, mining, material recycling, air pollution monitoring, supply chain management, and their associated operations. In this paper, we present a survey of blockchain-based network applications. Our goal is to cover the evolution of blockchain-based systems that are trying to bring in a renaissance in the existing, mostly centralized, space of network applications. While re-imagining the space with blockchain, we highlight various common challenges, pitfalls, and shortcomings that can occur. Our aim is to make this work as a guiding reference manual for someone interested in shifting towards a blockchain-based solution for one's existing use case or automating one from the ground up.Comment: Under Review in IEEE COMS

The Price of Governance: A Middle Ground Solution to Coordination in Organizational Control

Achieving coordination is crucial in organizational control. This paper investigates a middle ground solution between decentralized interactions and centralized administrations for coordinating agents beyond inefficient behavior. We first propose the price of governance (PoG) to evaluate how such a middle ground solution performs in terms of effectiveness and cost. We then propose a hierarchical supervision framework to explicitly model the PoG, and define step by step how to realize the core principle of the framework and compute the optimal PoG for a control problem. Two illustrative case studies are carried out to exemplify the applications of the proposed framework and its methodology. Results show that by properly formulating and implementing each step, the hierarchical supervision framework is capable of promoting coordination among agents while bounding administrative cost to a minimum in different kinds of organizational control problems

Asynchronous Decentralized 20 Questions for Adaptive Search

This paper considers the problem of adaptively searching for an unknown target using multiple agents connected through a time-varying network topology. Agents are equipped with sensors capable of fast information processing, and we propose a decentralized collaborative algorithm for controlling their search given noisy observations. Specifically, we propose decentralized extensions of the adaptive query-based search strategy that combines elements from the 20 questions approach and social learning. Under standard assumptions on the time-varying network dynamics, we prove convergence to correct consensus on the value of the parameter as the number of iterations go to infinity. The convergence analysis takes a novel approach using martingale-based techniques combined with spectral graph theory. Our results establish that stability and consistency can be maintained even with one-way updating and randomized pairwise averaging, thus providing a scalable low complexity method with performance guarantees. We illustrate the effectiveness of our algorithm for random network topologies.Comment: 19 pages, Submitted. arXiv admin note: substantial text overlap with arXiv:1312.784

The Role of Cloud-MANET Framework in the Internet of Things (IoT)

In the next generation of computing, Mobile ad-hoc network (MANET) will play a very important role in the Internet of Things (IoT). The MANET is a kind of wireless networks that are self-organizing and auto connected in a decentralized system. Every device in MANET can be moved freely from one location to another in any direction. They can create a network with their neighbors smart devices and forward data to another device. The IoT-Cloud-MANET framework of smart devices is composed of IoT, cloud computing, and MANET. This framework can access and deliver cloud services to the MANET users through their smart devices in the IoT framework where all computations, data handling, and resource management are performed. The smart devices can move from one location to another within the range of the MANET network. Various MANETs can connect to the same cloud, they can use cloud service in a real time. For connecting the smart device of MANET to cloud needs integration with mobile apps. My main contribution in this research links a new methodology for providing secure communication on the internet of smart devices using MANET Concept in 5G. The research methodology uses the correct and efficient simulation of the desired study and can be implemented in a framework of the Internet of Things in 5G.Comment: arXiv admin note: text overlap with arXiv:1902.0974