Information Exchange and Conflict Resolution in Particle Swarm Optimization Variants

Single population, biologically-inspired algorithms such as Genetic Algorithm and Particle Swarm Optimization are effective tools for solving a variety of optimization problems. Like many such algorithms, however, they fall victim to the curse of dimensionality. Additionally, these algorithms often suffer from a phenomenon known as hitchhiking where improved solutions are not unequivocally better for all variables. Insofar as individuals within these populations are deemed to be competitive, one solution to both the curse of dimensionality and the problem of hitchhiking has been to introduce more cooperation. These multi-population algorithms cooperate by decomposing a problem into parts and assigning a population to each part. Factored Evolutionary Algorithms (FEA) generalize this decomposition and cooperation to any evolutionary algorithm. A key element of FEA is a global solution that provides missing information to individual populations and coordinates them. This dissertation extends FEA to the distributed case by having individual populations maintain and coordinate local solutions that maintain consensus. This Distributed FEA (DFEA) is demonstrated to perform well on a variety of problems and, sometimes, even if consensus is lost. However, DFEA fails to maintain the same semantics as FEA. To address this issue, we develop an alternative framework to the ``cooperation versus competition'' dichotomy. In this framework, information flows are modeled as a blackboard architecture. Changes in the blackboard are modeled as merge operations that require conflict resolution between existing and candidate values. Conflict resolution is handled using Pareto efficiency, which avoids hitchhiking. We apply this framework to FEA and DFEA and develop revised DFEA, which performs identically to FEA. We then apply our framework to a single population algorithm, Particle Swarm Optimization (PSO), to create Pareto Improving PSO (PI-PSO). We demonstrate that PI-PSO outperforms PSO and sometimes FEA-PSO, often with fewer individuals. Finally, we extend our information based approach by implementing parallel, distributed versions of FEA and DFEA using the Actor model. The Actor model is based on message passing, which accords well with our information-centric framework. We use validation experiments to verify that we have successfully implemented the semantics of the serial versions of FEA and DFEA

Distributed reinforcement learning for self-reconfiguring modular robots

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2007.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Includes bibliographical references (p. 101-106).In this thesis, we study distributed reinforcement learning in the context of automating the design of decentralized control for groups of cooperating, coupled robots. Specifically, we develop a framework and algorithms for automatically generating distributed controllers for self-reconfiguring modular robots using reinforcement learning. The promise of self-reconfiguring modular robots is that of robustness, adaptability and versatility. Yet most state-of-the-art distributed controllers are laboriously handcrafted and task-specific, due to the inherent complexities of distributed, local-only control. In this thesis, we propose and develop a framework for using reinforcement learning for automatic generation of such controllers. The approach is profitable because reinforcement learning methods search for good behaviors during the lifetime of the learning agent, and are therefore applicable to online adaptation as well as automatic controller design. However, we must overcome the challenges due to the fundamental partial observability inherent in a distributed system such as a self reconfiguring modular robot. We use a family of policy search methods that we adapt to our distributed problem. The outcome of a local search is always influenced by the search space dimensionality, its starting point, and the amount and quality of available exploration through experience.(cont) We undertake a systematic study of the effects that certain robot and task parameters, such as the number of modules, presence of exploration constraints, availability of nearest-neighbor communications, and partial behavioral knowledge from previous experience, have on the speed and reliability of learning through policy search in self-reconfiguring modular robots. In the process, we develop novel algorithmic variations and compact search space representations for learning in our domain, which we test experimentally on a number of tasks. This thesis is an empirical study of reinforcement learning in a simulated lattice based self-reconfiguring modular robot domain. However, our results contribute to the broader understanding of automatic generation of group control and design of distributed reinforcement learning algorithms.by Paulina Varshavskaya.Ph.D

Strategies for Scaleable Communication and Coordination in Multi-Agent (UAV) Systems

A system is considered in which agents (UAVs) must cooperatively discover interest-points (i.e., burning trees, geographical features) evolving over a grid. The objective is to locate as many interest-points as possible in the shortest possible time frame. There are two main problems: a control problem, where agents must collectively determine the optimal action, and a communication problem, where agents must share their local states and infer a common global state. Both problems become intractable when the number of agents is large. This survey/concept paper curates a broad selection of work in the literature pointing to a possible solution; a unified control/communication architecture within the framework of reinforcement learning. Two components of this architecture are locally interactive structure in the state-space, and hierarchical multi-level clustering for system-wide communication. The former mitigates the complexity of the control problem and the latter adapts to fundamental throughput constraints in wireless networks. The challenges of applying reinforcement learning to multi-agent systems are discussed. The role of clustering is explored in multi-agent communication. Research directions are suggested to unify these components

Multimedia delivery in the future internet

The term “Networked Media” implies that all kinds of media including text, image, 3D graphics, audio and video are produced, distributed, shared, managed and consumed on-line through various networks, like the Internet, Fiber, WiFi, WiMAX, GPRS, 3G and so on, in a convergent manner [1]. This white paper is the contribution of the Media Delivery Platform (MDP) cluster and aims to cover the Networked challenges of the Networked Media in the transition to the Future of the Internet. Internet has evolved and changed the way we work and live. End users of the Internet have been confronted with a bewildering range of media, services and applications and of technological innovations concerning media formats, wireless networks, terminal types and capabilities. And there is little evidence that the pace of this innovation is slowing. Today, over one billion of users access the Internet on regular basis, more than 100 million users have downloaded at least one (multi)media file and over 47 millions of them do so regularly, searching in more than 160 Exabytes1 of content. In the near future these numbers are expected to exponentially rise. It is expected that the Internet content will be increased by at least a factor of 6, rising to more than 990 Exabytes before 2012, fuelled mainly by the users themselves. Moreover, it is envisaged that in a near- to mid-term future, the Internet will provide the means to share and distribute (new) multimedia content and services with superior quality and striking flexibility, in a trusted and personalized way, improving citizens’ quality of life, working conditions, edutainment and safety. In this evolving environment, new transport protocols, new multimedia encoding schemes, cross-layer inthe network adaptation, machine-to-machine communication (including RFIDs), rich 3D content as well as community networks and the use of peer-to-peer (P2P) overlays are expected to generate new models of interaction and cooperation, and be able to support enhanced perceived quality-of-experience (PQoE) and innovative applications “on the move”, like virtual collaboration environments, personalised services/ media, virtual sport groups, on-line gaming, edutainment. In this context, the interaction with content combined with interactive/multimedia search capabilities across distributed repositories, opportunistic P2P networks and the dynamic adaptation to the characteristics of diverse mobile terminals are expected to contribute towards such a vision. Based on work that has taken place in a number of EC co-funded projects, in Framework Program 6 (FP6) and Framework Program 7 (FP7), a group of experts and technology visionaries have voluntarily contributed in this white paper aiming to describe the status, the state-of-the art, the challenges and the way ahead in the area of Content Aware media delivery platforms

Protecting the infrastructure: 3rd Australian information warfare & security conference 2002

The conference is hosted by the We-B Centre (working with a-business) in the School of Management Information System, the School of Computer & Information Sciences at Edith Cowan University. This year\u27s conference is being held at the Sheraton Perth Hotel in Adelaide Terrace, Perth. Papers for this conference have been written by a wide range of academics and industry specialists. We have attracted participation from both national and international authors and organisations. The papers cover many topics, all within the field of information warfare and its applications, now and into the future. The papers have been grouped into six streams: • Networks • IWAR Strategy • Security • Risk Management • Social/Education • Infrastructur

Coordination for Scalable Multiple Robot Planning Under Temporal Uncertainty

This dissertation incorporates coalition formation and probabilistic planning towards a domain-independent automated planning solution scalable to multiple heterogeneous robots in complex domains. The first research direction investigates the effectiveness of Task Fusion and introduces heuristics that improve task allocation and result in better quality plans, while requiring lower computational cost than the baseline approaches. The heuristics incorporate relaxed plans to estimate coupling and determine which tasks to fuse. As a result, larger temporal continuous planning problems involving multiple robots can be solved. The second research direction introduces new coordination methods to merge plans and resolve conflicts while extending the framework to domains with stochastic action duration. Merging distributedly generated plans becomes computationally costly when task plans are tightly coupled, and conflicts arise due to dependencies between plan actions. Existing methods either scale poorly as the number of agents and tasks increases, or do not minimize makespan, the overall time necessary to execute all tasks. A new family of plan coordination and conflict resolution algorithms is introduced to merge independently generated plans, minimize the resulting makespan, and scale to a large number of tasks and agents in complex problems. A thorough algorithmic analysis and empirical evaluation demonstrates how the new conflict identification and resolution models can impact the resulting plan quality and computational cost across three heterogeneous multiagent domains and outperform the baseline algorithms