Log-Distributional Approach for Learning Covariate Shift Ratios

Distributional Reinforcement Learning theory suggests that distributional fixed points could play a fundamental role to learning non additive value functions. In particular, we propose a distributional approach for learning Covariate Shift Ratios, whose update rule is originally multiplicative

Application of Techniques for MAP Estimation to Distributed Constraint Optimization Problem

The problem of efficiently finding near-optimal decisions in multi-agent systems has become increasingly important because of the growing number of multi-agent applications with large numbers of agents operating in real-world environments. In these systems, agents are often subject to tight resource constraints and agents have only local views. When agents have non-global constraints, each of which is independent, the problem can be formalized as a distributed constraint optimization problem (DCOP). The DCOP is closely associated with the problem of inference on graphical models. Many approaches from inference literature have been adopted to solve DCOPs. We focus on the Max-Sum algorithm and the Action-GDL algorithm that are DCOP variants of the popular inference algorithm called the Max-Product algorithm and the Belief Propagation algorithm respectively. The Max-Sum algorithm and the Action-GDL algorithm are well-suited for multi-agent systems because it is distributed by nature and requires less communication than most DCOP algorithms. However, the resource requirements of these algorithms are still high for some multi-agent domains and various aspects of the algorithms have not been well studied for use in general multi-agent settings. This thesis is concerned with a variety of issues of applying the Max-Sum algorithms and the Action-GDL algorithm to general multi-agent settings. We develop a hybrid algorithm of ADOPT and Action-GDL in order to overcome the communication complexity of DCOPs. Secondly, we extend the Max-Sum algorithm to operate more efficiently in more general multi-agent settings in which computational complexity is high. We provide an algorithm that has a lower expected computational complexity for DCOPs even with n-ary constraints. Finally, In most DCOP literature, a one-to-one mapping between a variable and an agent is assumed. However, in real applications, many-to-one mappings are prevalent and can also be beneficial in terms of communication and hardware cost in situations where agents are acting as independent computing units. We consider how to exploit such mapping in order to increase efficiency

Benchmarking Continuous Dynamic Optimization: Survey and Generalized Test Suite

Dynamic changes are an important and inescapable aspect of many real-world optimization problems. Designing algorithms to find and track desirable solutions while facing challenges of dynamic optimization problems is an active research topic in the field of swarm and evolutionary computation. To evaluate and compare the performance of algorithms, it is imperative to use a suitable benchmark that generates problem instances with different controllable characteristics. In this paper, we give a comprehensive review of existing benchmarks and investigate their shortcomings in capturing different problem features. We then propose a highly configurable benchmark suite, the generalized moving peaks benchmark, capable of generating problem instances whose components have a variety of properties such as different levels of ill-conditioning, variable interactions, shape, and complexity. Moreover, components generated by the proposed benchmark can be highly dynamic with respect to the gradients, heights, optimum locations, condition numbers, shapes, complexities, and variable interactions. Finally, several well-known optimizers and dynamic optimization algorithms are chosen to solve generated problems by the proposed benchmark. The experimental results show the poor performance of the existing methods in facing new challenges posed by the addition of new properties

Coalition Formation For Distributed Constraint Optimization Problems

This dissertation presents our research on coalition formation for Distributed Constraint Optimization Problems (DCOP). In a DCOP, a problem is broken up into many disjoint sub-problems, each controlled by an autonomous agent and together the system of agents have a joint goal of maximizing a global utility function. In particular, we study the use of coalitions for solving distributed k-coloring problems using iterative approximate algorithms, which do not guarantee optimal results, but provide fast and economic solutions in resource constrained environments. The challenge in forming coalitions using iterative approximate algorithms is in identifying constraint dependencies between agents that allow for effective coalitions to form. We first present the Virtual Structure Reduction (VSR) Algorithm and its integration with a modified version of an iterative approximate solver. The VSR algorithm is the first distributed approach for finding structural relationships, called strict frozen pairs, between agents that allows for effective coalition formation. Using coalition structures allows for both more efficient search and higher overall utility in the solutions. Secondly, we relax the assumption of strict frozen pairs and allow coalitions to form under a probabilistic relationship. We identify probabilistic frozen pairs by calculating the propensity between two agents, or the joint probability of two agents in a k-coloring problem having the same value in all satisfiable instances. Using propensity, we form coalitions in sparse graphs where strict frozen pairs may not exist, but there is still benefit to forming coalitions. Lastly, we present a cooperative game theoretic approach where agents search for Nash stable coalitions under the conditions of additively separable and symmetric value functions

Distributed optimisation for traffic management

This thesis reports on the development of a multi-agent approach to distributed traffic optimisation. In particular, I propose a solution to the dynamic traffic assignment problem in a decentralised manner and then I introduce the new infrastructurelessly decentralised traffic information system. By using this system, each vehicle agent is able to update the current traffic condition through vehicle-to-vehicle communication. For solving dynamic traffic assignment problem, I propose a novel completely decentralised multi-agent coordination algorithm, which is a synergy between dynamic distributed constraint optimisation problem (DynDCOP) algorithm and auction. Using this algorithm, vehicle agent is able to reduce its individual travel time as well as total travel time of overall system. This simulation is carried out in order to evaluate different traffic planning algorithms that include decentralised uncoordination, centralised coordination and decentralised coordination algorithms. Finally, the experimental results show that the performance of proposed decentalised coorindation algorithm is high in comparison to centralised coordination algorithm