Optimizing Memory-Bounded Controllers for Decentralized POMDPs

We present a memory-bounded optimization approach for solving infinite-horizon decentralized POMDPs. Policies for each agent are represented by stochastic finite state controllers. We formulate the problem of optimizing these policies as a nonlinear program, leveraging powerful existing nonlinear optimization techniques for solving the problem. While existing solvers only guarantee locally optimal solutions, we show that our formulation produces higher quality controllers than the state-of-the-art approach. We also incorporate a shared source of randomness in the form of a correlation device to further increase solution quality with only a limited increase in space and time. Our experimental results show that nonlinear optimization can be used to provide high quality, concise solutions to decentralized decision problems under uncertainty.Comment: Appears in Proceedings of the Twenty-Third Conference on Uncertainty in Artificial Intelligence (UAI2007

Genetic algorithms with immigrants and memory schemes for dynamic shortest path routing problems in mobile ad hoc networks

This article is posted here with permission of IEEE - Copyright @ 2010 IEEEIn recent years, the static shortest path (SP) problem has been well addressed using intelligent optimization techniques, e.g., artificial neural networks, genetic algorithms (GAs), particle swarm optimization, etc. However, with the advancement in wireless communications, more and more mobile wireless networks appear, e.g., mobile networks [mobile ad hoc networks (MANETs)], wireless sensor networks, etc. One of the most important characteristics in mobile wireless networks is the topology dynamics, i.e., the network topology changes over time due to energy conservation or node mobility. Therefore, the SP routing problem in MANETs turns out to be a dynamic optimization problem. In this paper, we propose to use GAs with immigrants and memory schemes to solve the dynamic SP routing problem in MANETs. We consider MANETs as target systems because they represent new-generation wireless networks. The experimental results show that these immigrants and memory-based GAs can quickly adapt to environmental changes (i.e., the network topology changes) and produce high-quality solutions after each change.This work was supported by the Engineering and Physical Sciences Research Council of U.K. underGrant EP/E060722/

Genetic algorithms with elitism-based immigrants for dynamic load balanced clustering problem in mobile ad hoc networks

This article is posted here with permission of IEEE - Copyright @ 2011 IEEEIn recent years, the static shortest path (SP) problem has been well addressed using intelligent optimization techniques, e.g., artificial neural networks, genetic algorithms (GAs), particle swarm optimization, etc. However, with the advancement in wireless communications, more and more mobile wireless networks appear, e.g., mobile networks [mobile ad hoc networks (MANETs)], wireless sensor networks, etc. One of the most important characteristics in mobile wireless networks is the topology dynamics, i.e., the network topology changes over time due to energy conservation or node mobility. Therefore, the SP routing problem in MANETs turns out to be a dynamic optimization problem. In this paper, we propose to use GAs with immigrants and memory schemes to solve the dynamic SP routing problem in MANETs. We consider MANETs as target systems because they represent new-generation wireless networks. The experimental results show that these immigrants and memory-based GAs can quickly adapt to environmental changes (i.e., the network topology changes) and produce high-quality solutions after each change.This work was supported by the Engineering and Physical Sciences Research Council (EPSRC) of UK under Grant EP/E060722/1 and Grant EP/E060722/2

The Dreaming Variational Autoencoder for Reinforcement Learning Environments

Reinforcement learning has shown great potential in generalizing over raw sensory data using only a single neural network for value optimization. There are several challenges in the current state-of-the-art reinforcement learning algorithms that prevent them from converging towards the global optima. It is likely that the solution to these problems lies in short- and long-term planning, exploration and memory management for reinforcement learning algorithms. Games are often used to benchmark reinforcement learning algorithms as they provide a flexible, reproducible, and easy to control environment. Regardless, few games feature a state-space where results in exploration, memory, and planning are easily perceived. This paper presents The Dreaming Variational Autoencoder (DVAE), a neural network based generative modeling architecture for exploration in environments with sparse feedback. We further present Deep Maze, a novel and flexible maze engine that challenges DVAE in partial and fully-observable state-spaces, long-horizon tasks, and deterministic and stochastic problems. We show initial findings and encourage further work in reinforcement learning driven by generative exploration.Comment: Best Student Paper Award, Proceedings of the 38th SGAI International Conference on Artificial Intelligence, Cambridge, UK, 2018, Artificial Intelligence XXXV, 201

Artificial Immune System Approach for Multi Objective Optimization

This paper presents a modified Artificial Immune System based approach to solve multi objective optimization problems. The main objective of the solution of multi objective optimization problem is to help a human decision maker in taking his/her decision for finding the most preferred solution as the final result. This artificial immune system algorithm makes use of mechanism inspired by vertebrate immune system and clonal selection principle. In the present model crossover mechanism is integrated into traditional artificial immune system algorithm based on clonal selection theory. The Algorithm is proposed with real parameters value not binary coded parameters. Only non dominated individual and feasible best antibodies will add to the memory set.  This algorithm will be used to solve various real life engineering multi-objective optimization problems. The attraction for choosing the artificial immune system to develop algorithm was that if an adaptive pool of antibodies can produce 'intelligent' behavior, we can use this power of computation to tackle the problem of multi objective optimization. Keywords: Artificial Immune System, Clonal Selection Theory, Multi Objective Optimization, Pareto Optimal