Adaptive Load Balancing: A Study in Multi-Agent Learning

We study the process of multi-agent reinforcement learning in the context of load balancing in a distributed system, without use of either central coordination or explicit communication. We first define a precise framework in which to study adaptive load balancing, important features of which are its stochastic nature and the purely local information available to individual agents. Given this framework, we show illuminating results on the interplay between basic adaptive behavior parameters and their effect on system efficiency. We then investigate the properties of adaptive load balancing in heterogeneous populations, and address the issue of exploration vs. exploitation in that context. Finally, we show that naive use of communication may not improve, and might even harm system efficiency.Comment: See http://www.jair.org/ for any accompanying file

Complexity in Foresight: experiences with INTERSECTIONS: an agent-based simulation workbench to help achieve adaptiveness in strategic planning

“Complexity in Foresight” is a new synthetic paradigm that crosses areas in strategic planning and the complexity sciences. It connects the fields of agent-based simulation and complex adapative systems, and provides the overall blueprint for the construction of a new generation of toolkits. The plan is ambitious: to help achieve adaptiveness in strategic planning. My proposal is to start the construction of an agent-based simulation workbench with the ingredients: would-be worlds, building-block approaches and learning-action networks. The workbench will be designed to support learning-action networks; the informal networks of scientists, policy-makers and stakeholders that have a critical role for sustainable development. Their interactions and learning will be facilitated by would-be worlds; agent-based simulation models that function as “laboratories”, which the used to generate crude images of transitional change. These images will be treated as thought experiments, designed to make it easier for the planners to switch between observable realities and possible realities. Building-block approaches help to organize the modeling, experimentation and learning processes in a very flexible way, so that the overall process becomes adaptive. In this thesis I present the “Framework for Synthesis” designed to facilitate a unifying process to the development and use of would-be worlds. I build tools and methods and integrate them into the “INTERSECTIONS” workbench. I apply different combinations of these tools and methods in two case studies. I evaluate the potential usefulness of the Framework for Synthesis to support learning-action networks. I present the Framework on the CD-ROM included with this thesis, so that the reader can interact with the tools and methods