6,835 research outputs found
Batch Reinforcement Learning on the Industrial Benchmark: First Experiences
The Particle Swarm Optimization Policy (PSO-P) has been recently introduced
and proven to produce remarkable results on interacting with academic
reinforcement learning benchmarks in an off-policy, batch-based setting. To
further investigate the properties and feasibility on real-world applications,
this paper investigates PSO-P on the so-called Industrial Benchmark (IB), a
novel reinforcement learning (RL) benchmark that aims at being realistic by
including a variety of aspects found in industrial applications, like
continuous state and action spaces, a high dimensional, partially observable
state space, delayed effects, and complex stochasticity. The experimental
results of PSO-P on IB are compared to results of closed-form control policies
derived from the model-based Recurrent Control Neural Network (RCNN) and the
model-free Neural Fitted Q-Iteration (NFQ). Experiments show that PSO-P is not
only of interest for academic benchmarks, but also for real-world industrial
applications, since it also yielded the best performing policy in our IB
setting. Compared to other well established RL techniques, PSO-P produced
outstanding results in performance and robustness, requiring only a relatively
low amount of effort in finding adequate parameters or making complex design
decisions
A Unified Framework for Multi-Agent Agreement
Multi-Agent Agreement problems (MAP) - the ability of a population of agents to search out and converge on a common state - are central issues in many multi-agent settings, from distributed sensor networks, to meeting scheduling, to development of norms, conventions, and language. While much work has been done on particular agreement problems, no unifying framework exists for comparing MAPs that vary in, e.g., strategy space complexity, inter-agent accessibility, and solution type, and understanding their relative complexities. We present such a unification, the Distributed Optimal Agreement Framework, and show how it captures a wide variety of agreement problems. To demonstrate DOA and its power, we apply it to two well-known MAPs: convention evolution and language convergence. We demonstrate the insights DOA provides toward improving known approaches to these problems. Using a careful comparative analysis of a range of MAPs and solution approaches via the DOA framework, we identify a single critical differentiating factor: how accurately an agent can discern other agent.s states. To demonstrate how variance in this factor influences solution tractability and complexity we show its effect on the convergence time and quality of Particle Swarm Optimization approach to a generalized MAP
Efficiency Analysis of Swarm Intelligence and Randomization Techniques
Swarm intelligence has becoming a powerful technique in solving design and
scheduling tasks. Metaheuristic algorithms are an integrated part of this
paradigm, and particle swarm optimization is often viewed as an important
landmark. The outstanding performance and efficiency of swarm-based algorithms
inspired many new developments, though mathematical understanding of
metaheuristics remains partly a mystery. In contrast to the classic
deterministic algorithms, metaheuristics such as PSO always use some form of
randomness, and such randomization now employs various techniques. This paper
intends to review and analyze some of the convergence and efficiency associated
with metaheuristics such as firefly algorithm, random walks, and L\'evy
flights. We will discuss how these techniques are used and their implications
for further research.Comment: 10 pages. arXiv admin note: substantial text overlap with
arXiv:1212.0220, arXiv:1208.0527, arXiv:1003.146
- …