1,778 research outputs found
Optimality-based Analysis of XCSF Compaction in Discrete Reinforcement Learning
Learning classifier systems (LCSs) are population-based predictive systems
that were originally envisioned as agents to act in reinforcement learning (RL)
environments. These systems can suffer from population bloat and so are
amenable to compaction techniques that try to strike a balance between
population size and performance. A well-studied LCS architecture is XCSF, which
in the RL setting acts as a Q-function approximator. We apply XCSF to a
deterministic and stochastic variant of the FrozenLake8x8 environment from
OpenAI Gym, with its performance compared in terms of function approximation
error and policy accuracy to the optimal Q-functions and policies produced by
solving the environments via dynamic programming. We then introduce a novel
compaction algorithm (Greedy Niche Mass Compaction - GNMC) and study its
operation on XCSF's trained populations. Results show that given a suitable
parametrisation, GNMC preserves or even slightly improves function
approximation error while yielding a significant reduction in population size.
Reasonable preservation of policy accuracy also occurs, and we link this metric
to the commonly used steps-to-goal metric in maze-like environments,
illustrating how the metrics are complementary rather than competitive
Distributed classifier migration in XCS for classification of electroencephalographic signals
This paper presents an investigation into combining migration strategies inspired by multi-deme Parallel Genetic Algorithms with the XCS Learning Classifier System to provide parallel and distributed classifier migration. Migrations occur between distributed XCS classifier sub-populations using classifiers ranked according to numerosity, fitness or randomly selected. The influence of the degree-of-connectivity introduced by Fully-Connected, Bi-directional Ring and Uni-directional Ring topologies is examined. Results indicate that classifier migration is an effective method for improving classification accuracy, improving learning speed and reducing final classifier population size, in the single-step classification of noisy, artefact-inclusive human electroencephalographic signals. The experimental results will be used as part of our larger research effort investigating the feasibility of using EEG signals as an interface to allow paralysed persons to control a powered wheelchair or other devices. © 2007 IEEE
- …