PhDThis thesis investigates reinforcement learning algorithms suitable for learning
in large state space problems and coevolution. In order to learn in large state
spaces, the state space must be collapsed to a computationally feasible size and
then generalised about. This thesis presents two new implementations of the
classic temporal difference (TD) reinforcement learning algorithm Sarsa that
utilise fuzzy logic principles for approximation, FQ Sarsa and Fuzzy Sarsa. The
effectiveness of these two fuzzy reinforcement learning algorithms is
investigated in the context of an agent marketplace. It presents a practical
investigation into the design of fuzzy membership functions and tile coding
schemas. A critical analysis of the fuzzy algorithms to a related technique in
function approximation, a coarse coding approach called tile coding is given in
the context of three different simulation environments; the mountain-car
problem, a predator/prey gridworld and an agent marketplace. A further
comparison between Fuzzy Sarsa and tile coding in the context of the nonstationary
environments of the agent marketplace and predator/prey gridworld is
presented.
This thesis shows that the Fuzzy Sarsa algorithm achieves a significant reduction
of state space over traditional Sarsa, without loss of the finer detail that the FQ
Sarsa algorithm experiences. It also shows that Fuzzy Sarsa and gradient descent
Sarsa(λ) with tile coding learn similar levels of distinction against a stationary
strategy. Finally, this thesis demonstrates that Fuzzy Sarsa performs better in a
competitive multiagent domain than the tile coding solution
