Skip to main content
Article thumbnail
Location of Repository

Combining Heuristics and Q-Learning in an Adaptive Light Seeking Robot

By Constantinos A Kroustis and Matthew C Casey


The use of adaptive techniques in robots that operate in a real environment is an important area of research. In particular, with our developing understanding of brain\ud function, robots can be used to explore biologically motivated algorithms. However, developing a framework in which such algorithms can be tested is a challenge, requiring low-level processing functions that can take care of sensory preprocessing and reflex motor actions for adaptive algorithms. In this paper, we present a framework that combines together low-level heuristics (reflex actions) and a high-level adaptive algorithm (Q-Learning). We implement this framework on the LEGO Mindstorms NXT. The robot is designed to learn how to find a light source in an unknown environment. Our results show that the combination of the heuristic with the adaptive algorithm reduce the number of execution steps required to find the light, and that the framework enables the adaptive algorithm to successfully complete its task. Through this implementation, we also demonstrate how the NXT can be used as a suitable platform for the development of complex algorithms, using remote commands via Bluetooth

Publisher: Department of Computing, University of Surrey
Year: 2008
OAI identifier:

Suggested articles

To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.