Skip to main content
Article thumbnail
Location of Repository

Learning visual docking for non-holonomic autonomous vehicles

By Tomas Martinez-Marin and Tom Duckett


This paper presents a new method of learning visual docking skills for non-holonomic vehicles by direct interaction with the environment. The method is based on a reinforcement algorithm, which speeds up Q-learning by applying memorybased sweeping and enforcing the “adjoining property”, a filtering mechanism to only allow transitions between states that satisfy a fixed distance. The method overcomes some limitations of reinforcement learning techniques when they are employed in applications with continuous non-linear systems, such as car-like vehicles. In particular, a good approximation to the optimal\ud behaviour is obtained by a small look-up table. The algorithm is tested within an image-based visual servoing framework on a docking task. The training time was less than 1 hour on the real vehicle. In experiments, we show the satisfactory performance of the algorithm

Topics: G700 Artificial Intelligence, G760 Machine Learning, G400 Computer Science, G740 Computer Vision
Year: 2008
OAI identifier:

Suggested articles


  1. (1985). A discrete method of optimal control based upon the cell state space concept,” doi
  2. (2002). Effective reinforcement learning for mobile robots,” in doi
  3. (2005). Fast reinforcement learning for vision-guided mobile robots,” doi
  4. (2003). ınez-Mar´ ın, “Improved optimal control methods based upon the adjoining cell mapping technique,” doi
  5. (2003). ınez-Mar´ ın, “Optimal path planning for car-like vehicles in the presence of obstacles,” in doi
  6. (1996). Neurodynamic Programming. Athena Scientific,
  7. (1990). Optimal path for a car that goes both forward and backward,” doi
  8. (2000). Path planning in image space for robust visual servoing,” in doi
  9. (1993). Priortized sweeping: Reinforcement learning with less data and less time,” doi
  10. (1991). Reinforcement learning architectures for animats,” in From Animals to Animats, doi
  11. (2000). Reinforcement learning for visual servoing of a mobile robot,” in doi
  12. (1998). Reinforcement Learning: An Introduction. doi
  13. (2003). Robot docking with neural vision and reinforcement,” in doi
  14. (1991). Robot dynamics and control. doi
  15. (1993). The adjoining cell mapping and its recursive unraveling, part i: Description of adaptive and recursive algorithms,” doi

To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.