Using Reinforcement Learning to Attenuate for Stochasticity in Robot Navigation Controllers

International audienceBraitenberg vehicles are bio-inspired controllers for sensor-based local navigation of wheeled robots that have been used in multiple real world robotic implementations. The common approach to implement such non-linear control mechanisms is through neural networks connecting sensing to motor action, yet tuning the weights to obtain appropriate closed-loop navigation behaviours can be very challenging. Standard approaches used hand tuned spiking or recurrent neural networks, or learnt the weights of feedforward networks using evolutionary approaches. Recently, Reinforcement Learning has been used to learn neural controllers for simulated Braitenberg vehicle 3a-a bio-inspired model of target seeking for wheeled robots-under the assumption of noiseless sensors. Real sensors, however, are subject to different levels of noise, and multiple works have shown that Braitenberg vehicles work even on outdoor robots, demonstrating that these control mechanisms work in harsh and dynamic environments. This paper shows that a robust neural controller for Braitenberg vehicle 3a can be learnt using policy gradient reinforcement learning in scenarios where sensor noise plays a non negligible role. The learnt controller is robust and tries to attenuate the effects of noise in the closed-loop navigation behaviour of the simulated stochastic vehicle. We compare the neural controller learnt using Reinforcement Learning with a simple hand tuned controller and show how the neural control mechanism outperforms a naïve controller. Results are illustrated through computer simulations of the closed-loop stochastic system

Mechanisms of Odor-Tracking: Multiple Sensors for Enhanced Perception and Behavior

Early in evolution, the ability to sense and respond to changing environments must have provided a critical survival advantage to living organisms. From bacteria and worms to flies and vertebrates, sophisticated mechanisms have evolved to enhance odor detection and localization. Here, we review several modes of chemotaxis. We further consider the relevance of a striking and recurrent motif in the organization of invertebrate and vertebrate sensory systems, namely the existence of two symmetrical olfactory sensors. By combining our current knowledge about the olfactory circuits of larval and adult Drosophila, we examine the molecular and neural mechanisms underlying robust olfactory perception and extend these analyses to recent behavioral studies addressing the relevance and function of bilateral olfactory input for gradient detection. Finally, using a comparative theoretical approach based on Braitenberg's vehicles, we speculate about the relationships between anatomy, circuit architecture and stereotypical orientation behaviors