An autonomous three–dimensional vision sensor with ears

Abstract- This paper describes our newly developed intelligent sensor system which comprises two eyes and four ears on a movable head. It can acquire its dynamical visual and auditory image of its surrounding 3-D environment while showing humanlike behavior naturally and autonomously. The most important feature of the sensor system is in a unified sensory architecture throughout low-level and intermediate level visual and auditory functions. This enables us to achieve 1) rapid (5ms) and accurate (rt2deg) auditory localization, 2) rapid (0.5s/65536pixel) extraction of motion and texture features, 3) rapid (O.ls/4096pixel) reconstruction of 3-D object profile, 4) rapid (several TV frame times) eye movement and binocular fixation which is activated by auditory localization and motion detection. We describe in this paper the several key items for realizing this sensor.