We report in this article a real time algorithm TIM that can track and estimate hand gesture, providing reliably 5 degrees of freedom from a single camera for controlling of a robot head. The advantages of this algorithm are: it does not require camera calibration and the syetem will operate under any configuration; secondly it converges fast and reliably which means special hardware, such as a powerful parallel platform, may be dispensed with. Kalman filter is used to track feature points on the finger tips. 