With a growing number of elderly people in the UK, more and more of them suffer from various kinds of cognitive impairment. Cognitive impairment can be divided into different stages such as mild cognitive impairment (MCI) and severe cognitive impairment like dementia. Its early detection can be of great importance. However, it is challenging to detect cognitive impairment in the early stage with high accuracy and low cost, when most of the symptoms may not be fully expressed. This survey paper mainly reviews the state of the art techniques for the early detection of cognitive impairment and compares their advantages and weaknesses. In order to build an effective and low-cost automatic system for detecting and monitoring the cognitive impairment for a wide range of elderly people, the applications of computer vision techniques for the early detection of cognitive impairment by monitoring facial expressions, body movements and eye movements are highlighted in this paper. In additional to technique review, the main research challenges for the early detection of cognitive impairment with high accuracy and low cost are analysed in depth. Through carefully comparing and contrasting the currently popular techniques for their advantages and weaknesses, some important research directions are particularly pointed out and highlighted from the viewpoints of the authors alone