Automatic Image Annotation using Multiple Grid Segmentation

Abstract. Automatic image annotation refers to the process of automatically labeling an image with a predefined set of keywords. Image annotation is an important step of content-based image retrieval (CBIR), which is relevant for many real-world applications. In this paper, a new algorithm based on multiple grid segmentation, entropy-based information and a Bayesian classifier, is proposed for an efficient, yet very effective, image annotation process. The proposed approach follows a two step process. In the first step, the algorithm generates grids of different sizes and different overlaps, and each grid is classified with a Naive Bayes classifier. In a second step, we used information based on the predicted class probability, its entropy, and the entropy of the neighbors of each grid element at the same and different resolutions, as input to a second binary classifier that qualifies the initial classification to select the correct segments. This significantly reduces false positives and improves the overall performance. We performed several experiments with images from the MSRC-9 database collection, which has manual ground truth segmentation and annotation information. The results show that the proposed approach has a very good performance compared to the initial labeling, and it also improves other scheme based on multiple segmentations.