Instagram Hashtags as Image Annotation Metadata

Part 3: Data Representation and AnalysisInternational audienceImage tagging is an essential step for developing automatic image annotation methods that are based on the learning by example paradigm. However, manual image annotation, even for creating training sets for machine learning algorithms, requires hard effort and contains human judgment errors and subjectivity. Thus, alternative ways for automatically creating training examples, i.e., pairs of images and tags, are pursued. In this work we investigate whether tags accompanying photos in social media and especially the Instagram hashtags, provide a form of image annotation. If such a claim is proved then Instagram could be a very rich source of training data, easily collectable automatically, for the development of automatic image annotation techniques. Our hypothesis is that Instagram hashtags, and especially those provided by the photo owner / creator, express more accurately the content of a photo compared to the tags assigned to a photo during explicit image annotation processes like crowdsourcing. In this context, we explore the descriptive power of hashtags by examining whether other users would use the same, with the owner, hashtags to annotate an image. For this purpose a set of 30 randomly chosen, from Instagram, images were used as a dataset for our research. Then, one to four hashtags, considered as the most descriptive ones for the image in question, were selected among the hashtags used by the image owner. Three online questionnaires with ten images each were distributed to experiment participants in order to choose the best suitable hashtag for every image according to their interpretation. Results show that an average of 55% of the participants hashtag choices coincide with those suggested by the photo owners; thus, an initial evidence towards our hypothesis confirmation can be claimed