Object-based Tag Propagation for Semi-automatic Annotation of Images

Over the last few years, social network systems have greatly increased users’ involvement in online content creation and annotation. Since such systems usually need to deal with a large amount of multimedia data, it becomes desirable to realize an interactive service that minimizes tedious and time consuming manual annotation. In this paper, we propose an interactive online platform that is capable of performing semi-automatic image annotation and tag recommendation for an extensive online database. First, when the user marks a specific object in an image, the system performs an object duplicate detection and returns the search results with images containing similar objects. Then, the annotation of the object can be performed in two ways: (1) In the tag recommendation process, the system recommends tags associated with the object in images of the search results, among which, the user can accept some tags for the object in the given image. (2) In the tag propagation process, when the user enters his/her tag for the object, it is propagated to images in the search results. Different techniques to speed-up the process of indexing and retrieval are presented in this paper and their effectiveness demonstrated through a set of experiments considering various classes of objects

Socializing the Semantic Gap: A Comparative Survey on Image Tag Assignment, Refinement and Retrieval

Where previous reviews on content-based image retrieval emphasize on what can be seen in an image to bridge the semantic gap, this survey considers what people tag about an image. A comprehensive treatise of three closely linked problems, i.e., image tag assignment, refinement, and tag-based image retrieval is presented. While existing works vary in terms of their targeted tasks and methodology, they rely on the key functionality of tag relevance, i.e. estimating the relevance of a specific tag with respect to the visual content of a given image and its social context. By analyzing what information a specific method exploits to construct its tag relevance function and how such information is exploited, this paper introduces a taxonomy to structure the growing literature, understand the ingredients of the main works, clarify their connections and difference, and recognize their merits and limitations. For a head-to-head comparison between the state-of-the-art, a new experimental protocol is presented, with training sets containing 10k, 100k and 1m images and an evaluation on three test sets, contributed by various research groups. Eleven representative works are implemented and evaluated. Putting all this together, the survey aims to provide an overview of the past and foster progress for the near future.Comment: to appear in ACM Computing Survey

Heritage image annotation via collective knowledge

© 2019 Elsevier Ltd The automatic image annotation can provide semantic illustrations to understand image contents, and builds a foundation to develop algorithms that can search images within a large database. However, most current methods focus on solving the annotation problem by modeling the image visual content and tag semantic information, which overlooks the additional information, such as scene descriptions and locations. Moreover, the majority of current annotation datasets are visually consistent and only annotated by common visual objects and attributes, which makes the classic methods vulnerable to handle the more diverse image annotation. To address above issues, we propose to annotate images via collective knowledge, that is, we uncover relationships between the image and its neighbors by measuring similarities among metadata and conduct the metric learning to obtain the representations of image contents, we also generate semantic representations for images given collective semantic information from their neighbors. Two representations from different paradigms are embedded together to train an annotation model. We ground our model on the heritage image collection we collected from the library online open data. Annotations on the heritage image collection are not limited to common visual objects, and are highly relevant to historical events, and the diversity of the heritage image content is much larger than the current datasets, which makes it more suitable for this task. Comprehensive experimental results on the benchmark dataset indicate that the proposed model achieves the best performance compared to baselines and state-of-the-art methods

TAG ME: An Accurate Name Tagging System for Web Facial Images using Search-Based Face Annotation

Now a day the demand of social media is increases rapidly and most of the part of social media is made up of multimedia content cognate as images, audio, video. Hence for taking this as a motivation we have proffer a framework for Name tagging or labeling For Web Facial Images, which are easily obtainable on the internet. TAG ME system does that name tagging by utilizing search-based face annotation (SBFA). Here we are going to select an image from a database which are weakly labeled on the internet and the "TAG ME" assign a correct and accurate names or tags to that facial image, for doing this a few challenges have to be faced the One exigent difficulty for search-based face annotation strategy is how to effectually conduct annotation by utilizing the list of nearly all identical face images and its labels which is weak that are habitually rowdy and deficient. In TAGME we have resolve this problem by utilizing an effectual semi supervised label refinement (SSLR) method for purify the labels of web and nonweb facial images with the help of machine learning techniques. Secondly we used convex optimization techniques to resolve learning problem and used effectual optimization algorithms to resolve the learning task which is based on the large scale integration productively. For additionally quicken the given system, finally TAGME system proposed clustering-based approximation algorithm which boost the scalability considerably