Socializing the Semantic Gap: A Comparative Survey on Image Tag Assignment, Refinement and Retrieval

Where previous reviews on content-based image retrieval emphasize on what can be seen in an image to bridge the semantic gap, this survey considers what people tag about an image. A comprehensive treatise of three closely linked problems, i.e., image tag assignment, refinement, and tag-based image retrieval is presented. While existing works vary in terms of their targeted tasks and methodology, they rely on the key functionality of tag relevance, i.e. estimating the relevance of a specific tag with respect to the visual content of a given image and its social context. By analyzing what information a specific method exploits to construct its tag relevance function and how such information is exploited, this paper introduces a taxonomy to structure the growing literature, understand the ingredients of the main works, clarify their connections and difference, and recognize their merits and limitations. For a head-to-head comparison between the state-of-the-art, a new experimental protocol is presented, with training sets containing 10k, 100k and 1m images and an evaluation on three test sets, contributed by various research groups. Eleven representative works are implemented and evaluated. Putting all this together, the survey aims to provide an overview of the past and foster progress for the near future.Comment: to appear in ACM Computing Survey

Automated image tagging through tag propagation

Trabalho apresentado no âmbito do Mestrado em Engenharia Informática, como requisito parcial Para obtenção do grau de Mestre em Engenharia InformáticaToday, more and more data is becoming available on the Web. In particular, we have recently witnessed an exponential increase of multimedia content within various content sharing websites. While this content is widely available, great challenges have arisen to effectively search and browse such vast amount of content. A solution to this problem is to annotate information, a task that without computer aid requires a large-scale human effort. The goal of this thesis is to automate the task of annotating multimedia information with machine learning algorithms. We propose the development of a machine learning framework capable of doing automated image annotation in large-scale consumer photos. To this extent a study on state of art algorithms was conducted, which concluded with a baseline implementation of a k-nearest neighbor algorithm. This baseline was used to implement a more advanced algorithm capable of annotating images in the situations with limited training images and a large set of test images – thus, a semi-supervised approach. Further studies were conducted on the feature spaces used to describe images towards a successful integration in the developed framework. We first analyzed the semantic gap between the visual feature spaces and concepts present in an image, and how to avoid or mitigate this gap. Moreover, we examined how users perceive images by performing a statistical analysis of the image tags inserted by users. A linguistic and statistical expansion of image tags was also implemented. The developed framework withstands uneven data distributions that occur in consumer datasets, and scales accordingly, requiring few previously annotated data. The principal mechanism that allows easier scaling is the propagation of information between the annotated data and un-annotated data

An Effective Technique for Removal of Facial Dupilcation by SBFA

Search based face annotation (SBFA) is an effective technique to annotate the weakly labeled facial images that are freely available on World Wide Web. The main objective of search based face annotation is to assign correct name labels to given query facial image. One difficult drawback for search based face annotation theme is how to effectively perform annotation by exploiting the list of most similar facial pictures and their weak labels that square measure typically droning and incomplete. To tackle this drawback, we tend to propose a good unattended label refinement (URL) approach for purification the labels of web facial pictures exploitation machine learning technique. We tend to formulate the educational drawback as a gibbose improvement and develop effective improvement algorithms to resolve the large scale learning task expeditiously. To additional speed up the projected theme, we also proposed clustering based approximation algorithmic program which may improve quantify ability significantly. We have conducted an in depth set of empirical studies on a large scale net facial image test bed, within which encouraging results showed that the projected URL algorithms will considerably boost the performance of the promising SBFA theme. In future work we will use HAAR algorithm. HAAR is feature based method for face detection. HAAR features, integral images, recognized detection of features improve face detection in terms of speed and accuracy. DOI: 10.17762/ijritcc2321-8169.150517

Labeling Faces Victimization Bunch Primarily Based Internet Pictures Annotation to Produce Authentication in Security

Auto face annotation is important in abounding absolute apple advice administration systems. Face tagging in images and videos enjoys abounding abeyant applications in multimedia advice retrieval. Face comment is a meadow of face apprehension and recognition. Mining abominably labeled facial images on the internet shows abeyant classic appear auto face annotation. This blazon of classic motivates the new assay botheration of defended authentication. The ambition of the arrangement is to comment disregarded faces in images and videos with the words that best alarm the image. A framework called seek based face comment (SBFA) provides the way to abundance abominably labeled facial images. Facial images that are accessible on Apple Wide Web (WWW) or the angel database created by the aegis administration can be annotated. A one arduous botheration with the seek based face comment arrangement is how finer accomplish comment by advertisement agnate facial images and their anemic labels which are blatant and incomplete. To affected this botheration proposed admission uses unsupervised characterization clarification (ULR) to clarify the labels of web facial images. To acceleration up the proposed arrangement a absorption based approximation algorithm is used. Uses of comment will advice for user to seek admiration angel and video. As well if arrangement gets implemented in amusing arrangement again it will affected the check of accepted absolute arrangement which tags manually

Social User Mining: User Profiling of Social Media Network Based on Multimedia Data Mining

In recent years, the pervasive use of social media has generated extraordinary amounts of data that has started to gain an increasing amount of attention. Each social media source utilizes different data types such as textual and visual. For example, Twitter is used to transmit short text messages, whereas Flickr is used to convey images and videos. Moreover, Facebook uses all of these data types. From the social media users’ standpoint, it is highly desirable to find patterns from different data formats. The result of the huge amount of data from different sources or types has provided many opportunities for researchers in the fields of data mining and data analytics. Not only the methods and tools to organize and manage such data have become extremely important, but also methods and tools to discover hidden knowledge from such data, which can be used for a variety of applications. For example, the mining of a user's profile on social media could help to discover any missing information, including the user's location or gender information. However, the task of developing such methods and tools is very challenging. Social media data is unstructured and different from traditional data because of its privacy settings, data noise, and large capacity of data. Moreover, combining image features and text information annotated by users reveals interesting properties of social user mining, and serves as a useful tool for discovering unknown information about the users. Minimal research has been conducted on the combination of image and text data for social user mining. To address these challenges and to discover unknown information about users, we proposed a novel mining framework for social user mining that includes: 1) a data assemble module for different media source, 2) a data integration module, and 3) mining applications. First, we introduced a data assemble module in order to process both the textual and the visual information from different media sources, and evaluated the appropriate multimedia features for social user mining. Then, we proposed a new data integration method in order to integrate the textual and the visual data. Unlike the previous approaches that used a content based approach to merge multiple types of features, our main approach is based on image semantics through a semi-automatic image tagging system. Lastly, we presented two different application as an example of social user mining, gender classification and user location

CHORUS Deliverable 2.1: State of the Art on Multimedia Search Engines

Based on the information provided by European projects and national initiatives related to multimedia search as well as domains experts that participated in the CHORUS Think-thanks and workshops, this document reports on the state of the art related to multimedia content search from, a technical, and socio-economic perspective. The technical perspective includes an up to date view on content based indexing and retrieval technologies, multimedia search in the context of mobile devices and peer-to-peer networks, and an overview of current evaluation and benchmark inititiatives to measure the performance of multimedia search engines. From a socio-economic perspective we inventorize the impact and legal consequences of these technical advances and point out future directions of research