Combining Text Semantics and Image Geometry to Improve Scene Interpretation

Inthispaper,wedescribeanovelsystemthatidentifiesrelationsbetweentheobjectsextractedfromanimage. We started from the idea that in addition to the geometric and visual properties of the image objects, we could exploit lexical and semantic information from the text accompanying the image. As experimental set up, we gathered a corpus of images from Wikipedia as well as their associated articles. We extracted two types of objects: human beings and horses and we considered three relations that could hold between them: Ride, Lead, or None. We used geometric features as a baseline to identify the relations between the entities and we describe the improvements brought by the addition of bag-of-wordf eatures and predicate–arguments tructures we derived from the text. The best semantic model resulted in a relative error reduction of more than 18% over the baseline

Search Behaviour On Photo Sharing Platforms

The behaviour, goals, and intentions of users while searching for images in large scale online collections are not well understood, with image search log analysis providing limited insights, in part because they tend only to have access to user search and result click information. In this paper we study user search behaviour in a large photo-sharing platform, analyzing all user actions during search sessions (i.e. including post result-click pageviews). Search accounts for a significant part of user interactions with such platforms, and we show differences between the queries issued on such platforms and those on general image search. We show that search behaviour is influenced by the query type, and also depends on the user. Finally, we analyse how users behave when they reformulate their queries, and develop URL class prediction models for image search, showing that query-specific models significantly outperform query-agnostic models. The insights provided in this paper are intended as a launching point for the design of better interfaces and ranking models for image search. © 2013 IEEE.published_or_final_versio

The Daily Image Information Needs and Seeking Behavior of Chinese Undergraduate Students

A survey was conducted at Beijing Normal University to explore subjects’ motives for image seeking; the image types they need; how and where they seek images; and the difficulties they encounter. The survey also explored subjects’ attitudes toward current image services and their perceptions of how university libraries might provide assistance. Based on the findings, this article summarizes the features of Chinese undergraduate students’ daily image needs and their information behavior related to images. The findings reveal the need to improve the image services offered by academic libraries and strengthen undergraduates’ information literacy with respect to image search and use

On machine learning and knowledge organisation in Multimedia Information Retrieval

Recent technological developments have increased the use of machine learning to solve many problems, including many in information retrieval (IR). Deployment of machine-learning techniques is widespread in text search, notability web search engines (Dai et al., 2011). Multimedia information retrieval as a problem however still represents a significant challenge to machine learning as a technological solution, but some problems in IR can still be addressed by using appropriate AI techniques. In this paper we review the technological developments, and provide a perspective on the use of machine-learning techniques in conjunction with knowledge organisation techniques to address multimedia IR needs. We take the perspective from the MacFarlane (2016) position paper, that there are some problems in multimedia IR that AI and machine learning cannot currently solve. The semantic gap in multimedia IR (Enser, 2008) remains a significant problem in the field, and solutions to them are many years off. However, there are occasions where the new technological developments allow the use of knowledge organisation and machine learning in multimedia search systems and services. Specifically we argue that the improvement of detection of some classes of low level features in images (Karpathy and Li, 2015), music (Byrd and Crawford, 2002) and video (Hu et al., 2011) can be used in conjunction with knowledge organisation to tag or label multimedia content for better retrieval performance. We advocate the use of supervised learning techniques. We provide an overview of the use of knowledge organisation schemes in machine learning, and make recommendations to information professionals on the use of this technology with knowledge organisation techniques to solve multimedia IR problems

Expeditions through image jungles The commercial use of image libraries in an online environment

Purpose: Searching for appropriate images as part of a work task is a non-trivial problem. Journalists and copywriters need to find images that are not only visually appropriate to accompany the documents they are creating, but are acceptably priced and licensed. Methodology: A work based study methodology and grounded theory are used to collect qualitative data from a variety of creative professionals including journalists. Findings: We report the findings of a study to investigate image search, retrieval and use by creative professionals who routinely use images as part of their work in an online environment. We describe the commercial constraints that have an impact on the image users’ behaviour that are not reported in other more academic and lab based studies of image use (Westman, 2009). Practical implications: We show that the commercial image retrieval systems are based on document retrieval systems, and that this is not the most appropriate approach in the Journalism domain. Originality/value: We describe the properties of an ‘information expedition’; the image seeking behaviour exhibited by journalists in an online environment, and contend that it is significantly different to existing image seeking models which represent other user types

Analysing Creative Image Search Information Needs

Creative professionals in advertising, marketing, design and journalism search for images to visually represent a concept for their project. The main purpose of this paper is to present search facets derived from an analysis of documents known as briefs, which are widely used in creative industries as requirement documents describing information needs. The briefs specify the type of image required, such as the content and context of use for the image and represent the topic from which the searcher builds an image query. We take three main sources—user image search behaviour, briefs, and image search engine search facets to examine the search facets for image searching in order to examine the following research question—are search facet schemes for image search engines sufficient for user needs, or is revision needed? We found that there are three main classes of user search facet, which include business, contextual and image related information. The key argument in the paper is that the facet “keyword/tag” is ambiguous and does not support user needs for more generic descriptions to broaden search or specific descriptions to narrow their search — we suggest that a more detailed search facet scheme would be appropriate

Analysing creative image search information needs.

Creative professionals in advertising, marketing, design and journalism search for images to visually represent a concept for their project. The main purpose of this paper is to present an analysis of documents known as briefs to find search facets, which are widely used in creative industries as a requirements document to describe an information need. The briefs specify the type of image required, such as the content and context of use for the image, and represent the topic from which the searcher builds an image query. This research takes three main sources - user image search behaviour, briefs, search engine meta-data - to examine the search facets for image searching in order to examine the following research question - are meta-data schemes for image search engines sufficient for user needs, or is revision needed? This research found that there are three main classes of user search facet, which include business, contextual and image related information. The key argument in the paper is that the facet 'keyword/tag' is ambiguous and does not support user needs for more generic descriptions to broaden search or specific descriptions to narrow their search - we suggest that a more detailed search facet scheme would be appropriate

Online Visual Image Resources and Reference Services: Understanding Preferred Resources

As students and teachers in higher education begin to use images in their courses, assignments, and research more frequently, new skills and literacies are needed to find and use images on the Web. Images can be found online in several different types of resources, including subscription image databases, freely available digital libraries and collections, user-generated collections such as Fickr or Picasa, and the general Web. Academic libraries and librarians can serve the image needs of their users by providing access to online image resources and visual literacy instruction. This paper presents a research study that explored the types of image reference questions librarians receive, the resources they use most often, and the difficulties of searching for images online

Bibliographie

Précisions des auteurs : un nombre relativement important de nos références bibliographiques est désormais disponible dans les archives ouvertes, les pages de chercheurs dans les sites de leurs universités ou leurs sites personnels. Cependant, nous ne transcrivons pas l’adresse (URL) de ces publications en libre accès pour éviter d’allonger une bibliographie déjà bien fournie et nous invitons les lecteurs à les retrouver avec leur moteur de recherche favori, de préférence spécialisé dans les ..