The Relative Effectiveness of Text and Images in Image Search Result Listings

This study was conducted to determine the best type of image surrogate to use within search result sets: Text, Image Preview, or Text + Image Preview. Users' performance and satisfaction with the three different image surrogates within search result sets were evaluated. Data was collected from 28 participants via a web-based system of questionnaires and logs of their interactions with result set presentations. Of the three image surrogate types, Image Preview and Text + Image Preview surrogates consistently outperformed Text surrogates on measures of the time required to make relevance judgments, the quality of those relevance judgments, perceived ease of use and perceived usefulness. While relevance judgment scoring with Image Preview and Text + Image Preview surrogates was identical, answers to the post-session questionnaire indicated that users may prefer the Text + Image Preview surrogate, as it was "liked best overall" by more people

Multi-modal surrogates for retrieving and making sense of videos: is synchronization between the multiple modalities optimal?

Video surrogates can help people quickly make sense of the content of a video before downloading or seeking more detailed information. Visual and audio features of a video are primary information carriers and might become important components of video retrieval and video sense-making. In the past decades, most research and development efforts on video surrogates have focused on visual features of the video, and comparatively little work has been done on audio surrogates and examining their pros and cons in aiding users' retrieval and sense-making of digital videos. Even less work has been done on multi-modal surrogates, where more than one modality are employed for consuming the surrogates, for example, the audio and visual modalities. This research examined the effectiveness of a number of multi-modal surrogates, and investigated whether synchronization between the audio and visual channels is optimal. A user study was conducted to evaluate six different surrogates on a set of six recognition and inference tasks to answer two main research questions: (1) How do automatically-generated multi-modal surrogates compare to manually-generated ones in video retrieval and video sense-making? and (2) Does synchronization between multiple surrogate channels enhance or inhibit video retrieval and video sense-making? Forty-eight participants participated in the study, in which the surrogates were measured on the the time participants spent on experiencing the surrogates, the time participants spent on doing the tasks, participants' performance accuracy on the tasks, participants' confidence in their task responses, and participants' subjective ratings on the surrogates. On average, the uncoordinated surrogates were more helpful than the coordinated ones, but the manually-generated surrogates were only more helpful than the automatically-generated ones in terms of task completion time. Participants' subjective ratings were more favorable for the coordinated surrogate C2 (Magic A + V) and the uncoordinated surrogate U1 (Magic A + Storyboard V) with respect to usefulness, usability, enjoyment, and engagement. The post-session questionnaire comments demonstrated participants' preference for the coordinated surrogates, but the comments also revealed the value of having uncoordinated sensory channels

Learning object metadata surrogates in search result interfaces: user evaluation, design and content

The purpose of this research was to evaluate user interaction with learning object metadata surrogates both in terms of content and presentation. The main objectives of this study were: (1) to review the literature on learning object metadata and user-centred evaluation of metadata surrogates in the context of cognitive information retrieval (including user-centred relevance and usability research); (2) to develop a framework for the evaluation of user interaction with learning object metadata surrogates in search result interfaces; (3) to investigate the usability of metadata surrogates in search result interfaces of learning object repositories (LORs) in terms of various presentation aspects (such as amount of information, structure and highlighting of query terms) as a means for facilitating the user relevance judgment process; (4) to investigate in-depth the type of content that should be included in learning object metadata surrogates in order to facilitate the process of relevance judgment; (5) to provide a set of recommendations—guidelines for the design of learning object metadata surrogates in search result interfaces both in terms of content and presentation. [Continues.

Spoken content retrieval: A survey of techniques and technologies

Speech media, that is, digital audio and video containing spoken content, has blossomed in recent years. Large collections are accruing on the Internet as well as in private and enterprise settings. This growth has motivated extensive research on techniques and technologies that facilitate reliable indexing and retrieval. Spoken content retrieval (SCR) requires the combination of audio and speech processing technologies with methods from information retrieval (IR). SCR research initially investigated planned speech structured in document-like units, but has subsequently shifted focus to more informal spoken content produced spontaneously, outside of the studio and in conversational settings. This survey provides an overview of the field of SCR encompassing component technologies, the relationship of SCR to text IR and automatic speech recognition and user interaction issues. It is aimed at researchers with backgrounds in speech technology or IR who are seeking deeper insight on how these fields are integrated to support research and development, thus addressing the core challenges of SCR

The Use of Social Tags in Text and Image Searching on the Web.

In recent years, tags have become a standard feature on a diverse range of sites on the Web, accompanying blog posts, photos, videos, and online news stories. Tags are descriptive terms attached to Internet resources. Despite the rapid adoption of tagging, how people use tags during the search process is not well understood. There is little empirical data on the use and perceptions of tags created by those other than the searcher. Previous research on tags focused on the motivations and behaviors of taggers, although non-taggers represent a larger proportion of Web users than taggers. This study examines how people use tags, created by others, during the search process. Forty-eight subjects were each assigned four search tasks in a within-subjects study. Subjects searched for text documents and images in a controlled laboratory setting, using information retrieval interfaces differing in their incorporation of tags. User behavior and perception data were collected through search logs and interviews. Both direct and indirect uses of tags across the search process were examined. Tags are used directly when they are clicked on, resulting in a new query, while tags are used indirectly when used for judgments of relevance or to obtain additional terms for query reformulation. Tags increased interactions with the information retrieval system, as subjects issued more queries and saw more search results when using the tagged interface. For both text and image searches, tags were used for query reformulation, predictive judgment, and evaluative judgment of relevance. Subjects interacted most frequently with tags on the search results page, using them for query reformulation and predictive judgment. Tags were more likely to be used for predictive judgment in text searches than in image searches. Subjects’ understanding of tags focused on the role of tags in search, especially findability through a search engine. Tags were not uniformly perceived as being user-generated; site owners and automatic generation were mentioned as sources of tags. Several implications for the design of search interfaces and presentation of tags to support information interactions are discussed in the conclusion.Ph.D.InformationUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/89816/1/kimym_1.pd

Relevance Criteria when Searching and Evaluating Online Video for Informational Use

Relevance is a core concept in the field of Information Science and a common term in everyday vernacular that generally refers to the usefulness of information. However, relevance has not been sufficiently or consistently defined or explored in the information science literature. Relevance criteria are the factors that information users employ when determining whether information they encounter is relevant. Identifying relevance criteria is a crucial step to understanding relevance. Relevance criteria employed with newer information formats like online video are especially important to study. Online video is now widespread, and people are increasingly likely to rely on video for information. This study identifies relevance criteria employed during relevance assessments of online video through a explanatory sequential mixed-methods study of frequent online video users including students, faculty, librarians, and video professionals. Methods included an online survey and interviews

Information needs and use of online video by educators: implications for the design of web resources

A case study of the nasa.ibiblio.org Web site was undertaken to deepen the current understanding we have of how educators search for and use online video information in classroom settings. Three methods were used to conduct a user needs assessment: transaction log analysis, online survey, and cognitive walkthrough interviews. The findings support previous research which found educators used videos with difficult topics and they used them to help students visualize concepts. In the study, visual surrogates of video were evaluated and educators suggested a series of revisions which would make them more interactive. Some educators were able to compensate for the system’s lack of educational standards metadata by applying their personal knowledge to the system. The major contributions of this dissertation are the introduction of the Web of Criteria, the recommendations for educational digital libraries and implications for the design of storyboard and posterframe visual surrogates. Findings from the current study and from previous research suggested social interactions educators have are important and need to be facilitated in digital environments. Digital library providers might view the interface as the point where they can intervene and lessen the barriers educators have to retrieving educational information

Journalistic image access : description, categorization and searching

The quantity of digital imagery continues to grow, creating a pressing need to develop efficient methods for organizing and retrieving images. Knowledge on user behavior in image description and search is required for creating effective and satisfying searching experiences. The nature of visual information and journalistic images creates challenges in representing and matching images with user needs. The goal of this dissertation was to understand the processes in journalistic image access (description, categorization, and searching), and the effects of contextual factors on preferred access points. These were studied using multiple data collection and analysis methods across several studies. Image attributes used to describe journalistic imagery were analyzed based on description tasks and compared to a typology developed through a meta-analysis of literature on image attributes. Journalistic image search processes and query types were analyzed through a field study and multimodal image retrieval experiment. Image categorization was studied via sorting experiments leading to a categorization model. Advances to research methods concerning search tasks and categorization procedures were implemented. Contextual effects on image access were found related to organizational contexts, work, and search tasks, as well as publication context. Image retrieval in a journalistic work context was contextual at the level of image needs and search process. While text queries, together with browsing, remained the key access mode to journalistic imagery, participants also used visual access modes in the experiment, constructing multimodal queries. Assigned search task type and searcher expertise had an effect on query modes utilized. Journalistic images were mostly described and queried for on the semantic level but also syntactic attributes were used. Constraining the description led to more abstract descriptions. Image similarity was evaluated mainly based on generic semantics. However, functionally oriented categories were also constructed, especially by domain experts. Availability of page context promoted thematic rather than object-based categorization. The findings increase our understanding of user behavior in image description, categorization, and searching, as well as have implications for future solutions in journalistic image access. The contexts of image production, use, and search merit more interest in research as these could be leveraged for supporting annotation and retrieval. Multiple access points should be created for journalistic images based on image content and function. Support for multimodal query formulation should also be offered. The contributions of this dissertation may be used to create evaluation criteria for journalistic image access systems

A Framework for Investigating Volunteered Geographic Information Relevance in Planning

Advances in information and communication technology and the ready availability of Global Positioning Systems (GPS) have made it possible for citizens to create information on the internet expressing their personal perceptions in the form of pictures, videos and text narratives associated with geographic locations. The term Volunteered Geographic information (VGI) was coined to describe the processes whereby non-professionals or “citizen scientists” participate directly in spatial data creation, editing and shared use. VGI offers promise as an innovative way for members of the public to participate directly in the use, production and sharing of spatial information that is relevant to issues of personal or community concern and as a means of addressing some of the issues associated with traditional public participation methods. Planners can find meaning in the heterogeneous, time-sensitive, geo-social geographic information created by citizen volunteers in a bottom-up participation process where planners give up some control over what data is collected and from whom. However, uncertainties associated with volunteered geographic information include relevance, credibility, representativeness and quality of the geographic information. This thesis investigates the opportunities and barriers to the use of volunteered geographic information as public participation in planning. A framework and methodology for collaborative quality control of VGI through multi-criteria subjective relevance ratings of the VGI by its producers and users is put forward in this thesis. The relevance rating framework for quality control of VGI is based on the use of relevance in information retrieval in information science to improve the relevance of search engine results. This concept is transferred to the quality control of VGI contributions to determine the best VGI contributions to be used in planning as public participation. A VGI web application prototype, including the subjective relevance rating system, was created and a methodology and demonstration of its use for public participation was presented

Credibility: A multidisciplinary framework

No Abstract.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/61241/1/1440410114_ftp.pd