Learning object metadata surrogates in search result interfaces: user evaluation, design and content

The purpose of this research was to evaluate user interaction with learning object metadata surrogates both in terms of content and presentation. The main objectives of this study were: (1) to review the literature on learning object metadata and user-centred evaluation of metadata surrogates in the context of cognitive information retrieval (including user-centred relevance and usability research); (2) to develop a framework for the evaluation of user interaction with learning object metadata surrogates in search result interfaces; (3) to investigate the usability of metadata surrogates in search result interfaces of learning object repositories (LORs) in terms of various presentation aspects (such as amount of information, structure and highlighting of query terms) as a means for facilitating the user relevance judgment process; (4) to investigate in-depth the type of content that should be included in learning object metadata surrogates in order to facilitate the process of relevance judgment; (5) to provide a set of recommendations—guidelines for the design of learning object metadata surrogates in search result interfaces both in terms of content and presentation. [Continues.

Information retrieval from civil engineering repositories: the importance of context and granularity

Information about the design and construction of buildings can be structured in a particular way. This is especially correct given the increasing complexity of building product models and the emergence of building information models with project documents linked to them. In addition, engineers usually have distinct information needs. Research shows that engineers working with building information models place particular importance on the understanding of retrieved content before using it or applying it and that exploration of context is essential for this understanding. Both these factors (the nature of engineering content and the information needs of engineers) make general information retrieval techniques for computing relevance and visualizing search results less applicable in civil engineering information retrieval systems. This paper argues that granularity is a fundamental concept that needs to be considered when measuring relevance and visualizing search results in information retrieval systems for repositories of building design and construction content. It is hypothesized that the design of systems with careful regard for granularity would improve engineers’ relevance judgment behavior. To test this hypothesis, a prototype system, called CoMem-XML, was developed and evaluated in terms of the time needed for users to find relevant information, the accuracy of their relevance judgment, and their subjective satisfaction with the prototype. A user study was conducted in which test subjects were asked to complete tasks by using various forms of the prototype, to complete a satisfaction questionnaire, and to be interviewed. The findings show that users perform better and are more satisfied when the search result interface of the CoMem-XML system presents only relevant information in context. On the other hand, interfaces that present the retrieved information out of context (i.e., without highlighting its position in the parts hierarchy) are less effective for participants to judge relevance

A study of students’ information searching strategies

Concerns have been expressed with respect to students’ ability to search for information using electronic search engines and databases. This research adopted a structured method comprising a combination of questionnaire surveys, an observational study and a ‘sense making’ interview to assess the information searching skills of a group of 14 students undertaking their final year dissertation studies on undergraduate programmes within the Department of Civil and Building Engineering at Loughborough University. The findings reveal that the participants encountered problems with each type of search engine used (Google, Metalib, the Library OPAC system, and individual databases) and lacked knowledge of how to use advanced search strategies. All the participants formulated queries using simple words or free text and there was no evidence of consideration of structured word searching using systematically selected keywords. The results indicate priority areas for additional tuition in information literacy

Design factors affecting relevance judgment behaviour in the context of metadata surrogates

This paper reports on the findings of a user study which investigated the effects of structure, highlighting and quantity of information on university students’ interaction with metadata surrogates of learning object repositories (LORs) during the relevance judgement process. The literature review revealed a lack of studies investigating how metadata surrogates should be designed to meet users’ needs and improve the effectiveness of relevance judgement. In order to address this issue, different versions of a prototype system, called METALOR 2, were developed and evaluated in terms of the time needed for users to find relevant information, the accuracy of their relevance judgements and their satisfaction. To evaluate the prototype, a user study was conducted where participants were asked to complete a set of tasks and fill in satisfaction questionnaires. The findings showed that participants performed better and were more satisfied with those versions of the prototype which highlighted metadata elements that included the query terms, used clusters or categories to organize metadata elements in the surrogates and included metadata elements which were relevant to the query or task at hand. The paper concludes with some recommendations for improving the design of metadata surrogates in search result interfaces

Additional file 1: of Presentation of laboratory test results in patient portals: influence of interface design on risk interpretation and visual search behaviour

Figures S1-S9 that show details of the three Presentations, highlighting the three areas of interest under study; Table S1 that reports a contingency table for the number of patients who underestimated and overestimated the need for action; Tables S2 and S3 that show the characteristics of patients who underestimated and overestimated the need for actions and those who did not; Figure S10 that shows participant’s dwell time on the different areas of interest across the different presentations and clinical scenarios, stratified by patients who underestimated the need for action and those who did not. (DOCX 2455 kb