Reasoning about Relevance

This study focuses on how non-expert assessors judge relevance guided by mental models of relevance developed and applied during the assessment process. Components of relevance models are identified as well as challenges and changes associated with their construction and use. Study participants evaluated the relevance of news articles with respect to an assigned search topic. They commented on their reasoning in assessing each article, challenges they experienced in determining relevance and changes in their ability to assess relevance over the course of the evaluation session. Content analysis of these comments revealed that relevance models are derived from participants' understandings of the search topic, the documents they viewed and the relationships between them. Relevance manifestations (topical, situational, cognitive) and criteria (information scope, specificity and detail) guide the development and application of the relevance models, which may also be influenced by situational, cognitive and motivational factors.Master of Science in Information Scienc

Selection Mechanisms Underlying High Impact Biomedical Research - A Qualitative Analysis and Causal Model

BACKGROUND: Although scientific innovation has been a long-standing topic of interest for historians, philosophers and cognitive scientists, few studies in biomedical research have examined from researchers' perspectives how high impact publications are developed and why they are consistently produced by a small group of researchers. Our objective was therefore to interview a group of researchers with a track record of high impact publications to explore what mechanism they believe contribute to the generation of high impact publications. METHODOLOGY/PRINCIPAL FINDINGS: Researchers were located in universities all over the globe and interviews were conducted by phone. All interviews were transcribed using standard qualitative methods. A Grounded Theory approach was used to code each transcript, later aggregating concept and categories into overarching explanation model. The model was then translated into a System Dynamics mathematical model to represent its structure and behavior. Five emerging themes were found in our study. First, researchers used heuristics or rules of thumb that came naturally to them. Second, these heuristics were reinforced by positive feedback from their peers and mentors. Third, good communication skills allowed researchers to provide feedback to their peers, thus closing a positive feedback loop. Fourth, researchers exhibited a number of psychological attributes such as curiosity or open-mindedness that constantly motivated them, even when faced with discouraging situations. Fifth, the system is dominated by randomness and serendipity and is far from a linear and predictable environment. Some researchers, however, took advantage of this randomness by incorporating mechanisms that would allow them to benefit from random findings. The aggregation of these themes into a policy model represented the overall expected behavior of publications and their impact achieved by high impact researchers. CONCLUSIONS: The proposed selection mechanism provides insights that can be translated into research coaching programs as well as research policy models to optimize the introduction of high impact research at a broad scale among institutional and governmental agencies

Ilona Johnson

20857 Prepared by: RTI Internationa