A study of factors affecting the utility of implicit relevance feedback

Implicit relevance feedback (IRF) is the process by which a search system unobtrusively gathers evidence on searcher interests from their interaction with the system. IRF is a new method of gathering information on user interest and, if IRF is to be used in operational IR systems, it is important to establish when it performs well and when it performs poorly. In this paper we investigate how the use and effectiveness of IRF is affected by three factors: search task complexity, the search experience of the user and the stage in the search. Our findings suggest that all three of these factors contribute to the utility of IRF

CHORUS Deliverable 2.2: Second report - identification of multi-disciplinary key issues for gap analysis toward EU multimedia search engines roadmap

After addressing the state-of-the-art during the first year of Chorus and establishing the existing landscape in multimedia search engines, we have identified and analyzed gaps within European research effort during our second year. In this period we focused on three directions, notably technological issues, user-centred issues and use-cases and socio- economic and legal aspects. These were assessed by two central studies: firstly, a concerted vision of functional breakdown of generic multimedia search engine, and secondly, a representative use-cases descriptions with the related discussion on requirement for technological challenges. Both studies have been carried out in cooperation and consultation with the community at large through EC concertation meetings (multimedia search engines cluster), several meetings with our Think-Tank, presentations in international conferences, and surveys addressed to EU projects coordinators as well as National initiatives coordinators. Based on the obtained feedback we identified two types of gaps, namely core technological gaps that involve research challenges, and “enablers”, which are not necessarily technical research challenges, but have impact on innovation progress. New socio-economic trends are presented as well as emerging legal challenges

Slicing and dicing the information space using local contexts

In recent years there has been growing interest in faceted grouping of documents for Interactive Information Retrieval (IIR). It is suggested that faceted grouping can offer a flexible way of browsing a collection compared to clustering. However, the success of faceted grouping seems to rely on sufficient knowledge of collection structure. In this paper we propose an approach based on the local contexts of query terms, which is inspired by the interaction of faceted search and browsing. The use of local contexts is appealing since it requires less knowledge of the collection than existing approaches. A task-based user study was carried out to investigate the effectiveness of our interface in varied complexity. The results suggest that the local contexts can be exploited as the source of search result browsing in IIR, and that our interface appears to facilitate different aspects of search process over the task complexity. The implication of the evaluation methodology using high complexity tasks is also discussed

Gender and participation in mathematics and further mathematics A-levels: a literature review for the Further Mathematics Support Programme

In preparing this report I have considered evidence from over 60 documents that relate to raising girls’ participation in mathematics. These include published research papers and reports compiled by expert bodies that present an evidence base. Although research specifically addressing Further Mathematics A-level is rare, the last ten years have seen considerable efforts to synthesise and update knowledge from different research perspectives about the relationship between gender and participation. For this reason, the review process started with papers from 2008 onwards

Using the CIPP Model of Evaluation on a Health Disparities Curriculum Delivered to Medical Residents in 12 Residency Programs

Background: Although the Accreditation Council for Graduate Medical Education (ACGME) has provided educational directives on health disparities education during residency program for the medical residents, there is paucity of curricula focused on disparities education within the graduate medical education. Objective: The purpose of this study was to evaluate a Health Disparities Curriculum (HDC) that was delivered to twelve residency programs at a College of Medicine. Methodology: The study employed a cross-sectional study design. Stufflebeam’s Context-Input-Process-Product (CIPP) model was utilized as theoretical framework to assess whether the curriculum achieved its stated goals and objectives, the teaching and learning processes, and the outcomes of the health disparities curriculum through the feedback received from medical residents who participated in the electronic and paper-based surveys. Results: The response rate for the study was 46.68%. Context evaluation showed that there was agreement among the residents on the curriculum meeting its stated goals and objectives. In input evaluation, the residents expressed the need for community engagement opportunities, interactive content, and actional strategies. Time constraints were identified as one of the barriers to attending the sessions and for curricular incorporation. A unique attitudinal barrier was identified where residents perceived lack of control in addressing disparities. Findings of process evaluation indicated statistically significant difference in the proportion of residents attending the sessions by race, specialty, and year in the residency program. The medical residents reported favorable perceptions on the relevance of content and the instructor’s competence. Residents from the non-primary care specialties reviewed the learning resources statistically significantly more as compared to the residents in the primary care specialties. Findings of product evaluation showed statistically significant increase in self-efficacy after curriculum participation, increased awareness, statistically significant increase in disparities discussions and community partnerships, and favorable perceptions on curricular utility. There was no difference in inclination to incorporate cultural humility among residents and no change in scholarly activity development. INDEX WORDS: Graduate Medical Education, Health Disparities Curriculum, Context-Input-Process-Product, CIPP, Evaluatio

Expanding the social-cognitive framework: understanding the role of implicit person theories in a complex task feedback environment

Department Head: Ernest L. Chavez.2010 Spring.Includes bibliographical references (pages 139-155).The current study employed a longitudinal design to examine the effects of implicit person theories (IPTs; Dweck & Leggett, 1988) on task performance over two measurement occasions and a week of temporal separation. This design allowed for a direct inquiry into the pivotal role that one's lay beliefs about the malleability of attributes play in a simulated task feedback environment. In addition to examining the direct effects of IPT on task performance, the study investigated the role of the mediated mechanisms of appraisal effectiveness (i.e., feedback reactions), goal setting, effort, and attributions in the IPT-task performance relationship. Further, the study investigated the conditional indirect effect of feedback sign (i.e., positive vs. negative) on the meditational mechanisms, otherwise known as moderated mediation. 242 psychology students participated in the study for course credit. Limited evidence for the proposed meditational models was provided. Overall, IPTs significantly positively predicted task performance following the receipt of feedback after a one-week lag in measurement. Furthermore, the sign of the feedback was a significant, positive predictor of the full class of feedback reactions. Notably, the full class of reactions to feedback were all positively related to task 2 performance, providing empirical evidence of the predictive validity of appraisal feedback reactions for subsequent performance. Implications of the results, future research directions, and limitations are discussed

Design implications for task-specific search utilities for retrieval and re-engineering of code

The importance of information retrieval systems is unquestionable in the modern society and both individuals as well as enterprises recognise the benefits of being able to find information effectively. Current code-focused information retrieval systems such as Google Code Search, Codeplex or Koders produce results based on specific keywords. However, these systems do not take into account developers’ context such as development language, technology framework, goal of the project, project complexity and developer’s domain expertise. They also impose additional cognitive burden on users in switching between different interfaces and clicking through to find the relevant code. Hence, they are not used by software developers. In this paper, we discuss how software engineers interact with information and general-purpose information retrieval systems (e.g. Google, Yahoo!) and investigate to what extent domain-specific search and recommendation utilities can be developed in order to support their work-related activities. In order to investigate this, we conducted a user study and found that software engineers followed many identifiable and repeatable work tasks and behaviours. These behaviours can be used to develop implicit relevance feedback-based systems based on the observed retention actions. Moreover, we discuss the implications for the development of task-specific search and collaborative recommendation utilities embedded with the Google standard search engine and Microsoft IntelliSense for retrieval and re-engineering of code. Based on implicit relevance feedback, we have implemented a prototype of the proposed collaborative recommendation system, which was evaluated in a controlled environment simulating the real-world situation of professional software engineers. The evaluation has achieved promising initial results on the precision and recall performance of the system