A framework for evaluating automatic indexing or classification in the context of retrieval

Tools for automatic subject assignment help deal with scale and sustainability in creating and enriching metadata, establishing more connections across and between resources and enhancing consistency. While some software vendors and experimental researchers claim the tools can replace manual subject indexing, hard scientific evidence of their performance in operating information environments is scarce. A major reason for this is that research is usually conducted in laboratory conditions, excluding the complexities of real-life systems and situations. The paper reviews and discusses issues with existing evaluation approaches such as problems of aboutness and relevance assessments, implying the need to use more than a single “gold standard” method when evaluating indexing and retrieval and proposes a comprehensive evaluation framework. The framework is informed by a systematic review of the literature on indexing, classification and approaches: evaluating indexing quality directly through assessment by an evaluator or through comparison with a gold standard; evaluating the quality of computer-assisted indexing directly in the context of an indexing workflow, and evaluating indexing quality indirectly through analyzing retrieval performance

Latent profile analysis of accelerometer-measured sleep, physical activity, and sedentary time and differences in health characteristics in adult women.

ObjectivesIndependently, physical activity (PA), sedentary behavior (SB), and sleep are related to the development and progression of chronic diseases. Less is known about how rest-activity behaviors cluster within individuals and how rest-activity behavior profiles relate to health. In this study we aimed to investigate if adult women cluster into profiles based on how they accumulate rest-activity behavior (including accelerometer-measured PA, SB, and sleep), and if participant characteristics and health outcomes differ by profile membership.MethodsA convenience sample of 372 women (mean age 55.38 + 10.16) were recruited from four US cities. Participants wore ActiGraph GT3X+ accelerometers on the hip and wrist for a week. Total daily minutes in moderate-to-vigorous PA (MVPA) and percentage of wear-time spent in SB was estimated from the hip device. Total sleep time (hours/minutes) and sleep efficiency (% of in bed time asleep) were estimated from the wrist device. Latent profile analysis (LPA) was performed to identify clusters of participants based on accumulation of the four rest-activity variables. Adjusted ANOVAs were conducted to explore differences in demographic characteristics and health outcomes across profiles.ResultsRest-activity variables clustered to form five behavior profiles: Moderately Active Poor Sleepers (7%), Highly Actives (9%), Inactives (41%), Moderately Actives (28%), and Actives (15%). The Moderately Active Poor Sleepers (profile 1) had the lowest proportion of whites (35% vs 78-91%, p < .001) and college graduates (28% vs 68-90%, p = .004). Health outcomes did not vary significantly across all rest-activity profiles.ConclusionsIn this sample, women clustered within daily rest-activity behavior profiles. Identifying 24-hour behavior profiles can inform intervention population targets and innovative behavioral goals of multiple health behavior interventions

The history of information retrieval research

This paper describes a brief history of the research and development of information retrieval systems starting with the creation of electromechanical searching devices, through to the early adoption of computers to search for items that are relevant to a user's query. The advances achieved by information retrieval researchers from the 1950s through to the present day are detailed next, focusing on the process of locating relevant information. The paper closes with speculation on where the future of information retrieval lies

The Expectations for Faculty in Latin America

Faculty load in the US varies from 40-40-10 (research-teaching-service) in research oriented institutions to 25-50-25 in teaching oriented institutions. Consulting is normally not part of the formula and case writing is generally not considered as part of the research load. Latin America is quite different. A 25-25-25-25 load (that includes consulting as a separate area) is not atypical and research can often be accomplished through case writing. However, Latin America is not homogenous and some very specific institutions are exceptional even including few faculty positions that are 100% research. The panel will provide an overview of the expectations in different countries (particularly Brazil, Chile, Mexico, and Peru) and discuss how current expectations deal with international accrediting bodies that might be requiring different mixtures

A Time-Aware Approach to Improving Ad-hoc Information Retrieval from Microblogs

There is an immense number of short-text documents produced as the result of microblogging. The content produced is growing as the number of microbloggers grows, and as active microbloggers continue to post millions of updates. The range of topics discussed is so vast, that microblogs provide an abundance of useful information. In this work, the problem of retrieving the most relevant information in microblogs is addressed. Interesting temporal patterns were found in the initial analysis of the study. Therefore the focus of the current work is to first exploit a temporal variable in order to see how effectively it can be used to predict the relevance of the tweets and, then, to include it in a retrieval weighting model along with other tweet-specific features. Generalized Linear Mixed-effect Models (GLMMs) are used to analyze the features and to propose two re-ranking models. These two models were developed through an exploratory process on a training set and then were evaluated on a test set

Temporal Feedback for Tweet Search with Non-Parametric Density Estimation

This paper investigates the temporal cluster hypothesis: in search tasks where time plays an important role, do relevant documents tend to cluster together in time? We explore this question in the context of tweet search and temporal feedback: starting with an initial set of results from a baseline retrieval model, we estimate the temporal density of relevant documents, which is then used for result reranking. Our contributions lie in a method to characterize this temporal density function using kernel density estimation, with and without human relevance judgments, and an approach to integrating this information into a standard retrieval model. Experiments on TREC datasets confirm that our temporal feedback formulation improves search effectiveness, thus providing support for our hypothesis. Our approach outperforms both a standard baseline and previous temporal retrieval models. Temporal feedback improves over standard lexical feedback (with and without human judgments), illustrating that temporal relevance signals exist independently of document content

Profile: Our Place in the World

This issue of Profile speaks to the ways in which campus members are interacting with the world and illuminates our active participation in the 21st-century global community. It includes a story on the ways in which Morris students, faculty, and staff have reached out to and benefited from their interactions with the local and growing Latino community. It also features recent recipients of the David C. Johnson Award for International Service Learning. --From Chancellor Johnson\u27s Messagehttps://digitalcommons.morris.umn.edu/profile/1015/thumbnail.jp

Large expert-curated database for benchmarking document similarity detection in biomedical literature search

Document recommendation systems for locating relevant literature have mostly relied on methods developed a decade ago. This is largely due to the lack of a large offline gold-standard benchmark of relevant documents that cover a variety of research fields such that newly developed literature search techniques can be compared, improved and translated into practice. To overcome this bottleneck, we have established the RElevant LIterature SearcH consortium consisting of more than 1500 scientists from 84 countries, who have collectively annotated the relevance of over 180 000 PubMed-listed articles with regard to their respective seed (input) article/s. The majority of annotations were contributed by highly experienced, original authors of the seed articles. The collected data cover 76% of all unique PubMed Medical Subject Headings descriptors. No systematic biases were observed across different experience levels, research fields or time spent on annotations. More importantly, annotations of the same document pairs contributed by different scientists were highly concordant. We further show that the three representative baseline methods used to generate recommended articles for evaluation (Okapi Best Matching 25, Term Frequency-Inverse Document Frequency and PubMed Related Articles) had similar overall performances. Additionally, we found that these methods each tend to produce distinct collections of recommended articles, suggesting that a hybrid method may be required to completely capture all relevant articles. The established database server located at https://relishdb.ict.griffith.edu.au is freely available for the downloading of annotation data and the blind testing of new methods. We expect that this benchmark will be useful for stimulating the development of new powerful techniques for title and title/abstract-based search engines for relevant articles in biomedical science