Quantitative evaluation of recall and precision of CAT Crawler, a search engine specialized on retrieval of Critically Appraised Topics

BACKGROUND: Critically Appraised Topics (CATs) are a useful tool that helps physicians to make clinical decisions as the healthcare moves towards the practice of Evidence-Based Medicine (EBM). The fast growing World Wide Web has provided a place for physicians to share their appraised topics online, but an increasing amount of time is needed to find a particular topic within such a rich repository. METHODS: A web-based application, namely the CAT Crawler, was developed by Singapore's Bioinformatics Institute to allow physicians to adequately access available appraised topics on the Internet. A meta-search engine, as the core component of the application, finds relevant topics following keyword input. The primary objective of the work presented here is to evaluate the quantity and quality of search results obtained from the meta-search engine of the CAT Crawler by comparing them with those obtained from two individual CAT search engines. From the CAT libraries at these two sites, all possible keywords were extracted using a keyword extractor. Of those common to both libraries, ten were randomly chosen for evaluation. All ten were submitted to the two search engines individually, and through the meta-search engine of the CAT Crawler. Search results were evaluated for relevance both by medical amateurs and professionals, and the respective recall and precision were calculated. RESULTS: While achieving an identical recall, the meta-search engine showed a precision of 77.26% (±14.45) compared to the individual search engines' 52.65% (±12.0) (p < 0.001). CONCLUSION: The results demonstrate the validity of the CAT Crawler meta-search engine approach. The improved precision due to inherent filters underlines the practical usefulness of this tool for clinicians

Funding Source and Research Report Quality in Nutrition Practice-Related Research

BACKGROUND: The source of funding is one of many possible causes of bias in scientific research. One method of detecting potential for bias is to evaluate the quality of research reports. Research exploring the relationship between funding source and nutrition-related research report quality is limited and in other disciplines the findings are mixed. OBJECTIVE: The purpose of this study is to determine whether types of funding sources of nutrition research are associated with differences in research report quality. DESIGN: A retrospective study of research reporting quality, research design and funding source was conducted on 2539 peer reviewed research articles from the American Dietetic Association's Evidence Analysis Library® database. RESULTS: Quality rating frequency distributions indicate 43.3% of research reports were rated as positive, 50.1% neutral, and 6.6% as negative. Multinomial logistic regression results showed that while both funding source and type of research design are significant predictors of quality ratings (χ2 = 118.99, p≤0.001), the model's usefulness in predicting overall research report quality is little better than chance. Compared to research reports with government funding, those not acknowledging any funding sources, followed by studies with University/hospital funding were more likely to receive neutral vs positive quality ratings, OR = 1.85, P <0.001 and OR = 1.54, P<0.001, respectively and those that did not report funding were more likely to receive negative quality ratings (OR = 4.97, P<0.001). After controlling for research design, industry funded research reports were no more likely to receive a neutral or negative quality rating than those funded by government sources. CONCLUSION: Research report quality cannot be accurately predicted from the funding source after controlling for research design. Continued vigilance to evaluate the quality of all research regardless of the funding source and to further understand other factors that affect quality ratings are warranted