Optimal search strategies for identifying sound clinical prediction studies in EMBASE

BACKGROUND: Clinical prediction guides assist clinicians by pointing to specific elements of the patient's clinical presentation that should be considered when forming a diagnosis, prognosis or judgment regarding treatment outcome. The numbers of validated clinical prediction guides are growing in the medical literature, but their retrieval from large biomedical databases remains problematic and this presents a barrier to their uptake in medical practice. We undertook the systematic development of search strategies ("hedges") for retrieval of empirically tested clinical prediction guides from EMBASE. METHODS: An analytic survey was conducted, testing the retrieval performance of search strategies run in EMBASE against the gold standard of hand searching, using a sample of all 27,769 articles identified in 55 journals for the 2000 publishing year. All articles were categorized as original studies, review articles, general papers, or case reports. The original and review articles were then tagged as 'pass' or 'fail' for methodologic rigor in the areas of clinical prediction guides and other clinical topics. Search terms that depicted clinical prediction guides were selected from a pool of index terms and text words gathered in house and through request to clinicians, librarians and professional searchers. A total of 36,232 search strategies composed of single and multiple term phrases were trialed for retrieval of clinical prediction studies. The sensitivity, specificity, precision, and accuracy of search strategies were calculated to identify which were the best. RESULTS: 163 clinical prediction studies were identified, of which 69 (42.3%) passed criteria for scientific merit. A 3-term strategy optimized sensitivity at 91.3% and specificity at 90.2%. Higher sensitivity (97.1%) was reached with a different 3-term strategy, but with a 16% drop in specificity. The best measure of specificity (98.8%) was found in a 2-term strategy, but with a considerable fall in sensitivity to 60.9%. All single term strategies performed less well than 2- and 3-term strategies. CONCLUSION: The retrieval of sound clinical prediction studies from EMBASE is supported by several search strategies

Developing search strategies for clinical practice guidelines in SUMSearch and Google Scholar and assessing their retrieval performance

<p>Abstract</p> <p>Background</p> <p>Information overload, increasing time constraints, and inappropriate search strategies complicate the detection of clinical practice guidelines (CPGs). The aim of this study was to provide clinicians with recommendations for search strategies to efficiently identify relevant CPGs in SUMSearch and Google Scholar.</p> <p>Methods</p> <p>We compared the retrieval efficiency (retrieval performance) of search strategies to identify CPGs in SUMSearch and Google Scholar. For this purpose, a two-term GLAD (GuideLine And Disease) strategy was developed, combining a defined CPG term with a specific disease term (MeSH term). We used three different CPG terms and nine MeSH terms for nine selected diseases to identify the most efficient GLAD strategy for each search engine. The retrievals for the nine diseases were pooled. To compare GLAD strategies, we used a manual review of all retrievals as a reference standard. The CPGs detected had to fulfil predefined criteria, e.g., the inclusion of therapeutic recommendations. Retrieval performance was evaluated by calculating so-called diagnostic parameters (sensitivity, specificity, and "Number Needed to Read" [NNR]) for search strategies.</p> <p>Results</p> <p>The search yielded a total of 2830 retrievals; 987 (34.9%) in Google Scholar and 1843 (65.1%) in SUMSearch. Altogether, we found 119 unique and relevant guidelines for nine diseases (reference standard). Overall, the GLAD strategies showed a better retrieval performance in SUMSearch than in Google Scholar. The performance pattern between search engines was similar: search strategies including the term "guideline" yielded the highest sensitivity (SUMSearch: 81.5%; Google Scholar: 31.9%), and search strategies including the term "practice guideline" yielded the highest specificity (SUMSearch: 89.5%; Google Scholar: 95.7%), and the lowest NNR (SUMSearch: 7.0; Google Scholar: 9.3).</p> <p>Conclusion</p> <p>SUMSearch is a useful tool to swiftly gain an overview of available CPGs. Its retrieval performance is superior to that of Google Scholar, where a search is more time consuming, as substantially more retrievals have to be reviewed to detect one relevant CPG. In both search engines, the CPG term "guideline" should be used to obtain a comprehensive overview of CPGs, and the term "practice guideline" should be used if a less time consuming approach for the detection of CPGs is desired.</p

A generalised model for individualising a treatment recommendation based on group-level evidence from randomised clinical trials

Objectives: Randomised controlled trials report group-level treatment effects. However, an individual patient confronting a treatment decision needs to know whether that person's expected treatment benefit will exceed the expected treatment harm. We describe a flexible model for individualising a treatment decision. It individualises group-level results from randomised trials using clinical prediction guides. Methods: We constructed models that estimate the size of individualised absolute risk reduction (ARR) for the target outcome that is required to offset individualised absolute risk increase (ARI) for the treatment harm. Inputs to the model include estimates for the individualised predicted absolute treatment benefit and harm, and the relative value assigned by the patient to harm/benefit. A decision rule recommends treatment when the predicted benefit exceeds the predicted harm, value-adjusted. We also derived expressions for the maximum treatment harm, or the maximum relative value for harm/benefit, above which treatment would not be recommended. Results: For the simpler model, including one kind of benefit and one kind of harm, the individualised ARR required to justify treatment was expressed as required ARRtarget(i)=ARIharm(i) 7 RVharm/target(i). A complex model was also developed, applicable to treatments causing multiple kinds of benefits and/or harms. We demonstrated the applicability of the models to treatments tested in superiority trials (either placebo or active control, either fixed harm or variable harm) and non-inferiority trials. Conclusions: Individualised treatment recommendations can be derived using a model that applies clinical prediction guides to the results of randomised trials in order to identify which individual patients are likely to derive a clinically important benefit from the treatment. The resulting individualised prediction-based recommendations require validation by comparison with strategies of treat all or treat none

Desarrollo de un meta-modelo de predicción para el diagnóstico de infección cervical por Chlamydia trachomatis en mujeres de Colombia

Introducción: La cervicitis por clamidia tiene alta prevalencia, es de difícil diagnóstico (30-75% de los casos son asintomáticos) y pueden tener serias secuelas clínicas. En condiciones de escasos recursos, el diagnóstico y tratamiento principalmente son realizadas bajo el manejo sindrómico propuesto por la OMS, el cual ha mostrado desempeño subóptimo. Se desarrolló y validó un meta-modelo basado sobre una revisión sistemática de modelos predictivos publicados, Metodología: Se condujo una revisión sistemática de reglas de predicción clínica para el diagnóstico de infección de cervicitis por Chlamydia trachomatis, buscando en las bases de datos Medline, EMBASE y Bireme. La revisión, selección, extracción de datos y evaluación de la calidad fueron realizadas independientemente por dos investigadores, usando formas estandarizadas y nuevas herramientas específicas para riesgo de sesgo en modelos predictivos. Las características y las medidas de desempeño de los modelos incluidos fueron comparadas y resumidas. Adicionalmente, fueron agregados usando dos métodos disponibles (modelo promediado y regresión de Stacked) y una muestra externa de 1381 mujeres de Colombia con una prevalencia del 9.7% de casos diagnosticados con PCR. La máxima verosimilitud penalizada fue usada para corregir el sobreajustamiento de los coeficientes del meta-modelo obtenidos por la regresión de Stacked y bootstrapping fue usada para realizar la validación interna. Resultados: De 3665 registros identificados, 25 artículos fueron incluidos en la revisión sistemática, en donde se reportaron 31 reglas de predicción diagnóstica. Se encontró que los principales riesgos de sesgos están relacionados con selección de los participantes, tamaño de la muestra y análisis de los datos. Siete modelos fueron agregados. El método de la regresión de Stacked generó un meta-modelo con mejores características sobre los modelos existentes: Área bajo la curva 0.79, puntaje de Brier 0.077 y R2 0.20. La validación interna mostró escaso sobreajustamiento. La comparación de las razones de verosimilitud positiva y negativa fueron mostraron que el meta-modelo tubo mejor desempeño que el actual algoritmo de la OMS. Conclusiones: La regresión de Stacked permitió agregar modelos de predicción, a través datos de pacientes externos individuales. El meta-modelo desarrollado demostró mejores características operativas que el algoritmo de la OMS, sin embargo es necesario realizar estudios de impacto para evaluar su utilidad clínica.Abstract. Introduction: Cervicitis by chlamydia has high prevalence, is difficult to diagnose (30-75% asymptomatic cases) and might have serious clinical sequelae. In resource scarce settings, diagnosis and treatment are based on the WHO’s proposed syndromic approach, which has largely suboptimal performance. We developed and validated a meta-model based on a systematic review of published prediction models. Method: We conducted a systematic review on clinical prediction rules (CPRs) for the diagnosis of cervicitis infection by Chlamydia trachomatis, searching Medline, EMBASE and Bireme databases. Screening, data extraction and quality assessment were performed independently by two researchers, using standardized forms and a new specific risk of bias tool. The performance characteristics of the included models were summarized, and the models were aggregated using two available methods (model averaging and stacked regression method). and an external sample of 1381 women from Colombia (10% positive cases at PCR). Penalized maximum likelihood was used to correct for overfitting the coefficients of the meta-model obtained by the stacked regression, and bootstrapping was used for further internal validation. Results: Of 3665 records identified, 25 articles reporting on 31 different CPRs were included in the systematic review. The principle risk of bias was related to selection of participants, sample size and data analysis. Seven models were aggregated. The stacked regression method generated a meta-model with improved characteristics over each of the existing models: c-statistic 0.79, Brier score 0.077 and R2 0.20. Internal validation showed lack of overfitting. Comparison of likelihood ratios for positive and negative tests showed that the meta-model had better operational characteristics than the currently used WHO score. Conclusions: Stacked regression allows aggregation of prediction models, provided that suitable external individual patient data are available. The meta-model demonstrated better performance characteristics that the WHO algorithm, however, a prospective implementation study is needed in order to assess of clinical use.Maestrí

Revisión sistemática y metanálisis del rendimiento diagnóstico de los hallazgos ecográficos componentes del sonograma genético realizado en el segundo trimestre de gestación para la detección de Síndrome de Down

El impacto que tienen los defectos congénitos sobre la salud de las personas afectadas, sus familias y sobre la sociedad en su conjunto es muy considerable. En España se estima que existe una prevalencia de anomalías cromosómicas hasta 2010 de 1,49% de los recién nacidos vivos. El Síndrome de Down (SD) es la tercera causa de defecto congénito y la primera de cromosomopatía, con una prevalencia de 23 por cada 10000 nacidos vivos. La importante morbilidad asociada en los individuos con síndrome de Down se acompaña de un alto coste económico, estimándose en 329750,63 euros por cada nuevo caso, constituyendo un cargo a lo largo de la vida de 1316 millones de euros. Por todo ello, la detección de esta alteración es la indicación más frecuente de diagnóstico prenatal invasivo. Las pruebas invasivas, como amniocentesis, biopsia de vellosidades coriales y cordocentesis, se asocian con un aumento del 1% del riesgo de aborto y, por tanto, sólo se realizan cuando se considera que hay una probabilidad elevada de que el feto tenga un defecto cromosómico, siendo por tanto fundamental una buena selección del grupo de gestantes de alto riesgo..

Improving literature searching in systematic reviews: the application of tailored literature searching compared to ‘the conventional approach’

Background Literature searching is acknowledged as a crucial step in a systematic review. Information professionals, in response to the needs of intervention effectiveness systematic reviews, have developed a systematic process of literature searching which aims to be comprehensive, transparent and reproducible, and to minimise the introduction of bias in systematic reviews. The process which has evolved has not been examined in detail before but it has been adopted as the principal approach to literature searching in other types of systematic review. It is not clear if this is appropriate and if an alternative approach might be more effective. Thesis aims The aims of this thesis are to: 1) examine approaches to systematic literature searching for systematic reviews; and 2) propose and test a method of systematic literature searching for reviews which do not focus on the effectiveness of clinical interventions. Methods Two literature reviews, one systematic review and two comparative case studies were undertaken to meet the aims of the thesis. Results A critical literature review identified and described a conventional approach to literature searching common to nine leading handbooks of systematic review. An alternative, tailored approach to literature searching was developed. Two case studies illustrated that the tailored approach was more effective, and potentially offered better value, than the conventional approach. Conclusions Information professionals can develop tailored literature search approaches for use in systematic reviews and as a useful alternative to the conventional approach, particularly for reviews including study designs beyond controlled trials. The role of the information professional as decision maker, the involvement of the research team and experts, preparing for literature searching and the use of supplementary search methods, are important to the success of tailored literature search approaches