Chapter 1: Introduction to the Methods Guide for Medical Test Reviews

Evaluation of medical tests presents challenges distinct from those involved in the evaluation of therapies; in particular, the very great importance of context and the dearth of comprehensive RCTs aimed at comparing the clinical outcomes of different tests and test strategies. Available guidance provides some suggestions: 1) Use of the PICOTS typology for clarifying the context relevant to the review, and 2) use of an organizing framework for classifying the types of medical test evaluation studies and their relationship to potential key questions. However, there is a diversity of recommendations for reviewers of medical tests and a proliferation of concepts, terms, and methods. As a contribution to the field, this Methods Guide for Medical Test Reviews seeks to provide practical guidance for achieving the goals of clarity, consistency, tractability, and usefulness

Approaches to the evaluation of outbreak detection methods

BACKGROUND: An increasing number of methods are being developed for the early detection of infectious disease outbreaks which could be naturally occurring or as a result of bioterrorism; however, no standardised framework for examining the usefulness of various outbreak detection methods exists. To promote comparability between studies, it is essential that standardised methods are developed for the evaluation of outbreak detection methods. METHODS: This analysis aims to review approaches used to evaluate outbreak detection methods and provide a conceptual framework upon which recommendations for standardised evaluation methods can be based. We reviewed the recently published literature for reports which evaluated methods for the detection of infectious disease outbreaks in public health surveillance data. Evaluation methods identified in the recent literature were categorised according to the presence of common features to provide a conceptual basis within which to understand current approaches to evaluation. RESULTS: There was considerable variation in the approaches used for the evaluation of methods for the detection of outbreaks in public health surveillance data, and appeared to be no single approach of choice. Four main approaches were used to evaluate performance, and these were labelled the Descriptive, Derived, Epidemiological and Simulation approaches. Based on the approaches identified, we propose a basic framework for evaluation and recommend the use of multiple approaches to evaluation to enable a comprehensive and contextualised description of outbreak detection performance. CONCLUSION: The varied nature of performance evaluation demonstrated in this review supports the need for further development of evaluation methods to improve comparability between studies. Our findings indicate that no single approach can fulfil all evaluation requirements. We propose that the cornerstone approaches to evaluation identified provide key contributions to support internal and external validity and comparability of study findings, and suggest these be incorporated into future recommendations for performance assessment

The stroke oxygen pilot study: a randomized control trial of the effects of routine oxygen supplementation early after acute stroke--effect on key outcomes at six months

Introduction: Post-stroke hypoxia is common, and may adversely affect outcome. We have recently shown that oxygen supplementation may improve early neurological recovery. Here, we report the six-month outcomes of this pilot study. Methods: Patients with a clinical diagnosis of acute stroke were randomized within 24 h of admission to oxygen supplementation at 2 or 3 L/min for 72 h or to control treatment (room air). Outcomes (see below) were assessed by postal questionnaire at 6 months. Analysis was by intention-to-treat, and statistical significance was set at p#0.05. Results: Out of 301 patients randomized two refused/withdrew consent and 289 (148 in the oxygen and 141 in the control group) were included in the analysis: males 44%, 51%; mean (SD) age 73 (12), 71 (12); median (IQR) National Institutes of Health Stroke Scale score 6 (3, 10), 5 (3, 10) for the two groups respectively. At six months 22 (15%) patients in the oxygen group and 20 (14%) in the control group had died; mean survival in both groups was 162 days (p= 0.99). Median (IQR) scores for the primary outcome, the modified Rankin Scale, were 3 (1, 5) and 3 (1, 4) for the oxygen and control groups respectively. The covariate-adjusted odds ratio was 1.04 (95% CI 0.67, 1.60), indicating that the odds of a lower (i.e. better) score were non-significantly higher in the oxygen group (p= 0.86). The mean differences in the ability to perform basic (Barthel Index) and extended activities of daily living (NEADL), and quality of life (EuroQol) were also non-significant. Conclusions: None of the key outcomes differed at 6 months between the groups. Although not statistically significant and generally of small magnitude, the effects were predominantly in favour of the oxygen group; a larger trial, powered to show differences in longer-term functional outcomes, is now on-going. Trial Registration: Controlled-Trials.com ISRCTN12362720; Eudract.ema.europa.eu 2004-001866-4

Integrating the promotion of physical activity within a smoking cessation programme: Findings from collaborative action research in UK Stop Smoking Services

Background: Within the framework of collaborative action research, the aim was to explore the feasibility of developing and embedding physical activity promotion as a smoking cessation aid within UK 6/7-week National Health Service (NHS) Stop Smoking Services. Methods: In Phase 1 three initial cycles of collaborative action research (observation, reflection, planning, implementation and re-evaluation), in an urban Stop Smoking Service, led to the development of an integrated intervention in which physical activity was promoted as a cessation aid, with the support of a theoretically based self-help guide, and self monitoring using pedometers. In Phase 2 advisors underwent training and offered the intervention, and changes in physical activity promoting behaviour and beliefs were monitored. Also, changes in clients’ stage of readiness to use physical activity as a cessation aid, physical activity beliefs and behaviour and physical activity levels were assessed, among those who attended the clinic at 4-week post-quit. Qualitative data were collected, in the form of clinic observation, informal interviews with advisors and field notes. Results: The integrated intervention emerged through cycles of collaboration as something quite different to previous practice. Based on field notes, there were many positive elements associated with the integrated intervention in Phase 2. Self-reported advisors’ physical activity promoting behaviour increased as a result of training and adapting to the intervention. There was a significant advancement in clients’ stage of readiness to use physical activity as a smoking cessation aid. Conclusions: Collaboration with advisors was key in ensuring that a feasible intervention was developed as an aid to smoking cessation. There is scope to further develop tailored support to increasing physical activity and smoking cessation, mediated through changes in perceptions about the benefits of, and confidence to do physical activity

Pediatric resident and faculty attitudes toward self-assessment and self-directed learning: a cross-sectional study

<p>Abstract</p> <p>Background</p> <p>The development of self-assessment and self-directed learning skills is essential to lifelong learning and becoming an effective physician. Pediatric residents in the United States are now required to use Individualized Learning Plans (ILPs) to document self-assessment and self-directed learning. A better understanding of resident and faculty attitudes and skills about self-assessment and self-directed learning will allow more successful integration of lifelong learning into residency education. The objective of this study was to compare faculty and resident attitudes, knowledge and skills about self-assessment, self-directed learning and ILPs.</p> <p>Methods</p> <p>Survey of pediatric residents and faculty at a single institution. Respondents rated their attitudes, knowledge, and self-perceived skills surrounding self-assessment, self-directed learning and ILPs.</p> <p>Results</p> <p>Overall survey response rate was 81% (79/97); 100% (36/36) residents and 70% (43/61) faculty. Residents and faculty agreed that lifelong learning is a necessary part of being a physician. Both groups were comfortable with assessing their own strengths and weaknesses and developing specific goals to improve their own performance. However, residents were less likely than faculty to continuously assess their own performance (44% vs. 81%; p < 0.001) or continuously direct their own learning (53% vs. 86%; p < 0.001). Residents were more likely than faculty to believe that residents should be primarily responsible for directing their own learning (64% vs. 19%; p < 0.0001), but at the same time, more residents believed that assigned clinical (31% vs. 0%; p < 0.0001) or curricular (31% vs. 0%; p < 0.0001) experiences were sufficient to make them competent physicians. Interns were less likely than senior residents to have a good understanding of how to assess their own skills (8% vs. 58%; p = 0.004) or what it means to be a self-directed learner (50% vs. 83%; p = 0.04).</p> <p>Qualitative comments indicated that while ILPs have the potential to help learners develop individualized, goal-directed learning plans based on strengths and weaknesses, successful implementation will require dedicated time and resident and faculty development.</p> <p>Conclusion</p> <p>These findings suggest that training and experience are necessary for physicians to understand the role of self-directed learning in education. Deliberate practice, for example by requiring residents to use ILPs, may facilitate self-directed, lifelong learning.</p

Accuracy of syndrome definitions based on diagnoses in physician claims

<p>Abstract</p> <p>Background</p> <p>Community clinics offer potential for timelier outbreak detection and monitoring than emergency departments. However, the accuracy of syndrome definitions used in surveillance has never been evaluated in community settings. This study's objective was to assess the accuracy of syndrome definitions based on diagnostic codes in physician claims for identifying 5 syndromes (fever, gastrointestinal, neurological, rash, and respiratory including influenza-like illness) in community clinics.</p> <p>Methods</p> <p>We selected a random sample of 3,600 community-based primary care physicians who practiced in the fee-for-service system in the province of Quebec, Canada in 2005-2007. We randomly selected 10 visits per physician from their claims, stratifying on syndrome type and presence, diagnosis, and month. Double-blinded chart reviews were conducted by telephone with consenting physicians to obtain information on patient diagnoses for each sampled visit. The sensitivity, specificity, and positive predictive value (PPV) of physician claims were estimated by comparison to chart review.</p> <p>Results</p> <p>1,098 (30.5%) physicians completed the chart review. A chart entry on the date of the corresponding claim was found for 10,529 (95.9%) visits. The sensitivity of syndrome definitions based on diagnostic codes in physician claims was low, ranging from 0.11 (fever) to 0.44 (respiratory), the specificity was high, and the PPV was moderate to high, ranging from 0.59 (fever) to 0.85 (respiratory). We found that rarely used diagnostic codes had a higher probability of being false-positives, and that more commonly used diagnostic codes had a higher PPV.</p> <p>Conclusions</p> <p>Future research should identify physician, patient, and encounter characteristics associated with the accuracy of diagnostic codes in physician claims. This would enable public health to improve syndromic surveillance, either by focusing on physician claims whose diagnostic code is more likely to be accurate, or by using all physician claims and weighing each according to the likelihood that its diagnostic code is accurate.</p

Does physical activity counselling enhance the effects of a pedometer-based intervention over the long-term : 12-month findings from the Walking for Wellbeing in the West study

Peer reviewedPublisher PD

Is a Calorie Really a Calorie? Metabolic Advantage of Low-Carbohydrate Diets

The first law of thermodynamics dictates that body mass remains constant when caloric intake equals caloric expenditure. It should be noted, however, that different diets lead to different biochemical pathways that are not equivalent when correctly compared through the laws of thermodynamics. It is inappropriate to assume that the only thing that counts in terms of food consumption and energy balance is the intake of dietary calories and weight storage. Well-controlled studies suggest that calorie content may not be as predictive of fat loss as is reduced carbohydrate consumption. Biologically speaking, a calorie is certainly not a calorie. The ideal weight loss diet, if it even exists, remains to be determined, but a high-carbohydrate/low-protein diet may be unsatisfactory for many obese individuals

Generic and disease-specific health related quality of life in non-cirrhotic, cirrhotic and transplanted liver patients: a cross-sectional study

BACKGROUND: Studies on Health Related Quality of Life (HRQoL) of chronic liver patients were performed in clinical populations. These studies included various disease stages but small variations in aetiology and no transplanted patients. We performed a large HRQoL study in non-cirrhotic, cirrhotic and transplanted liver patients with sufficient variety in aetiology. We compared the generic HRQoL and fatigue between liver patients and healthy controls and compared the disease-specific and generic HRQoL and fatigue between non-cirrhotic, cirrhotic and transplanted liver patients, corrected for aetiology. METHODS: Members of the Dutch liver patient association received the Short Form-36, the Liver Disease Symptom Index and the Multidimensional Fatigue Index-20. Based on reported clinical characteristics we classified respondents (n = 1175) as non-cirrhotic, compensated cirrhotic, decompensated cirrhotic or transplants. We used linear, ordinal and logistic regression to compare the HRQoL between groups. RESULTS: All liver patients showed a significantly worse generic HRQoL and fatigue than healthy controls. Decompensated cirrhotic patients showed a significantly worse disease-specific and generic HRQoL and fatigue than non-cirrhotic patients, while HRQoL differences between non-cirrhotic and compensated cirrhotic patients were predominantly insignificant. Transplanted patients showed a better generic HRQoL, less fatigue and lower probabilities of severe symptoms than non-cirrhotic patients, but almost equal probabilities of symptom hindrance. CONCLUSIONS: HRQoL in chronic liver patients depends on disease stage and transplant history. Non-cirrhotic and compensated cirrhotic patients have a similar HRQoL. Decompensated patients show the worst HRQoL, while transplanted patients show a significantly better HRQoL than cirrhotic and non-cirrhotic patients

The effectiveness of interventions to change six health behaviours: a review of reviews

Background: Several World Health Organisation reports over recent years have highlighted the high incidence of chronic diseases such as diabetes, coronary heart disease and cancer. Contributory factors include unhealthy diets, alcohol and tobacco use and sedentary lifestyles. This paper reports the findings of a review of reviews of behavioural change interventions to reduce unhealthy behaviours or promote healthy behaviours. We included six different health-related behaviours in the review: healthy eating, physical exercise, smoking, alcohol misuse, sexual risk taking (in young people) and illicit drug use. We excluded reviews which focussed on pharmacological treatments or those which required intensive treatments (e. g. for drug or alcohol dependency). Methods: The Cochrane Library, Database of Abstracts of Reviews of Effectiveness (DARE) and several Ovid databases were searched for systematic reviews of interventions for the six behaviours (updated search 2008). Two reviewers applied the inclusion criteria, extracted data and assessed the quality of the reviews. The results were discussed in a narrative synthesis. Results: We included 103 reviews published between 1995 and 2008. The focus of interventions varied, but those targeting specific individuals were generally designed to change an existing behaviour (e. g. cigarette smoking, alcohol misuse), whilst those aimed at the general population or groups such as school children were designed to promote positive behaviours (e. g. healthy eating). Almost 50% (n = 48) of the reviews focussed on smoking (either prevention or cessation). Interventions that were most effective across a range of health behaviours included physician advice or individual counselling, and workplace- and school-based activities. Mass media campaigns and legislative interventions also showed small to moderate effects in changing health behaviours. Generally, the evidence related to short-term effects rather than sustained/longer-term impact and there was a relative lack of evidence on how best to address inequalities. Conclusions: Despite limitations of the review of reviews approach, it is encouraging that there are interventions that are effective in achieving behavioural change. Further emphasis in both primary studies and secondary analysis (e.g. systematic reviews) should be placed on assessing the differential effectiveness of interventions across different population subgroups to ensure that health inequalities are addressed.</p