Health-related quality of life after treatment for bladder cancer in England

Background Little is known about quality of life after bladder cancer treatment. This common cancer is managed using treatments that can affect urinary, sexual and bowel function. Methods To understand quality of life and inform future care, the Department of Health (England) surveyed adults surviving bladder cancer 1–5 years after diagnosis. Questions related to disease status, co-existing conditions, generic health (EQ-5D), cancer-generic (Social Difficulties Inventory) and cancer-specific outcomes (Functional Assessment of Cancer Therapy—Bladder). Results In total, 673 (54%) patients responded; including 500 (74%) men and 539 (80%) with co-existing conditions. Most respondents received endoscopic treatment (60%), while 92 (14%) and 99 (15%) received radical cystectomy or radiotherapy, respectively. Questionnaire completion rates varied (51–97%). Treatment groups reported ≥1 problem using EQ-5D generic domains (59–74%). Usual activities was the most common concern. Urinary frequency was common after endoscopy (34–37%) and radiotherapy (44–50%). Certain populations were more likely to report generic, cancer-generic and cancer-specific problems; notably those with co-existing long-term conditions and those treated with radiotherapy. Conclusion The study demonstrates the importance of assessing patient-reported outcomes in this population. There is a need for larger, more in-depth studies to fully understand the challenges patients with bladder cancer face

Interventions to Promote Cancer Awareness and Early Presentation: Systematic Review

Low cancer awareness contributes to delay in presentation for cancer symptoms and may lead to delay in cancer diagnosis. The aim of this study was to review the evidence for the effectiveness of interventions to raise cancer awareness and promote early presentation in cancer to inform policy and future research. We searched bibliographic databases and reference lists for randomised controlled trials of interventions delivered to individuals, and controlled or uncontrolled studies of interventions delivered to communities. We found some evidence that interventions delivered to individuals modestly increase cancer awareness in the short term and insufficient evidence that they promote early presentation. We found limited evidence that public education campaigns reduce stage at presentation of breast cancer, malignant melanoma and retinoblastoma

Proposed Standards for Medical Education Submissions to the Journal of General Internal Medicine

To help authors design rigorous studies and prepare clear and informative manuscripts, improve the transparency of editorial decisions, and raise the bar on educational scholarship, the Deputy Editors of the Journal of General Internal Medicine articulate standards for medical education submissions to the Journal. General standards include: (1) quality questions, (2) quality methods to match the questions, (3) insightful interpretation of findings, (4) transparent, unbiased reporting, and (5) attention to human subjects’ protection and ethical research conduct. Additional standards for specific study types are described. We hope these proposed standards will generate discussion that will foster their continued evolution

Standard setting: Comparison of two methods

BACKGROUND: The outcome of assessments is determined by the standard-setting method used. There is a wide range of standard – setting methods and the two used most extensively in undergraduate medical education in the UK are the norm-reference and the criterion-reference methods. The aims of the study were to compare these two standard-setting methods for a multiple-choice question examination and to estimate the test-retest and inter-rater reliability of the modified Angoff method. METHODS: The norm – reference method of standard -setting (mean minus 1 SD) was applied to the 'raw' scores of 78 4th-year medical students on a multiple-choice examination (MCQ). Two panels of raters also set the standard using the modified Angoff method for the same multiple-choice question paper on two occasions (6 months apart). We compared the pass/fail rates derived from the norm reference and the Angoff methods and also assessed the test-retest and inter-rater reliability of the modified Angoff method. RESULTS: The pass rate with the norm-reference method was 85% (66/78) and that by the Angoff method was 100% (78 out of 78). The percentage agreement between Angoff method and norm-reference was 78% (95% CI 69% – 87%). The modified Angoff method had an inter-rater reliability of 0.81 – 0.82 and a test-retest reliability of 0.59–0.74. CONCLUSION: There were significant differences in the outcomes of these two standard-setting methods, as shown by the difference in the proportion of candidates that passed and failed the assessment. The modified Angoff method was found to have good inter-rater reliability and moderate test-retest reliability

Assessing the Quality of Clinical Teachers: A Systematic Review of Content and Quality of Questionnaires for Assessing Clinical Teachers

BACKGROUND: Learning in a clinical environment differs from formal educational settings and provides specific challenges for clinicians who are teachers. Instruments that reflect these challenges are needed to identify the strengths and weaknesses of clinical teachers. OBJECTIVE: To systematically review the content, validity, and aims of questionnaires used to assess clinical teachers. DATA SOURCES: MEDLINE, EMBASE, PsycINFO and ERIC from 1976 up to March 2010. REVIEW METHODS: The searches revealed 54 papers on 32 instruments. Data from these papers were documented by independent researchers, using a structured format that included content of the instrument, validation methods, aims of the instrument, and its setting. Results : Aspects covered by the instruments predominantly concerned the use of teaching strategies (included in 30 instruments), supporter role (29), role modeling (27), and feedback (26). Providing opportunities for clinical learning activities was included in 13 instruments. Most studies referred to literature on good clinical teaching, although they failed to provide a clear description of what constitutes a good clinical teacher. Instrument length varied from 1 to 58 items. Except for two instruments, all had to be completed by clerks/residents. Instruments served to provide formative feedback ( instruments) but were also used for resource allocation, promotion, and annual performance review (14 instruments). All but two studies reported on internal consistency and/or reliability; other aspects of validity were examined less frequently. CONCLUSIONS: No instrument covered all relevant aspects of clinical teaching comprehensively. Validation of the instruments was often limited to assessment of internal consistency and reliability. Available instruments for assessing clinical teachers should be used carefully, especially for consequential decisions. There is a need for more valid comprehensive instruments

Assessment of higher order cognitive skills in undergraduate education: modified essay or multiple choice questions? Research paper

Background: Reliable and valid written tests of higher cognitive function are difficult to produce, particularly for the assessment of clinical problem solving. Modified Essay Questions (MEQs) are often used to assess these higher order abilities in preference to other forms of assessment, including multiple-choice questions (MCQs). MEQs often form a vital component of end-of-course assessments in higher education. It is not clear how effectively these questions assess higher order cognitive skills. This study was designed to assess the effectiveness of the MEQ to measure higher-order cognitive skills in an undergraduate institution. Methods: An analysis of multiple-choice questions and modified essay questions (MEQs) used for summative assessment in a clinical undergraduate curriculum was undertaken. A total of 50 MCQs and 139 stages of MEQs were examined, which came from three exams run over two years. The effectiveness of the questions was determined by two assessors and was defined by the questions ability to measure higher cognitive skills, as determined by a modification of Bloom's taxonomy, and its quality as determined by the presence of item writing flaws. Results: Over 50% of all of the MEQs tested factual recall. This was similar to the percentage of MCQs testing factual recall. The modified essay question failed in its role of consistently assessing higher cognitive skills whereas the MCQ frequently tested more than mere recall of knowledge. Conclusion: Construction of MEQs, which will assess higher order cognitive skills cannot be assumed to be a simple task. Well-constructed MCQs should be considered a satisfactory replacement for MEQs if the MEQs cannot be designed to adequately test higher order skills. Such MCQs are capable of withstanding the intellectual and statistical scrutiny imposed by a high stakes exit examination.Edward J Palmer, Peter G Devit

The past, present and future of Scientific discourse.

Published versio

An assessment of functioning and non-functioning distractors in multiple-choice questions: a descriptive analysis

<p>Abstract</p> <p>Background</p> <p>Four- or five-option multiple choice questions (MCQs) are the standard in health-science disciplines, both on certification-level examinations and on in-house developed tests. Previous research has shown, however, that few MCQs have three or four functioning distractors. The purpose of this study was to investigate non-functioning distractors in teacher-developed tests in one nursing program in an English-language university in Hong Kong.</p> <p>Methods</p> <p>Using item-analysis data, we assessed the proportion of non-functioning distractors on a sample of seven test papers administered to undergraduate nursing students. A total of 514 items were reviewed, including 2056 options (1542 distractors and 514 correct responses). Non-functioning options were defined as ones that were chosen by fewer than 5% of examinees and those with a positive option discrimination statistic.</p> <p>Results</p> <p>The proportion of items containing 0, 1, 2, and 3 functioning distractors was 12.3%, 34.8%, 39.1%, and 13.8% respectively. Overall, items contained an average of 1.54 (SD = 0.88) functioning distractors. Only 52.2% (n = 805) of all distractors were functioning effectively and 10.2% (n = 158) had a choice frequency of 0. Items with more functioning distractors were more difficult and more discriminating.</p> <p>Conclusion</p> <p>The low frequency of items with three functioning distractors in the four-option items in this study suggests that teachers have difficulty developing plausible distractors for most MCQs. Test items should consist of as many options as is feasible given the item content and the number of plausible distractors; in most cases this would be three. Item analysis results can be used to identify and remove non-functioning distractors from MCQs that have been used in previous tests.</p