Using item response theory to explore the psychometric properties of extended matching questions examination in undergraduate medical education

BACKGROUND: As assessment has been shown to direct learning, it is critical that the examinations developed to test clinical competence in medical undergraduates are valid and reliable. The use of extended matching questions (EMQ) has been advocated to overcome some of the criticisms of using multiple-choice questions to test factual and applied knowledge. METHODS: We analysed the results from the Extended Matching Questions Examination taken by 4th year undergraduate medical students in the academic year 2001 to 2002. Rasch analysis was used to examine whether the set of questions used in the examination mapped on to a unidimensional scale, the degree of difficulty of questions within and between the various medical and surgical specialties and the pattern of responses within individual questions to assess the impact of the distractor options. RESULTS: Analysis of a subset of items and of the full examination demonstrated internal construct validity and the absence of bias on the majority of questions. Three main patterns of response selection were identified. CONCLUSION: Modern psychometric methods based upon the work of Rasch provide a useful approach to the calibration and analysis of EMQ undergraduate medical assessments. The approach allows for a formal test of the unidimensionality of the questions and thus the validity of the summed score. Given the metric calibration which follows fit to the model, it also allows for the establishment of items banks to facilitate continuity and equity in exam standards

Applying the trigger review method after a brief educational intervention: potential for teaching and improving safety in GP specialty training?

<p>Background: The Trigger Review Method (TRM) is a structured approach to screening clinical records for undetected patient safety incidents (PSIs) and identifying learning and improvement opportunities. In Scotland, TRM participation can inform GP appraisal and has been included as a core component of the national primary care patient safety programme that was launched in March 2013. However, the clinical workforce needs up-skilled and the potential of TRM in GP training has yet to be tested. Current TRM training utilizes a workplace face-to-face session by a GP expert, which is not feasible. A less costly, more sustainable educational intervention is necessary to build capability at scale. We aimed to determine the feasibility and impact of TRM and a related training intervention in GP training.</p> Methods We recruited 25 west of Scotland GP trainees to attend a 2-hour TRM workshop. Trainees then applied TRM to 25 clinical records and returned findings within 4-weeks. A follow-up feedback workshop was held. <p>Results: 21/25 trainees (84%) completed the task. 520 records yielded 80 undetected PSIs (15.4%). 36/80 were judged potentially preventable (45%) with 35/80 classified as causing moderate to severe harm (44%). Trainees described a range of potential learning and improvement plans. Training was positively received and appeared to be successful given these findings. TRM was valued as a safety improvement tool by most participants.</p> <p>Conclusion: This small study provides further evidence of TRM utility and how to teach it pragmatically. TRM is of potential value in GP patient safety curriculum delivery and preparing trainees for future safety improvement expectations.</p&gt

QuantiFERON®-TB gold in-tube performance for diagnosing active tuberculosis in children and adults in a high burden setting.

To determine whether QuantiFERON®-TB Gold In-Tube (QFT) can contribute to the diagnosis of active tuberculosis (TB) in children in a high-burden setting and to assess the performance of QFT and tuberculin skin test (TST) in a prospective cohort of TB suspect children compared to adults with confirmed TB in Tanzania. Sensitivity and specificity of QFT and TST for diagnosing active TB as well as indeterminate QFT rates and IFN-γ levels were assessed in 211 TB suspect children in a Tanzanian district hospital and contrasted in 90 adults with confirmed pulmonary TB. Sensitivity of QFT and TST in children with confirmed TB was 19% (5/27) and 6% (2/31) respectively. In adults sensitivity of QFT and TST was 84% (73/87) and 85% (63/74). The QFT indeterminate rate in children and adults was 27% and 3%. Median levels of IFN-γ were lower in children than adults, particularly children <2 years and HIV infected. An indeterminate result was associated with age <2 years but not malnutrition or HIV status. Overall childhood mortality was 19% and associated with an indeterminate QFT result at baseline. QFT and TST showed poor performance and a surprisingly low sensitivity in children. In contrast the performance in Tanzanian adults was good and comparable to performance in high-income countries. Indeterminate results in children were associated with young age and increased mortality. Neither test can be recommended for diagnosing active TB in children with immature or impaired immunity in a high-burden setting

A cross-sectional study of vascular risk factors in a rural South African population : data from the Southern African Stroke Prevention Initiative (SASPI)

Background: Rural sub-Saharan Africa is at an early stage of economic and health transition. It is predicted that the 21st century will see a serious added economic burden from non-communicable disease including vascular disease in low-income countries as they progress through the transition. The stage of vascular disease in a population is thought to result from the prevalence of vascular risk factors. Already hypertension and stroke are common in adults in sub-Saharan Africa. Using a multidisciplinary approach we aimed to assess the prevalence of several vascular risk factors in Agincourt, a rural demographic surveillance site in South Africa. Methods: We performed a cross sectional random sample survey of adults aged over 35 in Agincourt (population ≈ 70 000). Participants were visited at home by a trained nurse who administered a questionnaire, carried out clinical measurements and took a blood sample. From this we assessed participants' history of vascular risk, blood pressure using an OMRON 705 CP monitor, waist circumference, body mass index (BMI), ankle brachial index (ABI), and total and HDL cholesterol. Results: 402 people (24% men) participated. There was a high prevalence of smoking in men, but the number of cigarettes smoked was small. There was a striking difference in mean BMI between men and women (22.8 kg/m2 versus 27.2 kg/m2), but levels of blood pressure were very similar. 43% of participants had a blood pressure greater than 140/90 or were on anti-hypertensive treatment and 37% of participants identified with measured high blood pressure were on pharmacological treatment. 12% of participants had an ABI of < 0.9, sugesting the presence of sub-clinical atheroma. 25.6% of participants had a total cholesterol level > 5 mmol/l. Conclusion: We found a high prevalence of hypertension, obesity in women, and a suggestion of subclinical atheroma despite relatively favourable cholesterol levels in a rural South African population. South Africa is facing the challenge of an emerging epidemic of vascular disease. Research to establish the social determinates of these risk factors and interventions to reduce both individual and population risk are required

Context and the evidence based paradigm: The potential for participatory research and systems thinking in oral health

The implementation of research evidence to promote oral health is critical, given the intransigent and emerging challenges for policy-makers at a population level. Despite this, little attention has been paid to implementation research within the evidence-based paradigm. This is important as getting research evidence into clinical practice is not a linear path that consists of simple sequential steps. In this article, we argue that we need to consider a broader range of conceptual and methodological approaches to increase the value of information generated. This should be undertaken either in parallel with empirical and experimental designs, or in some cases, instead of. This is important if we are going to understand the complexity and contextual knowledge of the ‘system’, within which interventions are implemented. Involving key stakeholders alongside empirical and experimental designs is one helpful approach. Examples of these approaches include Patient and Public Involvement and the development of Core Outcome Sets, where the views of those that will be potentially affected by the research, are included. The use of theoretical frameworks and process evaluations alongside trials are also important, if they are fully integrated into the approach taken to address the research question. A more radical approach is using participatory designs and ‘systems thinking’. Participatory approaches include subject matter 'experts by experience’. These include patients, their families, carers, healthcare professionals, services managers, policy-makers, commissioners and researchers. Participatory approaches raise important questions about who facilitates the process, when it should happen and how the diverse actors become meaningfully engaged so that their involvement is active, democratic and on-going. We argue that the issues of control, power and language are central to this and represent a paradigmatic shift to conventional approaches. Systems thinking captures the idea that public health problems commonly involve multiple interdependent and interconnected factors, which interact with each other dynamically. This approach challenges the simplicity of the hierarchy of evidence and linear sequential logic, when it doesn’t account for context. In contrast, systems thinking accepts complexity de novo and emphasises the need to understand the whole system rather than its individual component parts. We conclude with the idea that participatory and systems thinking helps to unpack the diverse agents that are often involved in the generation and translation of evidence into clinical dental practice. It moves our conception of research away from a simple exchange between ‘knowledge producers’ and ‘knowledge users’ and raises both methodological and epistemological challenges

Increasing the options for reducing adverse events: Results from a modified Delphi technique

which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Background: The aim of this paper is to illustrate a simple method for increasing the range of possible options for reducing adverse events in Australian hospitals, which could have been, but was not, adopted in the wake of the landmark 1995 &apos;Quality in Australian Health Care &apos; study, and to report the suggestions and the estimated lapse time before they would impact upon mortality and morbidity. Method: The study used a modified Delphi technique that first elicited options for reducing adverse events from an invited panel selected on the basis of their knowledge of the area of adverse events and quality assurance. Initial suggestions were collated and returned to them for reconsideration and comment. Results: Completed responses from both stages were obtained from 20 of those initially approached. Forty-one options for reducing AEs were identified with an average lapse time of 3.5 years. Hospital regulation had the least delay (2.4 years) and out of hospital information the greatest (6.4 years). Conclusion: Following identification of the magnitude of the problem of adverse events in the &apos;Quality in Australian Health Care &apos; study a more rapid and broad ranging response was possible than occurred. Apparently viable options for reducing adverse events and associated mortality and morbidity remain unexploited

The prevalence and experience of oral diseases in Adelaide nursing home residents

The document attached has been archived with permission from the Australian Dental Association. An external link to the publisher’s copy is included.Background: The twenty-first century will see the evolution of a population of dentate older Australians with dental needs very different from those of older adults in past years. This study provided comprehensive information concerning oral disease prevalence in older South Australian nursing home residents. Methods: This paper presents cross-sectional baseline results. Results: Most of the 224 residents, from seven randomly selected nursing homes, were functionally dependent, medically compromised, cognitively impaired and behaviourally difficult older adults who presented many complex challenges to carers and to dental professionals. Two-thirds (66 per cent) were edentulous with many dental problems and treatment needs. Dentate residents had a mean of 11.9 teeth present, higher than previously reported. The prevalence and experience of coronal and root caries and plaque accumulation was very high in dentate residents; especially males, those admitted more than three years previously, those who ate fewer food types and those who were severely cognitively impaired. These residents had more retained roots, decayed teeth and missing teeth, and fewer filled teeth when compared with data for community-dwelling older adults. Conclusions: This study highlighted the poor oral health status of these nursing home residents and the great impact of dementia on their high levels of oral diseases.JM Chalmers, C Hodge, JM Fuss, AJ Spencer, KD Carte

Identifying work related injuries: comparison of methods for interrogating text fields

Background: Work-related injuries in Australia are estimated to cost around $57.5 billion annually, however there are currently insufficient surveillance data available to support an evidence-based public health response. Emergency departments (ED) in Australia are a potential source of information on work-related injuries though most ED’s do not have an ‘Activity Code’ to identify work-related cases with information about the presenting problem recorded in a short free text field. This study compared methods for interrogating text fields for identifying work-related injuries presenting at emergency departments to inform approaches to surveillance of work-related injury.---------- Methods: Three approaches were used to interrogate an injury description text field to classify cases as work-related: keyword search, index search, and content analytic text mining. Sensitivity and specificity were examined by comparing cases flagged by each approach to cases coded with an Activity code during triage. Methods to improve the sensitivity and/or specificity of each approach were explored by adjusting the classification techniques within each broad approach.---------- Results: The basic keyword search detected 58% of cases (Specificity 0.99), an index search detected 62% of cases (Specificity 0.87), and the content analytic text mining (using adjusted probabilities) approach detected 77% of cases (Specificity 0.95).---------- Conclusions The findings of this study provide strong support for continued development of text searching methods to obtain information from routine emergency department data, to improve the capacity for comprehensive injury surveillance

Dental skill mix: a cross-sectional analysis of delegation practices between dental and dental hygiene-therapy students involved in team training in the South of England

BACKGROUND: Research suggests that health professionals who have trained together have a better understanding of one another’s scope of practice and are thus equipped for teamwork during their professional careers. Dental hygiene-therapists (DHTs) are mid-level providers that can deliver routine care working alongside dentists. This study examines patterns of delegation (selected tasks and patients) by dental students to DHT students training together in an integrated team. METHODS: A retrospective sample of patient data (n = 2,063) was extracted from a patient management system showing the treatment activities of two student cohorts (dental and DHT) involved in team training in a primary care setting in the South of England over two academic years. The data extracted included key procedures delegated by dental students to DHT students coded by skill-mix of operator (e.g., fissure sealants, restorations, paediatric extractions) and patient demography. χ(2) tests were conducted to investigate the relationship between delegation and patient age group, gender, smoking status, payment-exemption status, and social deprivation. RESULTS: A total of 2,063 patients managed during this period received treatments that could be undertaken by either student type; in total, they received 14,996 treatment procedures. The treatments most commonly delegated were fissure sealants (90%) and restorations (51%); whilst the least delegated were paediatric extractions (2%). Over half of these patients (55%) had at least one instance of delegation from a dental to a DHT student. Associations were found between delegation and patient age group and smoking status (P <0.001). Children under 18 years old had a higher level of delegation (86%) compared with adults of working age (50%) and patients aged 65 years and over (56%). A higher proportion of smokers had been delegated compared with non-smokers (45% cf. 26%; P <0.001). CONCLUSIONS: The findings suggest that delegation of care to DHT students training as a team with dental students, involved significantly greater experience in treating children and adult smokers, and providing preventive rather than invasive care in this integrated educational and primary care setting. The implications for their contribution to dentistry and the dental team are discussed, along with recommendations for primary care data recording