The document attached has been archived with permission from the editor of the Medical Journal of Australia. An external link to the publisher’s copy is included.Although it is commonly assumed that the quality of medical school education in Australia is uniformly high, there is no national process for assessing its outcomes. There is substantial variability in the content of medical school curricula, and the process of curriculum change is becoming more challenging because of intense competition for time and space in the course. A national exit examination could provide a uniform standard of assessment for all medical school graduates in Australia, as well as foreign graduates applying to work in Australia. Such an examination could assess medical school outcomes, monitor the effects of curriculum change, and provide a benchmark for new medical schools that would help medical curricula evolve to better meet society’s needs.Bogda Koczwara, Martin H N Tattersall, Michael B Barton, Brendon J Coventry, Joanna M Dewar, Jeremy L Millar, Ian N Olver, Max A Schwarz, Darren L Starmer, David R Turner and Martin R Stockler, for the Cancer Council of Australia Oncology Education Committe

Barton, M.

Coventry, B.

Dewar, J.

Koczwara, B.

Miller, J.

Olver, I.

Schwarz, M.

Starmer, D.

Stockler, M.

Tattersall, M.

Turner, D.

The Medical Journal of Australia

Adelaide Research & Scholarship

 PUBLISHED VERSION   Koczwara, Bogda; Tattersall, Martin H.; Barton, Michael B.; Coventry, Brendon John; Dewar, Joanna M.; Miller, Jeremy L.; Olver, Ian N.; Schwarz, Max A.; Starmer, Darren L.; Turner, David Richard; Stockler, Martin R.  Achieving equal standards in medical student education: is a national exit examination the answer? Medical Journal of Australia, 2005; 182(5):228-230  This article is available from the Medical Journal of Australia at:   http://www.mja.com.au/public/issues/182_05_070305/koc10653_fm.html                         http://hdl.handle.net/2440/17499  PERMISSIONS   This document has been archived with permission from the editor of the Medical Journal of Australia, 26 April 2007.    FOR DEBATEAchieving equal standards in medical student education: is a national exit examination the answer?Bogda Koczwara, Martin H N Tattersall, Michael B Barton, Brendon J Coventry, Joanna M Dewar, Jeremy L Millar, Ian N Olver, Max A Schwarz, Darren L Starmer, David R Turner and Martin R Stockler,for the Cancer Council of Australia Oncology Education CommitteeFlinders Medical Centre, Adelaide, SA.Bogda Koczwara, FRACP, MBioethics, Head of Medical Oncology. University of Sydney, Sydney, NSW.Martin H N Tattersall, MD, FRACP, Professor of Cancer Medicine; Martin R Stockler, FRACP, MSc, Senior Lecturer in Medicine and Clinical Epidemiology. Liverpool Health Service, Sydney, NSW.Michael B Barton, FRANZCR, Research Director, Cancer Therapy Centre. Royal Adelaide Hospital, Adelaide, SA.Brendon J Coventry, PhD, FRACS, Surgeon, University Department of Surgery; Ian N Olver, MD, PhD, FRACP, Cancer Council Professor of Cancer Care, University of Adelaide. Sir Charles Gairdner Hospital, Perth, WA.Joanna M Dewar, FRACP, Medical Oncologist. The Alfred Hospital, Melbourne, VIC.Jeremy L Millar, FRANZCR, FAChPM, Radiation Oncologist; Max A Schwarz, FRACP, FACP, FAChPM, Head of Medical Oncology Unit. University of Western Australia, Perth, WA.Darren L Starmer, BN, RN, Project Officer – Curriculum Development, Faculty of Medicine and Dentistry; currently, Education Development Officer, Virtual Medical Centre.com. Flinders University, Adelaide, SA.David R Turner, PhD, Associate Dean, School of Medicine. Reprints will not be available from the authors. Correspondence: Dr Bogda Koczwara, Medical Oncology, Flinders Medical Centre, Flinders Drive, Bedford Park, SA 5042. Bogda.Koczwara@flinders.edu.auThe Medical Journal of Australia ISSN: 0025-729X 7 March 2005 182 5 228-230©The Medical Journal of Australia 2005www.mja.com.auFor Debateimportant to ensure that the outcomes of the teachinof an appropriate standard, irrespective of methodthere is no process to evaluate these outcomes.A national exit examination could help monitor kskills of medical graduates. Is it time for Australia to 228 MJA • Volume 182 NumABSTRACT• Although it is commonly assumed that the quality of medical school education in Australia is uniformly high, there is no national process for assessing its outcomes.• There is substantial variability in the content of medical school curricula, and the process of curriculum change is becoming more challenging because of intense competition for time and space in the course.• A national exit examination could provide a uniform standard of assessment for all medical school graduates in Australia, as well as foreign graduates applying to work in Australia.• Such an examination could assess medical school outcomes, monitor the effects of curriculum change, and provide a benchmark for new medical schools that would help medical MJA 2005; 182: 228–230curricula evolve to better meet society’s needs.edcrecaM ical schools provide an important service to society byating a cadre of practitioners responsible for health-re. It is thus important to ensure that medical educa-tion is of acceptable and consistent quality and producespractitioners who can meet society’s needs. While Australianmedical schools take considerable care in selecting applicants andintroducing innovative teaching methods, their programs varysubstantially in both process and content.1 In addition, curriculumchange is becoming more challenging because of intense competi-tion for time and space in medical courses (see Box). Whilediversity of teaching methods and content may be desirable, it isg process ares. At present,nowledge andconsider it?The need for changeMost data on standardised assessment programs come from theUnited States and Canada, which have had such programs formany years. These data demonstrate that an exit examination canbe valid7 and reliable, and can correlate well with clinical skills andfuture performance in multiple disciplines.8-10 Such an examina-tion can serve as a monitor of performance of individual medicalschools and as a selection tool for postgraduate training programsthat may be fairer than letters of recommendation. It may also actas a monitor of curriculum change or an indicator of the need forchange. Data on foreign medical graduates sitting the United StatesMedical Licensing Examination indicate that Australian graduatesperform very well (Dr J Boulet, Director of Research, Foundationfor Advancement of International Medical Education andResearch, USA, personal communication, Dec 2004). However, asthe cohort is small and highly selected, it may not be representa-tive of Australian graduates overall and does not allow analysis ofindividual medical schools.Some countries, notably Germany and France, have state-basedmedical examinations, while others, such as the United Kingdom,base eligibility to practise on evidence of graduation from anaccredited medical school. With the increased globalisation ofmedical workforces, the lack of a standard approach has beenidentified as a barrier to improving the medical workforce in theEuropean Union.11Australia has no standardised national exit examination. Whiledebate on its merits is not new, such an examination has beenpreviously regarded as unnecessary. This was because it has beenassumed that the quality of Australian medical graduates was high,and the process of accrediting medical schools (established toensure appropriate standards) was excellent, and because nomedical school has ever been singled out as of low quality. Whychange if there is no evidence of a problem? We propose thefollowing reasons.Firstly, as we do not systematically evaluate outcomes of individ-ual medical schools, we can not be certain that the aboveassumptions are correct. The challenge to curriculum deliverycaused by the continuing expansion of medical knowledge is butone justification for such scrutiny.ber 5 • 7 March 2005FOR DEBATESecondly, systematic monitoring and improvement of outcomesmay be desirable, even if outcomes are acceptable, and mayprevent future problems. While we recognise potential downsidesto a national exit examination, we argue for such an examinationas a way of monitoring outcomes to stimulate and inform furtherimprovement.Advantages of a national examination in AustraliaPassing a national examination requires a minimum standard ofperformance. This standard can serve as a target for attainment, ameasure for comparing outcomes between universities and formonitoring the effects of curriculum change, and a benchmark fornew medical schools. A national exit examination would necessar-ily entail the explicit statement of professional values and expecta-tions, a laudable process in itself.The examination can also be used to measure the performanceof overseas-trained doctors, allowing fairer assessment of thesedoctors. Standardisation of assessment might also facilitate therecognition of Australian-trained graduates overseas and the devel-opment of reciprocal arrangements with other countries that usestandard exit examinations.Finally, a change to the assessment process can influencecurriculum change.12The disadvantagesThe main disadvantage of an exit examination is that it might locksome medical schools into merely preparing students for theexamination and thus restrict their ability and willingness to exploremore innovative teaching. However, innovative teaching is not anend in itself but a means to obtain better outcomes. There is no wayof knowing if outcomes are improved unless they are measured.It is important to ensure that an exit examination does notbecome the sole source of evaluation of students and medicalschools, and thus become an inflexible tool that ignores regionaland other differences. An exit examination should be designed tocomplement rather than replace the range of evaluation methodsused by individual medical schools (including objective structuredclinical examinations [OSCEs], portfolios and logbooks). Thecontent as well as the format of the examination would need toreflect the desired outcomes. Thus, it might need to include aclinical component and might not be solely a written and/ormultiple-choice examination — although data from the UnitedStates suggest that performance on the written Medical LicensingExamination correlates well with performance in structured clini-cal examinations.13 As it is increasingly recognised that studentsneed to demonstrate skills (eg, problem solving) as well asknowledge, these skills must also be evaluated in the examination.An important challenge is the need for resources to create andmaintain a quality assessment tool. Are the advantages of theexamination sufficient to justify its expense? The answer may lie inthe observation of the founding father of the study of management,Peter Drucker, that “If you can’t measure it, you can’t manage it”.Our ability to measure outcomes of Australian medical educationis currently limited.A concern about the exit examination is its use to rank medicalschools, resulting in competition between them. This concern seemssurprising, as ranking of Australian universities already occurs.14Finally, it can be argued that an exit examination is superfluous,as a range of competency examinations are already performed atthe national level in specialty training programs. However, theseexaminations do not allow monitoring of medical school outcomesor feedback on the curricula. Furthermore, it is worth noting that,as all specialist examinations are already national, it is the individ-ual university assessment of medical students that is out of step.Is an exit examination the only answer?Other ways of influencing medical school outcomes are already inplace, including accreditation and the development of “ideal”curricula.5 The Australian Medical Council — the main accredita-tion body — sets out the principles and standards of medicaleducation, including assessment. However, these relate more toprocess than to curriculum content, with the latter left to thejudgement of individual medical schools.15 There are few data onwhether these processes actually lead to desired outcomes.It may be that defining the desired outcomes should be theintermediate step before an exit examination is considered. This wasthe strategy of the Scottish Deans’ Medical Curriculum Group, whichdeveloped an agreed set of outcomes defining qualities and abilitiesof graduates from any Scottish medical school.16 This was then usedto provide a common approach to curriculum and assessment.An alternative, indirect measure of medical school outcomeswould be feedback from the specialty colleges on results of theirexaminations. However, this may be cumbersome to coordinate,and demand as many resources as a national examination.ConclusionsWhile the common perception is that the general competence ofAustralian medical graduates is of a high standard, the unansweredquestion is whether it might be improved by better monitoring ofthe outcomes of medical student education.With the worsening shortage of doctors and the need forappropriately trained medical graduates, the time has come to askThe Oncology Education Committee and curriculum changeFor over 10 years, the Oncology Education Committee of the Cancer Council of Australia has been monitoring the quality of cancer education among medical students in Australia,2-4 working towards ensuring that the knowledge of medical students more closely meets the needs of society. Cancer affects one in three Australians, making it an important component of medical student education. The Committee has developed an Ideal Oncology Curriculum,5 which has been endorsed by the International Union Against Cancer. The Committee has also been gathering and promoting educational resources for medical students.Despite these attempts to promote adequate education, data show that graduates’ knowledge of cancer remains variable and at times suboptimal across different medical schools, while their exposure to cancer in medical curricula is less than it was 10 years ago.6 Discussions with students and medical school representatives about how best to ensure the inclusion of cancer knowledge and skills in medical school curricula have revealed enormous pressures from many sources to add to already overfilled programs. The final outcome is often driven more by the preferences, interests and capabilities of individual staff and students and less by externally imposed targets. Without uniformly agreed outcomes to aspire to, and a system to monitor their achievement, curriculum change becomes nearly impossible.MJA • Volume 182 Number 5 • 7 March 2005 229FOR DEBATEnot only how many medical graduates do we need, but also whatstandards should they achieve? A national exit examination givessociety a way to set standards for acceptable levels of competence,and to monitor and influence standards to match future needs andexpectations.There is no doubt that an exit examination is only as effective asthe assessment tools it uses. However, if some of the inventivenessand attention currently dedicated to curriculum design wereredirected to developing a national assessment process, it ispossible that this process could reliably inform the outcomes ofmedical student education and serve as a platform for continuingimprovement.And if medical education is as good as we believe, what are weafraid of?Competing interestsNone identified.References1 Barton MB, Simons RG, for the Oncology Education Committee of theAustralian Cancer Society. A survey of cancer curricula in Australian andNew Zealand medical schools in 1997. Med J Aust 1999; 170: 225-227.2 Smith WT, Tattersall MHN, Irwig LM, Langlands AO. Undergraduateeducation about cancer. Eur J Cancer 1991; 27: 1448-1453.3 Tattersall MHN, Langlands AO. Oncology curricula in Australia. Med JAust 1993; 158: 224-225.4 Tattersall MHN, Langlands AO, Smith W, Irwig L. Undergraduate educa-tion about cancer. A survey of clinical oncologists and clinicians responsi-ble for cancer teaching in Australian Medical Schools. Eur J Cancer 1993;29A: 1639-1642.5 Cancer Council Australia. Ideal oncology curriculum for medical schools.Sydney: Australian Cancer Society, 1999. Available at: www.can-cer.org.au/documents/Ideal_Oncology_Curric_HI.pdf (accessed Jan2005).6 Barton MB, Tattersall MH, Butow PN, et al. Cancer knowledge and skillsof interns in Australia and New Zealand in 2001: comparison with 1990,and between course types. Med J Aust 2003; 178: 285–289. 7 Van Zanten M, Boulet JR, McKinley DW. Correlates of performance of theECFMG Clinical Skills Assessment: influences of candidate characteristicson performance. Acad Med 2003; 78 (Suppl): S72-S74.8 Andriole DA, Jeffe DB, Whelan AJ. What predicts surgical internshipperformance? Am J Surg 2004; 188: 161-164.9 Tamblyn R, Abrahamowicz M, Dauphinee WD, et al. Association betweenlicensure examination scores and practice in primary care. JAMA 2002;288: 3019-3026.10 Tamblyn R, Abrahamowicz M, Brailovsky C, et al. Association betweenlicensing examination scores and resource use and quality of care inprimary care practice. JAMA 1998; 280: 989-996.11 Richards J. Medical issues within a European dimension. Europa 2000; 4(1). Web journal available at: www.intellectbooks.com/europa/number9/medic.htm (accessed Jan 2005).12 Newble DI, Jaeger K. The effect of assessments and examinations on thelearning of medical students. Med Educ 1983; 17: 165-171.13 Simon SR, Volkan K, Hamann C, et al. The relationship between second-year medical students’ OSCE scores and USMLE Step 1 scores. MedTeach 2002; 24: 535-539.14 Australian Education Network. Rankings of Australian universities. Availa-ble at: www.australian-universities.com/rankings.php (accessed Jan2005).15 Australian Medical Council. Assessment and accreditation of medicalschools: standards and procedures. Canberra: Australian Medical Coun-cil, 2002. Available at: www.amc.org.au/forms/AccredGuidelines.pdf(accessed Jan 2005).16 Scottish Deans’ Medical Curriculum Group. The Scottish doctor. Under-graduate learning outcomes and their assessment: a foundation forcompetent and reflective practitioners. Scottish Deans’ Medical Curricu-lum Group, 2002. Available at: www.scottishdoctor.org/resources/scotdoc2.pdf (accessed Jan 2005).(Received 17 Sep 2004, accepted 5 Jan 2005) ❏230 MJA • Volume 182 Number 5 • 7 March 2005

Achieving equal standards in medical student education: is a national exit examination the answer?

https://digital.library.adelaide.edu.au/dspace/bitstream/2440/17499/2/hdl_17499.pdf

Achieving equal standards in medical student education: is a national exit examination the answer?

Abstract

Similar works

Full text

Available Versions

Adelaide Research & Scholarship