Development of a portfolio of learning for postgraduate family medicine training in South Africa: a Delphi study

<p>Abstract</p> <p>Background</p> <p>Within the 52 health districts in South Africa, the family physician is seen as the clinical leader within a multi-professional district health team. Family physicians must be competent to meet 90% of the health needs of the communities in their districts. The eight university departments of Family Medicine have identified five unit standards, broken down into 85 training outcomes, for postgraduate training. The family medicine registrar must prove at the end of training that all the required training outcomes have been attained. District health managers must be assured that the family physician is competent to deliver the expected service. The Colleges of Medicine of South Africa (CMSA) require a portfolio to be submitted as part of the uniform assessment of all registrars applying to write the national fellowship examinations. This study aimed to achieve a consensus on the contents and principles of the first national portfolio for use in family medicine training in South Africa.</p> <p>Methods</p> <p>A workshop held at the WONCA Africa Regional Conference in 2009 explored the purpose and broad contents of the portfolio. The 85 training outcomes, ideas from the WONCA workshop, the literature, and existing portfolios in the various universities were used to develop a questionnaire that was tested for content validity by a panel of 31 experts in family medicine in South Africa, via the Delphi technique in four rounds. Eighty five content items (national learning outcomes) and 27 principles were tested. Consensus was defined as 70% agreement. For those items that the panel thought should be included, they were also asked how to provide evidence for the specific item in the portfolio, and how to assess that evidence.</p> <p>Results</p> <p>Consensus was reached on 61 of the 85 national learning outcomes. The panel recommended that 50 be assessed by the portfolio and 11 should not be. No consensus could be reached on the remaining 24 outcomes and these were also omitted from the portfolio. The panel recommended that various types of evidence be included in the portfolio. The panel supported 26 of the 27 principles, but could not reach consensus on whether the portfolio should reflect on the relationship between the supervisor and registrar.</p> <p>Conclusion</p> <p>A portfolio was developed and distributed to the eight departments of Family Medicine in South Africa, and the CMSA, to be further tested in implementation.</p

The reliability of in-training assessment when performance improvement is taken into account

During in-training assessment students are frequently assessed over a longer period of time and therefore it can be expected that their performance will improve. We studied whether there really is a measurable performance improvement when students are assessed over an extended period of time and how this improvement affects the reliability of the overall judgement. In-training assessment results were obtained from 104 students on rotation at our university hospital or at one of the six affiliated hospitals. Generalisability theory was used in combination with multilevel analysis to obtain reliability coefficients and to estimate the number of assessments needed for reliable overall judgement, both including and excluding performance improvement. Students’ clinical performance ratings improved significantly from a mean of 7.6 at the start to a mean of 7.8 at the end of their clerkship. When taking performance improvement into account, reliability coefficients were higher. The number of assessments needed to achieve a reliability of 0.80 or higher decreased from 17 to 11. Therefore, when studying reliability of in-training assessment, performance improvement should be considered

Assessing fitness-to-practice of overseas-trained health practitioners by Australian registration & accreditation bodies

Assessment of fitness-to-practice of health professionals trained overseas and who wish to practice in Australia is undertaken by a range of organisations. These organisations conduct assessments using a range of methods. However there is very little published about how these organisations conduct their assessments. The purpose of the current paper is to investigate the methods of assessment used by these organisations and the issues associated with conducting these assessments

The national portfolio for postgraduate family medicine training in South Africa : a descriptive study of acceptability, educational impact, and usefulness for assessment

Background: Since 2007 a portfolio of learning has become a requirement for assessment of postgraduate family medicine training by the Colleges of Medicine of South Africa. A uniform portfolio of learning has been developed and content validity established among the eight postgraduate programmes. The aim of this study was to investigate the portfolio's acceptability, educational impact, and perceived usefulness for assessment of competence. Methods: Two structured questionnaires of 35 closed and open-ended questions were delivered to 53 family physician supervisors and 48 registrars who had used the portfolio. Categorical and nominal/ordinal data were analysed using simple descriptive statistics. The open-ended questions were analysed with ATLAS.ti software. Results: Half of registrars did not find the portfolio clear, practical or feasible. Workshops on portfolio use, learning, and supervision were supported, and brief dedicated time daily for reflection and writing. Most supervisors felt the portfolio reflected an accurate picture of learning, but just over half of registrars agreed. While the portfolio helped with reflection on learning, participants were less convinced about how it helped them plan further learning. Supervisors graded most rotations, suggesting understanding the summative aspect, while only 61% of registrars reflected on rotations, suggesting the formative aspects are not yet optimally utilised. Poor feedback, the need for protected academic time, and pressure of service delivery impacting negatively on learning. Conclusion: This first introduction of a national portfolio for postgraduate training in family medicine in South Africa faces challenges similar to those in other countries. Acceptability of the portfolio relates to a clear purpose and guide, flexible format with tools available in the workplace, and appreciating the changing educational environment from university-based to national assessments. The role of the supervisor in direct observations of the registrar and dedicated educational meetings, giving feedback and support, cannot be overemphasized

Assessment for learning

Most students enter medical school with the intent to become a physician and work in patient care; and rightfully so because good patient care is important for society. But there is also a need in society for physicians or MDs with special abilities in conducting biomedical and clinical research. This special need was the reason why at the University of Maastricht a graduate-entry medical program was begun with an annual intake of 30 students. But it was not the only reason. Educational innovation and experimentation have become increasingly difficult due to the rapidly increasing enrolment in many medical programs. Our new program was therefore also started with the intent of enabling experimentation and innovation at the curriculum level.Lambert Schuwirth, Helena Ward and Sylvia Heenema