Competency based assessment of speech pathology students' performance in the workplace

Ensuring that speech pathology students are sufficiently competent to practise their profession is of critical importance to the speech pathology profession, students, their future employers, and clients/patients. This thesis describes the development and validation of a competency based assessment of speech pathology students’ performance in the workplace and their readiness to enter the profession. Development involved an extensive literature review regarding the nature of competency and its relationship to professional practice, the purpose and nature of assessment, and the validation of performance assessments. An online and hard copy assessment tool (paper) was designed through integrating multiple sources of information regarding speech pathology and assessment of workplace performance. Sources included research, theory, expert opinion, current practice, and focus group consultations with clinical educators and speech pathology students. The resulting assessment tool and resource material included four generic components of competency (clinical reasoning, professional communication, lifelong learning, and professional role) and seven occupational competencies previously developed by the speech pathology profession. The tool comprised an assessment format, either in a booklet or online, for clinical educators to rate students’ performances on the competencies at mid and end placement using a visual analogue scale. Behavioural descriptors and an assessment resource booklet informed and supported clinical educators’ judgement. The validity of the assessment tool was evaluated through a national field trial and using Messick’s six interrelated validity criteria which address content, substantive, structural, generalisability, external, and consequential aspects of validity (Messick, 1996). The validity of the assessment tool and its use with speech pathology students was evaluated through Rasch analysis, parametric statistical evaluation of relationships existing between information yielded by the Rasch analysis and other factors, and student and clinical educator feedback. The assessment tool was found to have strong validity characteristics across all validity components. Item Fit statistics generated through Rasch analysis ranged from .81 to 1.17 strongly upholding that the assessment items sampled a unidimensional construct of workplace competency for speech pathology students and confirming that generic and occupational competencies are both necessary for competent practice of speech pathology. High Item and Person Reliabilities (analogous to Cronbach’s alpha) were found (.98 and .97 respectively) and a wide range of person measures (-14.2 to 13.1) were generated. This indicated that a large spread of ability and a clear hierarchy of development on the construct was identified and that the assessment tool was highly reliable. This was further confirmed by high Intra Class Correlation coefficients for a small group of paired clinical educators rating the same student in the same workplace (.87) or in different workplaces concurrently (.82). Rasch analysis of the visual analogue scale used to rate student performance on 11 items of competence identified that clinical educators were able to reliably discriminate 7 categories or levels of student performance. This, in combination with careful calibration procedures, has resulted in an assessment tool that Australian Speech Pathology pre-professional preparation programs can use with confidence to place their students’ level of workplace competence into 7 zones of competency, with the seventh representing sufficient competence to enter the profession. The assessment tool also showed strong potential for identifying marginal students and for future use in promoting quality teaching and learning of professional competence. Limitations to the research and the tool validity were discussed, and recommendations made regarding future research. First, the clinical educator, who has dual and possibly conflicting roles as facilitator and assessor of student learning, made the assessment. Second, situating the assessment in the real workplace limits the students’ opportunities to demonstrate competence to those that naturally arise in the workplace. Paradoxically, both these factors also contributed to the validity of the assessment tool. It was recommended that the assessment tool be revised on the basis of the information gathered from the field trial, that further data be collected to ensure a broader proportional representation of speech pathology programs, to investigate possible threats to validity as well as those areas for which the tool showed promise. This research developed the first prototype of a validated assessment of entry level speech pathology competence that is grounded in a unified theoretical conception of entry level competence to the profession of speech pathology and the developmental progression required to reach this competence. This research will assist the profession of speech pathology by ensuring that speech pathologists enter the workplace well equipped to provide quality care to their future clients, the ultimate goal of any professional preparation program. Messick, S. (1996). Validity of performance assessments. In G. W. Phillips (Ed.), Technical Issues in Large-Scale Performance Assessment (pp. 1-18). Washington: National Centre for Education Statistics

Engaging educators and students in the national roll-out of a new assessment tool (COMPASS).

This paper presents key results of an evaluation of a project (funded by ALTC), that led the integration of a newly developed competency based assessment tool (COMPASS™) within all 13 speech pathology education programs nationally. As part of the roll-out process, workshops were provided to close to 1,000 speech pathology clinical educators and students were introduced to the new tool through their lectures and tutorials. In order to provide formative feedback in the early stages of the project (end 2006 – early 2007); a questionnaire (designed to elicit both quantitative and qualitative data) was used following the first 6 workshops (214 educators) and after the first lectures to students at 2 universities (145 students). Most educators (95-97%) and students (74-85%) reported understanding the main concepts that inform key components of COMPASSTM (behavioural descriptors, generic competencies, and use of the Visual Analogue Scale). Qualitative feedback indicated a need for further support in relation to understanding the need for direct observation and the use of the Visual Analogue Scale. Toward the completion of the project (end 2007 – early 2008), a similar questionnaire was distributed to clinical educators (33 respondents) and to students in 3 universities (76 respondents). Results continued to be positive for understanding of main concepts for educators (79-100%) and for students (75-92%). An important finding was the close similarity between educators and students in relation to their understandings about the tool, the areas in which they reported wanting more support/training, and the ways in which they would like to obtain further experience. The implications of these findings for the further embedding of the new assessment tool are discussed.Adelaid

Impact of placement type on the development of clinical competency in speech–language pathology students

"This is the peer reviewed version of the following article: Sheepway L, Lincoln M, McAllister S. Impact of placement type on the development of clinical competency in speech-language pathology students. International Journal of Language and Communication Disorders . 2014 Mar-Apr;49(2):189-203. doi: 10.1111/1460-6984.12059., online 1 Nov 2013, which has been published in final form athttp://dx.doi.org/10.1111/1460-6984.12059. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Self-Archiving."Background: Speech language pathology students gain experience and clinical competency through clinical education placements. However, currently little empirical information exists regarding how competency develops. Existing research about the effectiveness of placement types and models in developing competency are generally descriptive and based on opinions and perceptions. The changing nature of education of speech language pathology students, diverse student cohorts, and the crisis in finding sufficient clinical education placements mean that establishing the most effective and efficient methods for developing clinical competency in students is needed. Aims: To gather empirical information regarding the development of competence in speech language pathology students, and to determine if growth of competency differs in groups of students completing placements which differ in terms of caseload, intensity and setting. Methods & Procedures: Participants were students in the third year of a four year undergraduate speech language pathology degree, who completed 3 clinical placements across the year and were assessed with the COMPASS® competency assessment tool. Competency development for the whole group across the three placements is described. Growth of competency in groups of students completing different placement types is compared. Interval level data generated from the students’ COMPASS® results were subjected to parametric statistical analyses. Outcomes and Results: The whole group of students increased significantly in competency from placement to placement across different placement settings, intensities and client age groups. Groups completing child placements achieved significantly higher growth in competency when compared to competency growth of students completing adult placements. Growth of competency was not significantly different for students experiencing different intensity of placements, or different placement settings. Conclusions and Implications: These results confirm that the competency of speech language pathology students develops across three clinical placements over a one year period regardless of placement type or context, indicating that there may be transfer of learning between placements types. Further research investigating patterns of competency development in speech language pathology students is warranted to ensure that assumptions used to design clinical learning opportunities are based on valid evidence

Redefining ‘Chinese’ L1 in SLP: Considerations for the assessment of Chinese bilingual/bidialectal language skills

This is the peer reviewed version made available following 12 month embargo from the date of publication (4 December 2015) in accordance with publisher policy. It has been published in final form at DOI: http://dx.doi.org/10.3109/17549507.2015.1081285.'Purpose: Language assessment of bilingual/bidialectal children can be complex. This is particularly true for speakers from China, who are likely to be bilingual and bidialectal at the same time. There has been, however, a lack of understanding of the diversity of Chinese languages as well as data on bidialectal children’s L1 syntactic development and the development of L1 bidialectal children’s L2 acquisition. Method: This paper provides information on the complexity of the language system for people from China. It will present illustrative examples of the expressive language outputs of bilingual and bidialectal children from the perspective of bilingual, bidialectal linguists and speech-language pathologists. Then it will outline why appropriate assessment tools and practices for identification of language impairment in bilingual Chinese children need to be developed. Result: Considerations include that Chinese bilingual children may differ in L2 performance because of lack of exposure in the target language or because of their varied L1 dialectal backgrounds, but not necessarily because of language impairment. Conclusion: When evaluating morphosyntactic performance of bilingual children, a series of reliable threshold indicators for possible language impairment is urgently needed for SLPs to facilitate accurate diagnosis of language impairment

Prisoner death investigations: a means for improving safety in prisons and societies?

(Inter)national law imposes obligations to investigate prisoner deaths. These investigations create rich data which are little utilised by academia, policy or practice (inter)nationally, but provide a window to identify, organise and apply learning that could safeguard prisons and societies. We report preliminary findings from a research partnership, aiming to utilise England and Wales' Prisons and Probation Ombudsman fatal incident investigations to improve prison and societal safety. Data were gathered through 16 semi-structured interviews with Ombudsman staff spanning senior investigator to senior management levels. We consider the Ombudsman's success at establishing feedback loops, drawing on regulatory theory and using cybernetics as an analytical approach. We explore three research questions: i) how does the PPO seek to effect change in prisons following a prisoner death? ii) do PPO actions have the intended effects on prisons? iii) does the PPO adjust its actions? Analysis illustrates potential to: i) extend self-conscious, confident communication work, early in the investigation, across PPO investigators; ii) communicate praise as well as deficits to prison staff throughout investigations; iii) adjust template PPO recommendations. An evidence base to inform recommendations is urgently required and holds potential to produce a step change in (inter)national prison oversight, and potentially prison conditions around the world

Theoretically and empirically-informed narrative descriptions of competency development

Background: Determining whether a student has demonstrated readiness to enter their profession, particularly in professions where semi-autonomous practice is required on graduation, is challenging. Discourse regarding performance assessment has moved from measurement and psychometric validity towards the value of qualitative approaches and narratives to support sound judgement of readiness for practice. The objective of this study was to develop narrative descriptions of competency development. The research questions were: (i) how do experienced clinicians describe students who are developing their clinical competency, and (ii) how do they describe a student who is ready for semi-autonomous professional practice? We investigated these questions within the profession of clinical exercise physiology (CEP). CEP is a role emerging health profession in Australia and therefore is in the early stages of articulating its understanding of readiness for practice and related assessment of performance. Summary of Work: A social constructivist theoretical perspective informed the methodology. Across three focus groups, 17 CEP clinicians with clinical educator experience were asked to describe (i) what students who are developing their clinical competency 'look like' and (ii) what students who were ready for professional practice 'look like'. Data saturation was reached at the third focus group. A thematic analysis identified the behavioural constructs describing competency development which were then used to write three narrative descriptions of points along a competency continuum. These narrative descriptions were further evaluated and refined through semi-structured interviews. Summary of Results: The behavioural constructs which informed the narrative descriptions included: consistency, managing complexity, flexibility, safe practice, independence and insight into own performance. Participants considered the narrative descriptions to accurately describe the continuum of competency development and a student who is ready for professional practice. Discussion and Conclusions: The narrative descriptions fit with contemporary discourse on assessment in clinical workplace settings. The narrative descriptions may be useful in providing support for quality judgements of students' performances in the workplace setting. That is, the narrative descriptions could be used as reference points by those constructing their judgements of student performances. Take-home Messages: The broad behavioural constructs in the narratives may be relevant to other health professionals' practice

Improving management of student clinical placements: insights from activity theory

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.Background An approach to improve management of student clinical placements, the Building Teams for Quality Learning project, was trialed in three different health services. In a previous paper the authors explored in some detail the factors associated with considerable success of this approach at one of these services. In this paper, the authors extend this work with further analysis to determine if the more limited outcomes observed with participants at the other two services could be explained by application of activity theory and in particular the expansive learning cycle. Methods Staff at three health services participated in the Building Teams for Quality Learning project: a dental clinic, a community aged care facility and a rural hospital. At each site a team of seven multi-disciplinary staff completed the project over 9 to 12 months (total 21 participants). Evaluation data were collected through interviews, focus groups and direct observation of staff and students. Following initial thematic analysis, further analysis was conducted to compare the processes and outcomes at each participating health service drawing on activity theory and the expansive learning cycle. Results Fifty-one interview transcripts, 33 h of workplace observation and 31 sets of workshop field notes (from 36 h of workshops) were generated. All participants were individually supportive of, and committed to, high quality student learning experiences. As was observed with staff at the dental clinic, a number of potentially effective strategies were discussed at the aged care facility and the rural hospital workshops. However, participants in these two health services could not develop a successful implementation plan. The expansive learning cycle element of modeling and testing new solutions was not achieved and participants were unable, collectively to reassess and reinterpret the object of their activities. Conclusion The application of activity theory and the expansive learning cycle assisted a deeper understanding of the differences in outcomes observed across the three groups of participants. This included identifying specific points in the expansive learning cycle at which the three groups diverged. These findings support some practical recommendations for health services that host student clinical placements

Assessment of student competency in a simulated speech-language pathology clinical placement

“This is an Accepted Manuscript of an article published by Taylor & Francis in International Journal of Speech-Language Pathology on 30 August 2013, available online: http://wwww.tandfonline.com/10.3109/17549507.2013.809603.” http://dx.doi.org/10.3109/17549507.2013.809603.Clinical education programs in speech-language pathology enable the transition of students’ knowledge and skills from the classroom to the workplace. Simulated clinical learning experiences provide an opportunity to address the competency development of novice students. This study reports on the validation of an assessment tool designed to evaluate speech-language pathology students’ performance in a simulated clinical placement. The Assessment of Foundation Clinical Skills (AFCS) was designed to link to concepts and content of COMPASS®: Competency Assessment in Speech Pathology, a validated assessment of performance in the workplace. It incorporates units and elements of competency relevant to the placement. The validity of the AFCS was statistically investigated using Rasch analysis. Participants were 18 clinical educators and 130 speech-language pathology students undertaking the placement. Preliminary results support the validity of the AFCS as an assessment of foundation clinical skills of students in this simulated clinical placement. All units of competency and the majority of elements were relevant and representative of these skills. The use of a visual analogue scale which included a pre-Novice level to rate students’ performance on units of competency was supported. This research provides guidance for development of quality assessments of performance in simulated placements

Variables Impacting the Time Taken to Wean Children From Enteral Tube Feeding to Oral Intake

Made available per the LWW Author Permission Guidelines. Copyright © 2019 Espghan and Naspghan. This author accepted manuscript is made available following 12 month embargo from date of publication (June 2019) in accordance with the publisher’s archiving policyObjectives: This study investigated biological factors, which may influence the time taken for children to wean from enteral to oral intake. Methods: Retrospective case-note audit of 62 tube-fed children (nasogastric or percutaneous endoscopic gastrostomy) aged 6 months to 8 years, participating in an intensive tube weaning program. Program design included family-focused mealtimes, child autonomy, and appetite stimulation. A regression model was developed, which shows the combination of variables with the most predictive power for time taken to wean. Results: Data from 62 children who were highly dependent (minimum 93% of calories provided enterally) on tube feeding for an extended period of time (mean = 2.1 years) were analysed. Children's mean body mass index z score at time of weaning was -0.47 (standard deviation 1.03) (mean weight = 10.54 kg) and 54 (87%) presented with a range of medical conditions. Forty-four children (71%) remained completely tube free at 3 months postintervention and an additional 5 children (10%) were fully tube weaned within 10 months of program commencement. Type of feeding tube, medical complexity, age, and length of time tube fed all significantly correlated with time taken to wean. Logistic regression modelling indicated that the type of feeding tube in combination with the degree of medical complexity and time tube fed were the strongest predictors of time taken to wean. Conclusions: Biological factors usually considered to impact on successful weaning from tube feeding (volume of oral intake, oral skill, or mealtime behaviours) were not relevant; however, the type of feeding tube in combination with the degree of medical complexity and time tube fed were the strongest predictors. The impact of psychosocial factors should be investigated to identify if these mitigated the effects of the biological variables

Sustaining remote-area programs: retinal camera use by Aboriginal health workers and nurses in a Kimberley partnership

Objective: to describe how a novel program of diabetic retinopathy screening was conceived, refined and sustained in a remote region over 10 years, and to evaluate its activities and outcomes. Design: program description; analysis of regional screening database; audit of electronic client registers of Aboriginal community controlled health services (ACCHSs). Setting and participants: 1318 Aboriginal and 271 non-Aboriginal individuals who underwent retinal screening in the 5 years to September 2004 in the Kimberley region of north-west Australia; 11 758 regular local Aboriginal clients of Kimberley ACCHSs as at January 2005. Main outcome measures: characteristics of clients and camera operators, prevalence of retinopathy, photograph quality, screening intervals and coverage. Results: among Aboriginal clients, 21% had diabetic retinopathy: 19% with nonproliferative retinopathy, 1.2% with proliferative retinopathy, and 2.8% with maculopathy. Corresponding figures for non-Aboriginal clients were 11%, 11%, 0 and 0.4%, respectively. Photograph quality was generally high, and better for non-Aboriginal clients, younger Aboriginal clients and from 2002 (when mydriatic use became universal). Quality was not related to operator qualifications, certification or experience. Of 718 regular Aboriginal clients with diabetes on local ACCHS databases, 48% had a record of retinal screening within the previous 18 months, and 65% within the previous 30 months. Conclusions: Screening for diabetic retinopathy performed locally by Aboriginal health workers and nurses with fundus cameras can be successfully sustained with regional support. Formal certification appears unnecessary. Data sharing across services, client recall and point-of-care prompts generated by electronic information systems, together with policies making primary care providers responsible for care coordination, support appropriate timely screening