Competency based assessment of speech pathology students' performance in the workplace

Ensuring that speech pathology students are sufficiently competent to practise their profession is of critical importance to the speech pathology profession, students, their future employers, and clients/patients. This thesis describes the development and validation of a competency based assessment of speech pathology students’ performance in the workplace and their readiness to enter the profession. Development involved an extensive literature review regarding the nature of competency and its relationship to professional practice, the purpose and nature of assessment, and the validation of performance assessments. An online and hard copy assessment tool (paper) was designed through integrating multiple sources of information regarding speech pathology and assessment of workplace performance. Sources included research, theory, expert opinion, current practice, and focus group consultations with clinical educators and speech pathology students. The resulting assessment tool and resource material included four generic components of competency (clinical reasoning, professional communication, lifelong learning, and professional role) and seven occupational competencies previously developed by the speech pathology profession. The tool comprised an assessment format, either in a booklet or online, for clinical educators to rate students’ performances on the competencies at mid and end placement using a visual analogue scale. Behavioural descriptors and an assessment resource booklet informed and supported clinical educators’ judgement. The validity of the assessment tool was evaluated through a national field trial and using Messick’s six interrelated validity criteria which address content, substantive, structural, generalisability, external, and consequential aspects of validity (Messick, 1996). The validity of the assessment tool and its use with speech pathology students was evaluated through Rasch analysis, parametric statistical evaluation of relationships existing between information yielded by the Rasch analysis and other factors, and student and clinical educator feedback. The assessment tool was found to have strong validity characteristics across all validity components. Item Fit statistics generated through Rasch analysis ranged from .81 to 1.17 strongly upholding that the assessment items sampled a unidimensional construct of workplace competency for speech pathology students and confirming that generic and occupational competencies are both necessary for competent practice of speech pathology. High Item and Person Reliabilities (analogous to Cronbach’s alpha) were found (.98 and .97 respectively) and a wide range of person measures (-14.2 to 13.1) were generated. This indicated that a large spread of ability and a clear hierarchy of development on the construct was identified and that the assessment tool was highly reliable. This was further confirmed by high Intra Class Correlation coefficients for a small group of paired clinical educators rating the same student in the same workplace (.87) or in different workplaces concurrently (.82). Rasch analysis of the visual analogue scale used to rate student performance on 11 items of competence identified that clinical educators were able to reliably discriminate 7 categories or levels of student performance. This, in combination with careful calibration procedures, has resulted in an assessment tool that Australian Speech Pathology pre-professional preparation programs can use with confidence to place their students’ level of workplace competence into 7 zones of competency, with the seventh representing sufficient competence to enter the profession. The assessment tool also showed strong potential for identifying marginal students and for future use in promoting quality teaching and learning of professional competence. Limitations to the research and the tool validity were discussed, and recommendations made regarding future research. First, the clinical educator, who has dual and possibly conflicting roles as facilitator and assessor of student learning, made the assessment. Second, situating the assessment in the real workplace limits the students’ opportunities to demonstrate competence to those that naturally arise in the workplace. Paradoxically, both these factors also contributed to the validity of the assessment tool. It was recommended that the assessment tool be revised on the basis of the information gathered from the field trial, that further data be collected to ensure a broader proportional representation of speech pathology programs, to investigate possible threats to validity as well as those areas for which the tool showed promise. This research developed the first prototype of a validated assessment of entry level speech pathology competence that is grounded in a unified theoretical conception of entry level competence to the profession of speech pathology and the developmental progression required to reach this competence. This research will assist the profession of speech pathology by ensuring that speech pathologists enter the workplace well equipped to provide quality care to their future clients, the ultimate goal of any professional preparation program. Messick, S. (1996). Validity of performance assessments. In G. W. Phillips (Ed.), Technical Issues in Large-Scale Performance Assessment (pp. 1-18). Washington: National Centre for Education Statistics

Engaging educators and students in the national roll-out of a new assessment tool (COMPASS).

This paper presents key results of an evaluation of a project (funded by ALTC), that led the integration of a newly developed competency based assessment tool (COMPASS™) within all 13 speech pathology education programs nationally. As part of the roll-out process, workshops were provided to close to 1,000 speech pathology clinical educators and students were introduced to the new tool through their lectures and tutorials. In order to provide formative feedback in the early stages of the project (end 2006 – early 2007); a questionnaire (designed to elicit both quantitative and qualitative data) was used following the first 6 workshops (214 educators) and after the first lectures to students at 2 universities (145 students). Most educators (95-97%) and students (74-85%) reported understanding the main concepts that inform key components of COMPASSTM (behavioural descriptors, generic competencies, and use of the Visual Analogue Scale). Qualitative feedback indicated a need for further support in relation to understanding the need for direct observation and the use of the Visual Analogue Scale. Toward the completion of the project (end 2007 – early 2008), a similar questionnaire was distributed to clinical educators (33 respondents) and to students in 3 universities (76 respondents). Results continued to be positive for understanding of main concepts for educators (79-100%) and for students (75-92%). An important finding was the close similarity between educators and students in relation to their understandings about the tool, the areas in which they reported wanting more support/training, and the ways in which they would like to obtain further experience. The implications of these findings for the further embedding of the new assessment tool are discussed.Adelaid

Impact of placement type on the development of clinical competency in speech–language pathology students

"This is the peer reviewed version of the following article: Sheepway L, Lincoln M, McAllister S. Impact of placement type on the development of clinical competency in speech-language pathology students. International Journal of Language and Communication Disorders . 2014 Mar-Apr;49(2):189-203. doi: 10.1111/1460-6984.12059., online 1 Nov 2013, which has been published in final form athttp://dx.doi.org/10.1111/1460-6984.12059. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Self-Archiving."Background: Speech language pathology students gain experience and clinical competency through clinical education placements. However, currently little empirical information exists regarding how competency develops. Existing research about the effectiveness of placement types and models in developing competency are generally descriptive and based on opinions and perceptions. The changing nature of education of speech language pathology students, diverse student cohorts, and the crisis in finding sufficient clinical education placements mean that establishing the most effective and efficient methods for developing clinical competency in students is needed. Aims: To gather empirical information regarding the development of competence in speech language pathology students, and to determine if growth of competency differs in groups of students completing placements which differ in terms of caseload, intensity and setting. Methods & Procedures: Participants were students in the third year of a four year undergraduate speech language pathology degree, who completed 3 clinical placements across the year and were assessed with the COMPASS® competency assessment tool. Competency development for the whole group across the three placements is described. Growth of competency in groups of students completing different placement types is compared. Interval level data generated from the students’ COMPASS® results were subjected to parametric statistical analyses. Outcomes and Results: The whole group of students increased significantly in competency from placement to placement across different placement settings, intensities and client age groups. Groups completing child placements achieved significantly higher growth in competency when compared to competency growth of students completing adult placements. Growth of competency was not significantly different for students experiencing different intensity of placements, or different placement settings. Conclusions and Implications: These results confirm that the competency of speech language pathology students develops across three clinical placements over a one year period regardless of placement type or context, indicating that there may be transfer of learning between placements types. Further research investigating patterns of competency development in speech language pathology students is warranted to ensure that assumptions used to design clinical learning opportunities are based on valid evidence

Redefining ‘Chinese’ L1 in SLP: Considerations for the assessment of Chinese bilingual/bidialectal language skills

This is the peer reviewed version made available following 12 month embargo from the date of publication (4 December 2015) in accordance with publisher policy. It has been published in final form at DOI: http://dx.doi.org/10.3109/17549507.2015.1081285.'Purpose: Language assessment of bilingual/bidialectal children can be complex. This is particularly true for speakers from China, who are likely to be bilingual and bidialectal at the same time. There has been, however, a lack of understanding of the diversity of Chinese languages as well as data on bidialectal children’s L1 syntactic development and the development of L1 bidialectal children’s L2 acquisition. Method: This paper provides information on the complexity of the language system for people from China. It will present illustrative examples of the expressive language outputs of bilingual and bidialectal children from the perspective of bilingual, bidialectal linguists and speech-language pathologists. Then it will outline why appropriate assessment tools and practices for identification of language impairment in bilingual Chinese children need to be developed. Result: Considerations include that Chinese bilingual children may differ in L2 performance because of lack of exposure in the target language or because of their varied L1 dialectal backgrounds, but not necessarily because of language impairment. Conclusion: When evaluating morphosyntactic performance of bilingual children, a series of reliable threshold indicators for possible language impairment is urgently needed for SLPs to facilitate accurate diagnosis of language impairment

Variables Impacting the Time Taken to Wean Children From Enteral Tube Feeding to Oral Intake

Made available per the LWW Author Permission Guidelines. Copyright © 2019 Espghan and Naspghan. This author accepted manuscript is made available following 12 month embargo from date of publication (June 2019) in accordance with the publisher’s archiving policyObjectives: This study investigated biological factors, which may influence the time taken for children to wean from enteral to oral intake. Methods: Retrospective case-note audit of 62 tube-fed children (nasogastric or percutaneous endoscopic gastrostomy) aged 6 months to 8 years, participating in an intensive tube weaning program. Program design included family-focused mealtimes, child autonomy, and appetite stimulation. A regression model was developed, which shows the combination of variables with the most predictive power for time taken to wean. Results: Data from 62 children who were highly dependent (minimum 93% of calories provided enterally) on tube feeding for an extended period of time (mean = 2.1 years) were analysed. Children's mean body mass index z score at time of weaning was -0.47 (standard deviation 1.03) (mean weight = 10.54 kg) and 54 (87%) presented with a range of medical conditions. Forty-four children (71%) remained completely tube free at 3 months postintervention and an additional 5 children (10%) were fully tube weaned within 10 months of program commencement. Type of feeding tube, medical complexity, age, and length of time tube fed all significantly correlated with time taken to wean. Logistic regression modelling indicated that the type of feeding tube in combination with the degree of medical complexity and time tube fed were the strongest predictors of time taken to wean. Conclusions: Biological factors usually considered to impact on successful weaning from tube feeding (volume of oral intake, oral skill, or mealtime behaviours) were not relevant; however, the type of feeding tube in combination with the degree of medical complexity and time tube fed were the strongest predictors. The impact of psychosocial factors should be investigated to identify if these mitigated the effects of the biological variables

Improving management of student clinical placements: insights from activity theory

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.Background An approach to improve management of student clinical placements, the Building Teams for Quality Learning project, was trialed in three different health services. In a previous paper the authors explored in some detail the factors associated with considerable success of this approach at one of these services. In this paper, the authors extend this work with further analysis to determine if the more limited outcomes observed with participants at the other two services could be explained by application of activity theory and in particular the expansive learning cycle. Methods Staff at three health services participated in the Building Teams for Quality Learning project: a dental clinic, a community aged care facility and a rural hospital. At each site a team of seven multi-disciplinary staff completed the project over 9 to 12 months (total 21 participants). Evaluation data were collected through interviews, focus groups and direct observation of staff and students. Following initial thematic analysis, further analysis was conducted to compare the processes and outcomes at each participating health service drawing on activity theory and the expansive learning cycle. Results Fifty-one interview transcripts, 33 h of workplace observation and 31 sets of workshop field notes (from 36 h of workshops) were generated. All participants were individually supportive of, and committed to, high quality student learning experiences. As was observed with staff at the dental clinic, a number of potentially effective strategies were discussed at the aged care facility and the rural hospital workshops. However, participants in these two health services could not develop a successful implementation plan. The expansive learning cycle element of modeling and testing new solutions was not achieved and participants were unable, collectively to reassess and reinterpret the object of their activities. Conclusion The application of activity theory and the expansive learning cycle assisted a deeper understanding of the differences in outcomes observed across the three groups of participants. This included identifying specific points in the expansive learning cycle at which the three groups diverged. These findings support some practical recommendations for health services that host student clinical placements

Assessment of student competency in a simulated speech-language pathology clinical placement

“This is an Accepted Manuscript of an article published by Taylor & Francis in International Journal of Speech-Language Pathology on 30 August 2013, available online: http://wwww.tandfonline.com/10.3109/17549507.2013.809603.” http://dx.doi.org/10.3109/17549507.2013.809603.Clinical education programs in speech-language pathology enable the transition of students’ knowledge and skills from the classroom to the workplace. Simulated clinical learning experiences provide an opportunity to address the competency development of novice students. This study reports on the validation of an assessment tool designed to evaluate speech-language pathology students’ performance in a simulated clinical placement. The Assessment of Foundation Clinical Skills (AFCS) was designed to link to concepts and content of COMPASS®: Competency Assessment in Speech Pathology, a validated assessment of performance in the workplace. It incorporates units and elements of competency relevant to the placement. The validity of the AFCS was statistically investigated using Rasch analysis. Participants were 18 clinical educators and 130 speech-language pathology students undertaking the placement. Preliminary results support the validity of the AFCS as an assessment of foundation clinical skills of students in this simulated clinical placement. All units of competency and the majority of elements were relevant and representative of these skills. The use of a visual analogue scale which included a pre-Novice level to rate students’ performance on units of competency was supported. This research provides guidance for development of quality assessments of performance in simulated placements

Implications of Variability in Clinical Bedside Swallowing Assessment Practices by Speech Language Pathologists

Author version made available following 12 month embargo from date of publication (12 July 2016) in accordance with publisher copyright. The final publication is available at link.springer.com.Speech language pathology (SLP) clinical bedside swallowing assessments (CBSA) are a cornerstone of quality care for patients in acute hospitals who have dysphagia. The CBSA informs clinical diagnosis and decisions regarding further instrumental assessment, and is used to develop a management plan and monitor progress. However, self-report and retrospective research shows that SLPs are highly variable in their use of assessment components considered by experts to be important for quality CBSA, casting doubt on the validity and reliability of CBSA. This prospective study describes the components included by SLPs when designing a standardised evidence based dysphagia assessment protocol for acute care patients and observed patterns of component use. The findings confirm that SLPs use the CBSA for multiple purposes beyond diagnosis of aspiration risk and dysphagia presence/severity. They are highly variable in their use of certain components, but also demonstrate consistent use of a core set. It is apparent that SLPs prioritise the application of clinical reasoning to tailor their CBSA to the patient over following a highly structured item-based protocol. The variability in component use likely reflects a complex clinical reasoning process that draws on a wide variety of information combined with expert knowledge as is also observed in many other medical specialties. Rather than promoting the standardisation of CBSA protocols that constrain SLP practice to strict item-based assessment protocols, consideration should be given to promoting the value and facilitating the clinical reasoning process that supports the utility of the CBSA for diagnosis, patient centred management and treatment planning.Springer Science+Business Media New Yor

Rethinking attitudes to student clinical supervision and patient care: a change management success story

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.Background The aim of this project was to explore the process of change in a busy community dental clinic following a team development intervention designed to improve the management of student supervision during clinical placements. Methods An action research model was used. Seven members of a community dental clinic team (three dentists, two dental therapists, one dental assistant and the clinic manager), together with the university clinical placement supervisor participated in the team development intervention. The intervention consisted of two profiling activities and associated workshops spread six months apart. These activities focused on individual work preferences and overall team performance with the aim of improving the functioning of the clinic as a learning environment for dental students. Evaluation data consisted of 20 participant interviews, fourteen hours of workplace observation and six sets of field notes. Following initial thematic analysis, project outcomes were re-analysed using activity theory and expansive learning as a theoretical framework. Results At project commencement students were not well integrated into the day-to-day clinic functioning. Staff expressed a general view that greater attention to student supervision would compromise patient care. Following the intervention greater clinical team cohesion and workflow changes delivered efficiencies in practice, enhanced relationships among team members, and more positive attitudes towards students. The physical layout of the clinic and clinical workloads were changed to achieve greater involvement of all team members in supporting student learning. Unexpectedly, these changes also improved clinic functioning and increased the number of student placements available. Conclusions In navigating the sequential stages of the expansive learning cycle, the clinical team ultimately redefined the ‘object’ of their activity and crossed previously impervious boundaries between healthcare delivery and student supervision with benefits to all parties

A national initiative: Children of parents with a mental illness

Focuses on the national scoping project undertaken by the Australian Infant, Child, Adolescent and Family Mental Health Association Ltd. to identify services available and needed for children of parents with a mental illness. Recommendations of the scoping project report; Outcomes expected from the national initiative launched by the government in response to the scoping project report; Key stages of implementation for the national initiative.Garvin, Sue; McAllister, Sue and Robinson, Phili