European Society for Vascular Surgery (ESVS) Certification of Theoretical and Practical Competences in Basic Vascular Ultrasound: Validity Investigation of the Assessment Tools
Objective: The aim of this study was to gather validity evidence for the Assessment of basic Vascular Ultrasound
Expertise (AVAUSE) tool, and to establish a pass/fail score for each component, to support decisions for certification.
Methods: A cross sectional validation study performed during the European Society for Vascular Surgery’s annual
meeting.Validity evidence was sought for the theoretical test and two practical tests based on Messick’s framework.
The participants were vascular surgeons, vascular surgical trainees, sonographers, and nurses with varying
experience levels. Five vascular ultrasound experts developed the theoretical and two practical test components
of the AVAUSE tool for each test component. Two stations were set up for carotid examinations and two for
superficial venous incompetence (SVI) examinations. Eight raters were assigned in pairs to each station. Three
methods were used to set pass/fail scores: contrasting groups’ method; rater consensus; and extended Angoff.
Results: Nineteen participants were enrolled. Acceptable internal consistency reliability (Cronbach’s alpha) for the
AVAUSE theoretical (0.93), carotid (0.84), and SVI (0.65) practical test were shown. In the carotid examination,
inter-rater reliability (IRR) for the two rater pairs was good: 0.68 and 0.78, respectively. The carotid scores
correlated significantly with years of experience (Pearson’s r 1⁄4 0.56, p 1⁄4 .013) but not with number of
examinations in the last five years. For SVI, IRR was excellent at 0.81 and 0.87. SVI performance scores did not
correlate with years of experience and number of examinations. The pass/fail score set by the contrasting groups’
method was 29 points out of 50. The rater set pass/fail scores were 3.0 points for both carotid and SVI
examinations and were used to determine successful participants. Ten of 19 participants passed the tests and were
certified.
Conclusion: Validity evidence was sought and established for the AVAUSE comprehensive tool, including pass/fail
standards. AVAUSE can be used to assess competences in basic vascular ultrasound, allowing operators to
progress towards independent practice