Establishing construct validity of a virtual-reality training simulator for hysteroscopy via a multimetric scoring system

Background: The aims of this study are to determine construct validity for the HystSim virtual-reality (VR) training simulator for hysteroscopy via a new multimetric scoring system (MMSS) and to explore learning curves for both novices and experienced surgeons. Methods: Fifteen relevant metrics had been identified for diagnostic hysteroscopy by means of hierarchical task decomposition. They were grouped into four modules (visualization, ergonomics, safety, and fluid handling) and individually weighted, building the MMSS for this study. In a first step, 24 novice medical students and 12 experienced gynecologists went through a self-paced teaching tutorial, in which all participants received clearly stated goals and instructions on how to carry out hysteroscopic procedures properly for this study. All subjects performed five repeated trials on two different exercises on HystSim (exploration and diagnosis exercises). After each trial the results were presented to the participants in the form of an automated objective feedback report (AOFR). Construct validity for the MMSS and learning curves were investigated by comparing the performance between novices and experienced surgeons and in between the repeated trials. To study the effect of repeated practice, 23 of the novices returned 2weeks later for a second training session. Results: Comparing novices with the experienced group, the ergonomics and fluid handling modules resulted in construct validity, while the visualization module did not, and for the safety module the experienced group even scored significantly lower than novices in both exercises. The overall score showed only construct validity when the safety module was excluded. Concerning learning curves, all subjects improved significantly during the training on HystSim, with clear indication that the second training session was beneficial for novice surgeons. Conclusions: Construct validity for HystSim has been established for different modules of VR metrics on a new MMSS developed for diagnostic hysteroscopy. Careful refinement and further testing of metrics and scores is required before using them as assessment tools for operative skill

Evaluation of a new virtual-reality training simulator for hysteroscopy

Background: To determine realism and training capacity of HystSim, a new virtual-reality simulator for the training of hysteroscopic interventions. Methods: Sixty-two gynaecological surgeons with various levels of expertise were interviewed at the 13th Practical Course in Gynaecologic Endoscopy in Davos, Switzerland. All participants received a 20-min hands-on training on the simulator and filled out a four-page questionnaire. Twenty-three questions with respect to the realism of the simulation and the training capacity were answered on a seven-point Likert scale along with 11 agree-disagree statements concerning the HystSim training in general. Results: Twenty-six participants had performed more than 50 hysteroscopies ("experts”) and 36 equal to or fewer than 50 ("novices”). Four of 60 (6.6%) responding participants judged the overall impression as "7 - absolutely realistic”, 40 (66.6%) as "6 - realistic”, and 16 (26.6%) as "5 - somewhat realistic”. Novices (6.48; 95% confidence interval [CI] 6.28-6.7) rated the overall training capacity significantly higher than experts (6.08; 95% CI 5.85-6.3), however, high-grade acceptance was found in both groups. In response to the statements, 95.2% believe that HystSim allows procedural training of diagnostic and therapeutic hysteroscopy, and 85.5% suggest that HystSim training should be offered to all novices before performing surgery on real patients. Conclusion: Face validity has been established for a new hysteroscopic surgery simulator. Potential trainees and trainers assess it to be a realistic and useful tool for the training of hysteroscopy. Further systematic validation studies are needed to clarify how this system can be optimally integrated into the gynaecological curriculu

Evaluation of a new virtual-reality training simulator for hysteroscopy

BACKGROUND: To determine realism and training capacity of HystSim, a new virtual-reality simulator for the training of hysteroscopic interventions. METHODS: Sixty-two gynaecological surgeons with various levels of expertise were interviewed at the 13(th) Practical Course in Gynaecologic Endoscopy in Davos, Switzerland. All participants received a 20-min hands-on training on the simulator and filled out a four-page questionnaire. Twenty-three questions with respect to the realism of the simulation and the training capacity were answered on a seven-point Likert scale along with 11 agree-disagree statements concerning the HystSim training in general. RESULTS: Twenty-six participants had performed more than 50 hysteroscopies ("experts") and 36 equal to or fewer than 50 ("novices"). Four of 60 (6.6%) responding participants judged the overall impression as "7 - absolutely realistic", 40 (66.6%) as "6 - realistic", and 16 (26.6%) as "5 - somewhat realistic". Novices (6.48; 95% confidence interval [CI] 6.28-6.7) rated the overall training capacity significantly higher than experts (6.08; 95% CI 5.85-6.3), however, high-grade acceptance was found in both groups. In response to the statements, 95.2% believe that HystSim allows procedural training of diagnostic and therapeutic hysteroscopy, and 85.5% suggest that HystSim training should be offered to all novices before performing surgery on real patients. CONCLUSION: Face validity has been established for a new hysteroscopic surgery simulator. Potential trainees and trainers assess it to be a realistic and useful tool for the training of hysteroscopy. Further systematic validation studies are needed to clarify how this system can be optimally integrated into the gynaecological curriculum