Who's Better? Who's Best? Pairwise Deep Ranking for Skill Determination

We present a method for assessing skill from video, applicable to a variety of tasks, ranging from surgery to drawing and rolling pizza dough. We formulate the problem as pairwise (who's better?) and overall (who's best?) ranking of video collections, using supervised deep ranking. We propose a novel loss function that learns discriminative features when a pair of videos exhibit variance in skill, and learns shared features when a pair of videos exhibit comparable skill levels. Results demonstrate our method is applicable across tasks, with the percentage of correctly ordered pairs of videos ranging from 70% to 83% for four datasets. We demonstrate the robustness of our approach via sensitivity analysis of its parameters. We see this work as effort toward the automated organization of how-to video collections and overall, generic skill determination in video.Comment: CVPR 201

Surgical Skill Assessment on In-Vivo Clinical Data via the Clearness of Operating Field

Surgical skill assessment is important for surgery training and quality control. Prior works on this task largely focus on basic surgical tasks such as suturing and knot tying performed in simulation settings. In contrast, surgical skill assessment is studied in this paper on a real clinical dataset, which consists of fifty-seven in-vivo laparoscopic surgeries and corresponding skill scores annotated by six surgeons. From analyses on this dataset, the clearness of operating field (COF) is identified as a good proxy for overall surgical skills, given its strong correlation with overall skills and high inter-annotator consistency. Then an objective and automated framework based on neural network is proposed to predict surgical skills through the proxy of COF. The neural network is jointly trained with a supervised regression loss and an unsupervised rank loss. In experiments, the proposed method achieves 0.55 Spearman's correlation with the ground truth of overall technical skill, which is even comparable with the human performance of junior surgeons.Comment: MICCAI 201

Virtual exams: has COVID-19 provided the impetus to change assessment methods in medicine?

AIMS: The ongoing COVID-19 pandemic has disrupted and delayed medical and surgical examinations where attendance is required in person. Our article aims to outline the validity of online assessment, the range of benefits to both candidate and assessor, and the challenges to its implementation. In addition, we propose pragmatic suggestions for its introduction into medical assessment. METHODS: We reviewed the literature concerning the present status of online medical and surgical assessment to establish the perceived benefits, limitations, and potential problems with this method of assessment. RESULTS: Global experience with online, remote virtual examination has been largely successful with many benefits conferred to the trainee, and both an economic and logistical advantage conferred to the assessor or organization. Advances in online examination software and remote proctoring are overcoming practical caveats including candidate authentication, cheating prevention, cybersecurity, and IT failure. CONCLUSION: Virtual assessment provides benefits to both trainee and assessor in medical and surgical examinations and may also result in cost savings. Virtual assessment is likely to be increasingly used in the post-COVID world and we present recommendations for the continued adoption of virtual examination. It is, however, currently unable to completely replace clinical assessment of trainees. Cite this article: Bone Jt Open 2021;2(2):111-118

The Pros and Cons: Rank-aware Temporal Attention for Skill Determination in Long Videos

We present a new model to determine relative skill from long videos, through learnable temporal attention modules. Skill determination is formulated as a ranking problem, making it suitable for common and generic tasks. However, for long videos, parts of the video are irrelevant for assessing skill, and there may be variability in the skill exhibited throughout a video. We therefore propose a method which assesses the relative overall level of skill in a long video by attending to its skill-relevant parts. Our approach trains temporal attention modules, learned with only video-level supervision, using a novel rank-aware loss function. In addition to attending to task relevant video parts, our proposed loss jointly trains two attention modules to separately attend to video parts which are indicative of higher (pros) and lower (cons) skill. We evaluate our approach on the EPIC-Skills dataset and additionally annotate a larger dataset from YouTube videos for skill determination with five previously unexplored tasks. Our method outperforms previous approaches and classic softmax attention on both datasets by over 4% pairwise accuracy, and as much as 12% on individual tasks. We also demonstrate our model's ability to attend to rank-aware parts of the video.Comment: CVPR 201

Psychomotor learning theory informing the design and evaluation of an interactive augmented reality hand hygiene training app for healthcare workers

Hand hygiene is critical for infection control, but studies report poor transfer from training to practice. Hand hygiene training in hospitals typically involves one classroom session per year, but psychomotor skills require repetition and feedback for retention. We describe the design and independent evaluation of a mobile interactive augmented reality training tool for the World Health Organisation (WHO) hand hygiene technique. The design was based on a detailed analysis of the underlying educational theory relating to psychomotor skills learning. During the evaluation forty-seven subjects used AR hand hygiene training over 4 weeks. Hand hygiene proficiency was assessed at weekly intervals, both electronically and via human inspection. Thirty eight participants (81%) reached proficiency after 24.3 (SD=17.8) two-minute practice sessions. The study demonstrated that interactive mobile applications could empower learners to develop hand hygiene skills independently. Healthcare organizations could improve hand hygiene quality by using self-directed skills-based training combined with regular ward-based assessments