Automated Objective Surgical Skill Assessment and Visualization in the Operating Room Using Unstructured Tool Motion for Improved Surgical Training

Past attempts at surgical skill assessment using tool motion in the operating room have focused on highly-structured surgical tasks such as suturing. These methods considered only generic descriptive metrics such as the operating time and the number of movements made, which are of limited instructional value. In this thesis, we develop and evaluate an automated method of surgical skill assessment of flap elevation in nasal septoplasty in the operating room. The obstructed field of view and highly unstructured nature of septoplasty hinders trainees from efficiently learning how to effectively perform the procedure. Thus, we also present the development of a real-time visualization system that allows trainees and instructors to better observe tool motion with respect to patient anatomy during the operation. In this work, we propose a descriptive structure of septoplasty that consists of the following two activity types: (1) the brushing activity directed away from the septum plane that characterizes the consistency of the surgeon’s wrist motion and (2) the activity along the septal plane that characterizes the surgeon’s coverage pattern. We computed features related to these activity types that allow classification of a surgeon’s level of training with an average accuracy of about 72%. Further, as opposed to previously-measured generic motion metrics, the presented features provide surgeons with personalized, actionable feedback regarding their tool motion

Surgical Skill Assessment on In-Vivo Clinical Data via the Clearness of Operating Field

Surgical skill assessment is important for surgery training and quality control. Prior works on this task largely focus on basic surgical tasks such as suturing and knot tying performed in simulation settings. In contrast, surgical skill assessment is studied in this paper on a real clinical dataset, which consists of fifty-seven in-vivo laparoscopic surgeries and corresponding skill scores annotated by six surgeons. From analyses on this dataset, the clearness of operating field (COF) is identified as a good proxy for overall surgical skills, given its strong correlation with overall skills and high inter-annotator consistency. Then an objective and automated framework based on neural network is proposed to predict surgical skills through the proxy of COF. The neural network is jointly trained with a supervised regression loss and an unsupervised rank loss. In experiments, the proposed method achieves 0.55 Spearman's correlation with the ground truth of overall technical skill, which is even comparable with the human performance of junior surgeons.Comment: MICCAI 201

Automated robot‐assisted surgical skill evaluation: Predictive analytics approach

BackgroundSurgical skill assessment has predominantly been a subjective task. Recently, technological advances such as robot‐assisted surgery have created great opportunities for objective surgical evaluation. In this paper, we introduce a predictive framework for objective skill assessment based on movement trajectory data. Our aim is to build a classification framework to automatically evaluate the performance of surgeons with different levels of expertise.MethodsEight global movement features are extracted from movement trajectory data captured by a da Vinci robot for surgeons with two levels of expertise – novice and expert. Three classification methods – k‐nearest neighbours, logistic regression and support vector machines – are applied.ResultsThe result shows that the proposed framework can classify surgeons’ expertise as novice or expert with an accuracy of 82.3% for knot tying and 89.9% for a suturing task.ConclusionThis study demonstrates and evaluates the ability of machine learning methods to automatically classify expert and novice surgeons using global movement features.Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/141457/1/rcs1850.pdfhttps://deepblue.lib.umich.edu/bitstream/2027.42/141457/2/rcs1850_am.pd

A Survey on the Current Status and Future Challenges Towards Objective Skills Assessment in Endovascular Surgery

Minimally-invasive endovascular interventions have evolved rapidly over the past decade, facilitated by breakthroughs in medical imaging and sensing, instrumentation and most recently robotics. Catheter based operations are potentially safer and applicable to a wider patient population due to the reduced comorbidity. As a result endovascular surgery has become the preferred treatment option for conditions previously treated with open surgery and as such the number of patients undergoing endovascular interventions is increasing every year. This fact coupled with a proclivity for reduced working hours, results in a requirement for efficient training and assessment of new surgeons, that deviates from the “see one, do one, teach one” model introduced by William Halsted, so that trainees obtain operational expertise in a shorter period. Developing more objective assessment tools based on quantitative metrics is now a recognised need in interventional training and this manuscript reports the current literature for endovascular skills assessment and the associated emerging technologies. A systematic search was performed on PubMed (MEDLINE), Google Scholar, IEEXplore and known journals using the keywords, “endovascular surgery”, “surgical skills”, “endovascular skills”, “surgical training endovascular” and “catheter skills”. Focusing explicitly on endovascular surgical skills, we group related works into three categories based on the metrics used; structured scales and checklists, simulation-based and motion-based metrics. This review highlights the key findings in each category and also provides suggestions for new research opportunities towards fully objective and automated surgical assessment solutions

Objective and automated assessment of surgical technical skills with IoT systems: A systematic literature review

The assessment of surgical technical skills to be acquired by novice surgeons has been traditionally done by an expert surgeon and is therefore of a subjective nature. Nevertheless, the recent advances on IoT, the possibility of incorporating sensors into objects and environments in order to collect large amounts of data, and the progress on machine learning are facilitating a more objective and automated assessment of surgical technical skills. This paper presents a systematic literature review of papers published after 2013 discussing the objective and automated assessment of surgical technical skills. 101 out of an initial list of 537 papers were analyzed to identify: 1) the sensors used; 2) the data collected by these sensors and the relationship between these data, surgical technical skills and surgeons' levels of expertise; 3) the statistical methods and algorithms used to process these data; and 4) the feedback provided based on the outputs of these statistical methods and algorithms. Particularly, 1) mechanical and electromagnetic sensors are widely used for tool tracking, while inertial measurement units are widely used for body tracking; 2) path length, number of sub-movements, smoothness, fixation, saccade and total time are the main indicators obtained from raw data and serve to assess surgical technical skills such as economy, efficiency, hand tremor, or mind control, and distinguish between two or three levels of expertise (novice/intermediate/advanced surgeons); 3) SVM (Support Vector Machines) and Neural Networks are the preferred statistical methods and algorithms for processing the data collected, while new opportunities are opened up to combine various algorithms and use deep learning; and 4) feedback is provided by matching performance indicators and a lexicon of words and visualizations, although there is considerable room for research in the context of feedback and visualizations, taking, for example, ideas from learning analytics.This work was supported in part by the FEDER/Ministerio de Ciencia, Innovación y Universidades;Agencia Estatal de Investigación, through the Smartlet Project under Grant TIN2017-85179-C3-1-R, and in part by the Madrid Regional Government through the e-Madrid-CM Project under Grant S2018/TCS-4307, a project which is co-funded by the European Structural Funds (FSE and FEDER). Partial support has also been received from the European Commission through Erasmus + Capacity Building in the Field of Higher Education projects, more specifically through projects LALA (586120-EPP-1-2017-1-ES-EPPKA2-CBHE-JP), InnovaT (598758-EPP-1-2018-1-AT-EPPKA2-CBHE-JP), and PROF-XXI (609767-EPP-1-2019-1-ES-EPPKA2-CBHE-JP)

Eye-Tracking in the Study of Visual Expertise: Methodology and Approaches in Medicine

Eye-tracking is the measurement of eye motions and point of gaze of a viewer. Advances in this technology have been essential to our understanding of many forms of visual learning, including the development of visual expertise. In recent years, these studies have been extended to the medical professions, where eye-tracking technology has helped us to understand acquired visual expertise, as well as the importance of visual training in various medical specialties. Medical decision-making involves a complex interplay between knowledge and sensory information, and the study of eye-movements can reveal the mechanisms involved in acquiring the visual component of these skills. Eye-tracking studies have even been extended to develop computational models of procedures for “expert” skill assessment, and to eliminate potential sources of error in image-based diagnostics. This review will examine the current eye-tracking frontier for the study of visual expertise, with specific application to medical professions

Energy-based metrics for arthroscopic skills assessment

Minimally invasive skills assessment methods are essential in developing efficient surgical simulators and implementing consistent skills evaluation. Although numerous methods have been investigated in the literature, there is still a need to further improve the accuracy of surgical skills assessment. Energy expenditure can be an indication of motor skills proficiency. The goals of this study are to develop objective metrics based on energy expenditure, normalize these metrics, and investigate classifying trainees using these metrics. To this end, different forms of energy consisting of mechanical energy and work were considered and their values were divided by the related value of an ideal performance to develop normalized metrics. These metrics were used as inputs for various machine learning algorithms including support vector machines (SVM) and neural networks (NNs) for classification. The accuracy of the combination of the normalized energy-based metrics with these classifiers was evaluated through a leave-one-subject-out cross-validation. The proposed method was validated using 26 subjects at two experience levels (novices and experts) in three arthroscopic tasks. The results showed that there are statistically significant differences between novices and experts for almost all of the normalized energy-based metrics. The accuracy of classification using SVM and NN methods was between 70% and 95% for the various tasks. The results show that the normalized energy-based metrics and their combination with SVM and NN classifiers are capable of providing accurate classification of trainees. The assessment method proposed in this study can enhance surgical training by providing appropriate feedback to trainees about their level of expertise and can be used in the evaluation of proficiency

Développement et validation d’un outil d’évaluation de la compétence chirurgicale pour l’évidement cervical

Objectifs : Développer et valider un outil servant à évaluer les compétences chirurgicales des résidents en Oto-rhino-laryngologie et chirurgie cervico-faciale (ORL-CCF) pour l’évidement cervical. Méthodes : Une revue systématique de la littérature a été réalisée afin de recenser les méthodes validées pour évaluer les compétences chirurgicales en ORL-CCF. Les procédures chirurgicales pour lesquelles elles ont été développées ont été répertoriées. La revue a permis d’identifier un déficit en outils d’évaluation spécifiques aux procédures oncologiques en ORL-CCF, incluant l’évidement cervical. Une méthode de Delphi modifiée a été utilisée afin de développer une liste d’étapes essentielles à la complétion d’un évidement cervical. Cette liste a été combinée à la grille générale du Objective Structured Assessment of Technical Skills (OSATS) pour développer un outil spécifique à l’évidement cervical. Cet outil a été validé en salle d’opération auprès de l’équipe d’oncologie ORL-CCF de l’Université de Montréal et de ses résidents. Résultats : Un total de vingt-neuf évaluations ont été complétées au cours de l’année académique 2016-2017. L’acceptabilité a été jugée élevée auprès des résidents et des chirurgiens d’ORL-CCF, avec pour seule disparité l’utilisation formative ou sommative de l’outil. Les études de validation ont démontré des scores significativement plus élevés chez les résidents séniors que chez les juniors, ainsi qu’une progression significative des scores au fil du temps (p<0,05). La tendance des scores sur la grille spécifique corrélait avec les résultats obtenus sur la grille générale précédemment validée (p<0,05). ii Conclusions : Le premier outil évaluant spécifiquement les compétences chirurgicales des résidents en ORL pour l’évidement cervical a été développé et validé.Objectives: To develop and validate a new tool assessing surgical competency for Otolaryngology – Head & Neck Surgery (OTL-HNS) residents learning neck dissection. Methods: A systematic review of literature was done to list methods developed to assess surgical competency in OTL-HNS. The surgical procedures for which these tools were developed were catalogued. A lack of evaluation tools specific to oncologic procedures in OTLHNS was identified, which includes neck dissection. A modified Delphi method was used to develop a list of steps deemed essential to a group of experts in order to complete a neck dissection. This list was combined to the general list of the Objective Structured Assessment of Technical Skills (OSATS) tool to develop a tool specific to neck dissection. This tool was validated in the operating room with the collaboration of the OTL-HNS oncology team of Université de Montréal and of the residents of this program. Results: A total of twenty-nine evaluations were completed throughout the 2016-2017 academic year. Acceptability ranked high for both residents and staff, with a single discrepancy in responses regarding a potential formative as opposed to summative use of the tool. Validation study results demonstrated significantly higher checklist scores for senior residents as opposed to junior residents, as well as a significant score progression over time (p<0,05). Trends in scores on the task-specific tool correlated highly to results obtained on a validated global rating scale (p<0,05). Conclusion: The first tool assessing surgical competency in OTL-HNS residents for neck dissection was successfully developed and validated