Automated Objective Surgical Skill Assessment and Visualization in the Operating Room Using Unstructured Tool Motion for Improved Surgical Training

Past attempts at surgical skill assessment using tool motion in the operating room have focused on highly-structured surgical tasks such as suturing. These methods considered only generic descriptive metrics such as the operating time and the number of movements made, which are of limited instructional value. In this thesis, we develop and evaluate an automated method of surgical skill assessment of flap elevation in nasal septoplasty in the operating room. The obstructed field of view and highly unstructured nature of septoplasty hinders trainees from efficiently learning how to effectively perform the procedure. Thus, we also present the development of a real-time visualization system that allows trainees and instructors to better observe tool motion with respect to patient anatomy during the operation. In this work, we propose a descriptive structure of septoplasty that consists of the following two activity types: (1) the brushing activity directed away from the septum plane that characterizes the consistency of the surgeon’s wrist motion and (2) the activity along the septal plane that characterizes the surgeon’s coverage pattern. We computed features related to these activity types that allow classification of a surgeon’s level of training with an average accuracy of about 72%. Further, as opposed to previously-measured generic motion metrics, the presented features provide surgeons with personalized, actionable feedback regarding their tool motion

Surgical Skill Assessment on In-Vivo Clinical Data via the Clearness of Operating Field

Surgical skill assessment is important for surgery training and quality control. Prior works on this task largely focus on basic surgical tasks such as suturing and knot tying performed in simulation settings. In contrast, surgical skill assessment is studied in this paper on a real clinical dataset, which consists of fifty-seven in-vivo laparoscopic surgeries and corresponding skill scores annotated by six surgeons. From analyses on this dataset, the clearness of operating field (COF) is identified as a good proxy for overall surgical skills, given its strong correlation with overall skills and high inter-annotator consistency. Then an objective and automated framework based on neural network is proposed to predict surgical skills through the proxy of COF. The neural network is jointly trained with a supervised regression loss and an unsupervised rank loss. In experiments, the proposed method achieves 0.55 Spearman's correlation with the ground truth of overall technical skill, which is even comparable with the human performance of junior surgeons.Comment: MICCAI 201

Technical skill assessment in minimally invasive surgery using artificial intelligence: a systematic review.

BACKGROUND Technical skill assessment in surgery relies on expert opinion. Therefore, it is time-consuming, costly, and often lacks objectivity. Analysis of intraoperative data by artificial intelligence (AI) has the potential for automated technical skill assessment. The aim of this systematic review was to analyze the performance, external validity, and generalizability of AI models for technical skill assessment in minimally invasive surgery. METHODS A systematic search of Medline, Embase, Web of Science, and IEEE Xplore was performed to identify original articles reporting the use of AI in the assessment of technical skill in minimally invasive surgery. Risk of bias (RoB) and quality of the included studies were analyzed according to Quality Assessment of Diagnostic Accuracy Studies criteria and the modified Joanna Briggs Institute checklists, respectively. Findings were reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement. RESULTS In total, 1958 articles were identified, 50 articles met eligibility criteria and were analyzed. Motion data extracted from surgical videos (n = 25) or kinematic data from robotic systems or sensors (n = 22) were the most frequent input data for AI. Most studies used deep learning (n = 34) and predicted technical skills using an ordinal assessment scale (n = 36) with good accuracies in simulated settings. However, all proposed models were in development stage, only 4 studies were externally validated and 8 showed a low RoB. CONCLUSION AI showed good performance in technical skill assessment in minimally invasive surgery. However, models often lacked external validity and generalizability. Therefore, models should be benchmarked using predefined performance metrics and tested in clinical implementation studies

A Survey on the Current Status and Future Challenges Towards Objective Skills Assessment in Endovascular Surgery

Minimally-invasive endovascular interventions have evolved rapidly over the past decade, facilitated by breakthroughs in medical imaging and sensing, instrumentation and most recently robotics. Catheter based operations are potentially safer and applicable to a wider patient population due to the reduced comorbidity. As a result endovascular surgery has become the preferred treatment option for conditions previously treated with open surgery and as such the number of patients undergoing endovascular interventions is increasing every year. This fact coupled with a proclivity for reduced working hours, results in a requirement for efficient training and assessment of new surgeons, that deviates from the “see one, do one, teach one” model introduced by William Halsted, so that trainees obtain operational expertise in a shorter period. Developing more objective assessment tools based on quantitative metrics is now a recognised need in interventional training and this manuscript reports the current literature for endovascular skills assessment and the associated emerging technologies. A systematic search was performed on PubMed (MEDLINE), Google Scholar, IEEXplore and known journals using the keywords, “endovascular surgery”, “surgical skills”, “endovascular skills”, “surgical training endovascular” and “catheter skills”. Focusing explicitly on endovascular surgical skills, we group related works into three categories based on the metrics used; structured scales and checklists, simulation-based and motion-based metrics. This review highlights the key findings in each category and also provides suggestions for new research opportunities towards fully objective and automated surgical assessment solutions

Automated robot‐assisted surgical skill evaluation: Predictive analytics approach

BackgroundSurgical skill assessment has predominantly been a subjective task. Recently, technological advances such as robot‐assisted surgery have created great opportunities for objective surgical evaluation. In this paper, we introduce a predictive framework for objective skill assessment based on movement trajectory data. Our aim is to build a classification framework to automatically evaluate the performance of surgeons with different levels of expertise.MethodsEight global movement features are extracted from movement trajectory data captured by a da Vinci robot for surgeons with two levels of expertise – novice and expert. Three classification methods – k‐nearest neighbours, logistic regression and support vector machines – are applied.ResultsThe result shows that the proposed framework can classify surgeons’ expertise as novice or expert with an accuracy of 82.3% for knot tying and 89.9% for a suturing task.ConclusionThis study demonstrates and evaluates the ability of machine learning methods to automatically classify expert and novice surgeons using global movement features.Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/141457/1/rcs1850.pdfhttps://deepblue.lib.umich.edu/bitstream/2027.42/141457/2/rcs1850_am.pd

Evolving robotic surgery training and improving patient safety, with the integration of novel technologies

INTRODUCTION: Robot-assisted surgery is becoming increasingly adopted by multiple surgical specialties. There is evidence of inherent risks of utilising new technologies that are unfamiliar early in the learning curve. The development of standardised and validated training programmes is crucial to deliver safe introduction. In this review, we aim to evaluate the current evidence and opportunities to integrate novel technologies into modern digitalised robotic training curricula. METHODS: A systematic literature review of the current evidence for novel technologies in surgical training was conducted online and relevant publications and information were identified. Evaluation was made on how these technologies could further enable digitalisation of training. RESULTS: Overall, the quality of available studies was found to be low with current available evidence consisting largely of expert opinion, consensus statements and small qualitative studies. The review identified that there are several novel technologies already being utilised in robotic surgery training. There is also a trend towards standardised validated robotic training curricula. Currently, the majority of the validated curricula do not incorporate novel technologies and training is delivered with more traditional methods that includes centralisation of training services with wet laboratories that have access to cadavers and dedicated training robots. CONCLUSIONS: Improvements to training standards and understanding performance data have good potential to significantly lower complications in patients. Digitalisation automates data collection and brings data together for analysis. Machine learning has potential to develop automated performance feedback for trainees. Digitalised training aims to build on the current gold standards and to further improve the 'continuum of training' by integrating PBP training, 3D-printed models, telementoring, telemetry and machine learning

Objective and automated assessment of surgical technical skills with IoT systems: A systematic literature review

The assessment of surgical technical skills to be acquired by novice surgeons has been traditionally done by an expert surgeon and is therefore of a subjective nature. Nevertheless, the recent advances on IoT, the possibility of incorporating sensors into objects and environments in order to collect large amounts of data, and the progress on machine learning are facilitating a more objective and automated assessment of surgical technical skills. This paper presents a systematic literature review of papers published after 2013 discussing the objective and automated assessment of surgical technical skills. 101 out of an initial list of 537 papers were analyzed to identify: 1) the sensors used; 2) the data collected by these sensors and the relationship between these data, surgical technical skills and surgeons' levels of expertise; 3) the statistical methods and algorithms used to process these data; and 4) the feedback provided based on the outputs of these statistical methods and algorithms. Particularly, 1) mechanical and electromagnetic sensors are widely used for tool tracking, while inertial measurement units are widely used for body tracking; 2) path length, number of sub-movements, smoothness, fixation, saccade and total time are the main indicators obtained from raw data and serve to assess surgical technical skills such as economy, efficiency, hand tremor, or mind control, and distinguish between two or three levels of expertise (novice/intermediate/advanced surgeons); 3) SVM (Support Vector Machines) and Neural Networks are the preferred statistical methods and algorithms for processing the data collected, while new opportunities are opened up to combine various algorithms and use deep learning; and 4) feedback is provided by matching performance indicators and a lexicon of words and visualizations, although there is considerable room for research in the context of feedback and visualizations, taking, for example, ideas from learning analytics.This work was supported in part by the FEDER/Ministerio de Ciencia, Innovación y Universidades;Agencia Estatal de Investigación, through the Smartlet Project under Grant TIN2017-85179-C3-1-R, and in part by the Madrid Regional Government through the e-Madrid-CM Project under Grant S2018/TCS-4307, a project which is co-funded by the European Structural Funds (FSE and FEDER). Partial support has also been received from the European Commission through Erasmus + Capacity Building in the Field of Higher Education projects, more specifically through projects LALA (586120-EPP-1-2017-1-ES-EPPKA2-CBHE-JP), InnovaT (598758-EPP-1-2018-1-AT-EPPKA2-CBHE-JP), and PROF-XXI (609767-EPP-1-2019-1-ES-EPPKA2-CBHE-JP)