Visual Perception and Cognition in Image-Guided Intervention

Surgical image visualization and interaction systems can dramatically affect the efficacy and efficiency of surgical training, planning, and interventions. This is even more profound in the case of minimally-invasive surgery where restricted access to the operative field in conjunction with limited field of view necessitate a visualization medium to provide patient-specific information at any given moment. Unfortunately, little research has been devoted to studying human factors associated with medical image displays and the need for a robust, intuitive visualization and interaction interfaces has remained largely unfulfilled to this day. Failure to engineer efficient medical solutions and design intuitive visualization interfaces is argued to be one of the major barriers to the meaningful transfer of innovative technology to the operating room. This thesis was, therefore, motivated by the need to study various cognitive and perceptual aspects of human factors in surgical image visualization systems, to increase the efficiency and effectiveness of medical interfaces, and ultimately to improve patient outcomes. To this end, we chose four different minimally-invasive interventions in the realm of surgical training, planning, training for planning, and navigation: The first chapter involves the use of stereoendoscopes to reduce morbidity in endoscopic third ventriculostomy. The results of this study suggest that, compared with conventional endoscopes, the detection of the basilar artery on the surface of the third ventricle can be facilitated with the use of stereoendoscopes, increasing the safety of targeting in third ventriculostomy procedures. In the second chapter, a contour enhancement technique is described to improve preoperative planning of arteriovenous malformation interventions. The proposed method, particularly when combined with stereopsis, is shown to increase the speed and accuracy of understanding the spatial relationship between vascular structures. In the third chapter, an augmented-reality system is proposed to facilitate the training of planning brain tumour resection. The results of our user study indicate that the proposed system improves subjects\u27 performance, particularly novices\u27, in formulating the optimal point of entry and surgical path independent of the sensorimotor tasks performed. In the last chapter, the role of fully-immersive simulation environments on the surgeons\u27 non-technical skills to perform vertebroplasty procedure is investigated. Our results suggest that while training surgeons may increase their technical skills, the introduction of crisis scenarios significantly disturbs the performance, emphasizing the need of realistic simulation environments as part of training curriculum

Virtual Reality Simulator for Training in Myringotomy with Tube Placement

Myringotomy refers to a surgical incision in the eardrum, and it is often followed by ventilation tube placement to treat middle-ear infections. The procedure is difficult to learn; hence, the objectives of this work were to develop a virtual-reality training simulator, assess its face and content validity, and implement quantitative performance metrics and assess construct validity. A commercial digital gaming engine (Unity3D) was used to implement the simulator with support for 3D visualization of digital ear models and support for major surgical tasks. A haptic arm co-located with the stereo scene was used to manipulate virtual surgical tools and to provide force feedback. A questionnaire was developed with 14 face validity questions focusing on realism and 6 content validity questions focusing on training potential. Twelve participants from the Department of Otolaryngology were recruited for the study. Responses to 12 of the 14 face validity questions were positive. One concern was with contact modeling related to tube insertion into the eardrum, and the second was with movement of the blade and forceps. The former could be resolved by using a higher resolution digital model for the eardrum to improve contact localization. The latter could be resolved by using a higher fidelity haptic device. With regard to content validity, 64% of the responses were positive, 21% were neutral, and 15% were negative. In the final phase of this work, automated performance metrics were programmed and a construct validity study was conducted with 11 participants: 4 senior Otolaryngology consultants and 7 junior Otolaryngology residents. Each participant performed 10 procedures on the simulator and metrics were automatically collected. Senior Otolaryngologists took significantly less time to completion compared to junior residents. Junior residents had 2.8 times more errors as compared to experienced surgeons. The senior surgeons also had significantly longer incision lengths, more accurate incision angles, and lower magnification keeping both the umbo and annulus in view. All metrics were able to discriminate senior Otolaryngologists from junior residents with a significance of p \u3c 0.002. The simulator has sufficient realism, training potential and performance discrimination ability to warrant a more resource intensive skills transference study

Perception and Orientation in Minimally Invasive Surgery

During the last two decades, we have seen a revolution in the way that we perform abdominal surgery with increased reliance on minimally invasive techniques. This paradigm shift has come at a rapid pace, with laparoscopic surgery now representing the gold standard for many surgical procedures and further minimisation of invasiveness being seen with the recent clinical introduction of novel techniques such as single-incision laparoscopic surgery and natural orifice translumenal endoscopic surgery. Despite the obvious benefits conferred on the patient in terms of morbidity, length of hospital stay and post-operative pain, this paradigm shift comes at a significantly higher demand on the surgeon, in terms of both perception and manual dexterity. The issues involved include degradation of sensory input to the operator compared to conventional open surgery owing to a loss of three-dimensional vision through the use of the two-dimensional operative interface, and decreased haptic feedback from the instruments. These changes have led to a much higher cognitive load on the surgeon and a greater risk of operator disorientation leading to potential surgical errors. This thesis represents a detailed investigation of disorientation in minimally invasive surgery. In this thesis, eye tracking methodology is identified as the method of choice for evaluating behavioural patterns during orientation. An analysis framework is proposed to profile orientation behaviour using eye tracking data validated in a laboratory model. This framework is used to characterise and quantify successful orientation strategies at critical stages of laparoscopic cholecystectomy and furthermore use these strategies to prove that focused teaching of this behaviour in novices can significantly increase performance in this task. Orientation strategies are then characterised for common clinical scenarios in natural orifice translumenal endoscopic surgery and the concept of image saliency is introduced to further investigate the importance of specific visual cues associated with effective orientation. Profiling of behavioural patterns is related to performance in orientation and implications on education and construction of smart surgical robots are drawn. Finally, a method for potentially decreasing operator disorientation is investigated in the form of endoscopic horizon stabilization in a simulated operative model for transgastric surgery. The major original contributions of this thesis include: Validation of a profiling methodology/framework to characterise orientation behaviour Identification of high performance orientation strategies in specific clinical scenarios including laparoscopic cholecystectomy and natural orifice translumenal endoscopic surgery Evaluation of the efficacy of teaching orientation strategies Evaluation of automatic endoscopic horizon stabilization in natural orifice translumenal endoscopic surgery The impact of the results presented in this thesis, as well as the potential for further high impact research is discussed in the context of both eye tracking as an evaluation tool in minimally invasive surgery as well as implementation of means to combat operator disorientation in a surgical platform. The work also provides further insight into the practical implementation of computer-assistance and technological innovation in future flexible access surgical platforms

The next level : video gaming, cognition and motivation in surgical simulator training

Image guided surgery uses small incisions or existing entry ports of the body in order to decrease surgical trauma which will hopefully lead to less infections, complications and number of days needed in the hospital. It is associated with great difficulties and a steep learning curve due to several reasons such as inverted movements and 2D representation of the operating area. In order to excel in minimal invasive surgery a large amount of training is required which has spurred the rise of virtual reality (VR) surgical simulators offering a safe training environment with the possibility of customized scenarios and quantifiable feedback. Another advantage with VR simulators is easily conducted research due to objective assessment of performance and standardized task content and scenarios. Previous research has identified visual-spatial ability, the ability to mentally rotate and manipulate objects as well as visual working memory, the ability to hold visual information during a delay and recall that information, to be important for simulator training and performance. Video gaming experience has also been identified as an important background factor. This thesis consists of work derived from four studies that all take advantage of VR simulators as a tool for investigating which individual factors are needed for performance and training of minimal invasive surgery and whether or not they can be altered. In the first study the hypothesis was that the importance of visual-spatial ability (VSA) and visual working memory (VWM) would differ with different simulator task content. 25 subjects were tested for VSA, VWM and performance in three different simulators. A multivariate analysis showed that the importance differed; one task seemed to be more visually loaded than the others. That result was followed up in study II that examined whether it was possible to actually improve simulator performance by video game training and if the transfer effect differed according to simulator and video game task content. 30 subjects were matched and randomized into training with a 2D chess game or a 3D first person shooter game for five weeks, pre and post training subjects were tested with two different simulators. A control group consisting of 10 subjects was also tested. There was a transfer effect, surprisingly also from the 2D game. Suspicions about a general cognitive workload lead to the aim in study III that investigated whether simulator performance would predict written examinations results. 158 subjects were tested in a simulator and a written examination in basic surgery. There was a performanceexamination correlation in female but not male subjects, which lead to study IV that investigated the role of motivation for surgical performance. In study IV, 30 subjects were tested for motivation while training in a surgical simulator. Motivation as defined by the selfdetermination theory correlated only in male subjects to performance when highly motivated medical students were examined. It appeared to be less important for performance than visual spatial ability. Training in surgical simulators enhanced subjects’ interest in choosing surgery as a future work field. The thesis identifies the importance of certain background factors and suggests alternate means of minimal invasive training that will meet the requirements of tomorrow´s surgeons, taking surgical training to the next level

Observational Learning During Simulation-Based Training in Arthroscopy: Is It Useful to Novices?

This is the author accepted manuscript. The final version is available from Elsevier via the DOI in this record.OBJECTIVE: Observing experts constitutes an important and common learning experience for surgical residents before operating under direct guidance. However, studies suggest that exclusively observing experts may induce suboptimal motor learning, and watching errors from non-experts performing simple motor tasks may generate better performance. We investigated whether observational learning is transferrable to arthroscopy learning using virtual reality (VR) simulation. SETTING/DESIGN: In our surgical simulation laboratory, we compared students learning basic skills on a VR arthroscopy simulator after watching an expert video demonstration of VR arthroscopy tasks or a non-expert video demonstration of the same tasks to a Control group without video demonstration. Ninety students in 3 observing groups (expert, non-expert, and Control) subsequently completed the same procedure on a VR arthroscopy simulator. We hypothesized the non-expert-watching group would outperform the expert-watching group, and both groups to outperform the Control group. We examined performance pretest, posttest, and 1 week later. PARTICIPANTS: Participants were recruited from the final year of medical school and the very early first year of surgical residency training programs (orthopaedic surgery, urology, plastic surgery, and general surgery) at Western University (Ontario, Canada). RESULTS: All participants improved their overall performance from pretest to retention (p < 0.001). At initial retention testing, non-expert-watching group outperformed the other groups in camera path length p < 0.05 and time to completion, p < 0.05, and both the expert/non-expert groups surpassed the Control group in camera path length (p < 0.05). CONCLUSION: We suggest that error-observation may contribute to skills improvement in the non-expert-watching group. Allowing novices to observe techniques/errors of other novices may assist internalization of specific movements/skills required for effective motor performances. This study highlights the potential effect of observational learning on surgical skills acquisition and offers preliminary evidence for peer-based practice (combined non-experts and experts) as a complementary surgical motor skills training strategy.This project was supported by a Physicians׳ Services Incorporated (PSI) Foundation, Canada grant. Funds were used to pay for salary and employee benefits (LvE). The PSI Foundation did not play a role in the investigation

Microscope Embedded Neurosurgical Training and Intraoperative System

In the recent years, neurosurgery has been strongly influenced by new technologies. Computer Aided Surgery (CAS) offers several benefits for patients\u27 safety but fine techniques targeted to obtain minimally invasive and traumatic treatments are required, since intra-operative false movements can be devastating, resulting in patients deaths. The precision of the surgical gesture is related both to accuracy of the available technological instruments and surgeon\u27s experience. In this frame, medical training is particularly important. From a technological point of view, the use of Virtual Reality (VR) for surgeon training and Augmented Reality (AR) for intra-operative treatments offer the best results. In addition, traditional techniques for training in surgery include the use of animals, phantoms and cadavers. The main limitation of these approaches is that live tissue has different properties from dead tissue and that animal anatomy is significantly different from the human. From the medical point of view, Low-Grade Gliomas (LGGs) are intrinsic brain tumours that typically occur in younger adults. The objective of related treatment is to remove as much of the tumour as possible while minimizing damage to the healthy brain. Pathological tissue may closely resemble normal brain parenchyma when looked at through the neurosurgical microscope. The tactile appreciation of the different consistency of the tumour compared to normal brain requires considerable experience on the part of the neurosurgeon and it is a vital point. The first part of this PhD thesis presents a system for realistic simulation (visual and haptic) of the spatula palpation of the LGG. This is the first prototype of a training system using VR, haptics and a real microscope for neurosurgery. This architecture can be also adapted for intra-operative purposes. In this instance, a surgeon needs the basic setup for the Image Guided Therapy (IGT) interventions: microscope, monitors and navigated surgical instruments. The same virtual environment can be AR rendered onto the microscope optics. The objective is to enhance the surgeon\u27s ability for a better intra-operative orientation by giving him a three-dimensional view and other information necessary for a safe navigation inside the patient. The last considerations have served as motivation for the second part of this work which has been devoted to improving a prototype of an AR stereoscopic microscope for neurosurgical interventions, developed in our institute in a previous work. A completely new software has been developed in order to reuse the microscope hardware, enhancing both rendering performances and usability. Since both AR and VR share the same platform, the system can be referred to as Mixed Reality System for neurosurgery. All the components are open source or at least based on a GPL license

The Next-Generation Surgical Robots

The chronicle of surgical robots is short but remarkable. Within 20 years since the regulatory approval of the first surgical robot, more than 3,000 units were installed worldwide, and more than half a million robotic surgical procedures were carried out in the past year alone. The exceptionally high speeds of market penetration and expansion to new surgical areas had raised technical, clinical, and ethical concerns. However, from a technological perspective, surgical robots today are far from perfect, with a list of improvements expected for the next-generation systems. On the other hand, robotic technologies are flourishing at ever-faster paces. Without the inherent conservation and safety requirements in medicine, general robotic research could be substantially more agile and explorative. As a result, various technical innovations in robotics developed in recent years could potentially be grafted into surgical applications and ignite the next major advancement in robotic surgery. In this article, the current generation of surgical robots is reviewed from a technological point of view, including three of possibly the most debated technical topics in surgical robotics: vision, haptics, and accessibility. Further to that, several emerging robotic technologies are highlighted for their potential applications in next-generation robotic surgery

Performance Factors in Neurosurgical Simulation and Augmented Reality Image Guidance

Virtual reality surgical simulators have seen widespread adoption in an effort to provide safe, cost-effective and realistic practice of surgical skills. However, the majority of these simulators focus on training low-level technical skills, providing only prototypical surgical cases. For many complex procedures, this approach is deficient in representing anatomical variations that present clinically, failing to challenge users’ higher-level cognitive skills important for navigation and targeting. Surgical simulators offer the means to not only simulate any case conceivable, but to test novel approaches and examine factors that influence performance. Unfortunately, there is a void in the literature surrounding these questions. This thesis was motivated by the need to expand the role of surgical simulators to provide users with clinically relevant scenarios and evaluate human performance in relation to image guidance technologies, patient-specific anatomy, and cognitive abilities. To this end, various tools and methodologies were developed to examine cognitive abilities and knowledge, simulate procedures, and guide complex interventions all within a neurosurgical context. The first chapter provides an introduction to the material. The second chapter describes the development and evaluation of a virtual anatomical training and examination tool. The results suggest that learning occurs and that spatial reasoning ability is an important performance predictor, but subordinate to anatomical knowledge. The third chapter outlines development of automation tools to enable efficient simulation studies and data management. In the fourth chapter, subjects perform abstract targeting tasks on ellipsoid targets with and without augmented reality guidance. While the guidance tool improved accuracy, performance with the tool was strongly tied to target depth estimation – an important consideration for implementation and training with similar guidance tools. In the fifth chapter, neurosurgically experienced subjects were recruited to perform simulated ventriculostomies. Results showed anatomical variations influence performance and could impact outcome. Augmented reality guidance showed no marked improvement in performance, but exhibited a mild learning curve, indicating that additional training may be warranted. The final chapter summarizes the work presented. Our results and novel evaluative methodologies lay the groundwork for further investigation into simulators as versatile research tools to explore performance factors in simulated surgical procedures

The assessment of visual behaviour and depth perception in surgery

Imperial Users onl