Augmented reality in spinal surgery: Highlights from augmented reality lectures at the Emerging Technologies annual meetings

Introduction Augmented reality (AR) is an advanced technology and emerging field that has been adopted into spine surgery to enhance care and outcomes. AR superimposes a three-dimensional computer-generated image over the normal anatomy of interest in order to facilitate visualization of deep structures without the ability to directly see them. Objective To summarize the latest literature and highlight AR from the annual Spinal Navigation, Emerging Technologies and Systems Integration meeting lectures presented by the Seattle Science Foundation (SSF) on the development and use of augmented reality in spinal surgery. Methods We performed a comprehensive literature review from 2016 to 2020 on PubMed to correlate with lectures given at the annual Emerging Technologies conferences. After the exclusion of papers that concerned non-spine surgery specialties, a total of 54 papers concerning AR in spinal applications were found. The articles were then categorized by content and focus. Results The 54 papers were divided into six major focused topics: training, proof of concept, feasibility and usability, clinical evaluation, state of technology, and nonsurgical applications. The greatest number of papers were published during 2020. Each paper discussed varied topics such as patient rehabilitation, proof of concept, workflow, applications in neurological and orthopedic spine surgery, and outcomes data. Conclusions The recent literature and SSF lectures on AR provide a solid base and demonstrate the emergence of an advanced technology that offers a platform for an advantageous technique that is superior, in that it allows the operating surgeon to focus directly on the patient rather than a guidance screen

Development and Validation of a Hybrid Virtual/Physical Nuss Procedure Surgical Trainer

With continuous advancements and adoption of minimally invasive surgery, proficiency with nontrivial surgical skills involved is becoming a greater concern. Consequently, the use of surgical simulation has been increasingly embraced by many for training and skill transfer purposes. Some systems utilize haptic feedback within a high-fidelity anatomically-correct virtual environment whereas others use manikins, synthetic components, or box trainers to mimic primary components of a corresponding procedure. Surgical simulation development for some minimally invasive procedures is still, however, suboptimal or otherwise embryonic. This is true for the Nuss procedure, which is a minimally invasive surgery for correcting pectus excavatum (PE) – a congenital chest wall deformity. This work aims to address this gap by exploring the challenges of developing both a purely virtual and a purely physical simulation platform of the Nuss procedure and their implications in a training context. This work then describes the development of a hybrid mixed-reality system that integrates virtual and physical constituents as well as an augmentation of the haptic interface, to carry out a reproduction of the primary steps of the Nuss procedure and satisfy clinically relevant prerequisites for its training platform. Furthermore, this work carries out a user study to investigate the system’s face, content, and construct validity to establish its faithfulness as a training platform

Mixed Reality in Modern Surgical and Interventional Practice: Narrative Review of the Literature

BACKGROUND Mixed reality (MR) and its potential applications have gained increasing interest within the medical community over the recent years. The ability to integrate virtual objects into a real-world environment within a single video-see-through display is a topic that sparks imagination. Given these characteristics, MR could facilitate preoperative and preinterventional planning, provide intraoperative and intrainterventional guidance, and aid in education and training, thereby improving the skills and merits of surgeons and residents alike. OBJECTIVE In this narrative review, we provide a broad overview of the different applications of MR within the entire spectrum of surgical and interventional practice and elucidate on potential future directions. METHODS A targeted literature search within the PubMed, Embase, and Cochrane databases was performed regarding the application of MR within surgical and interventional practice. Studies were included if they met the criteria for technological readiness level 5, and as such, had to be validated in a relevant environment. RESULTS A total of 57 studies were included and divided into studies regarding preoperative and interventional planning, intraoperative and interventional guidance, as well as training and education. CONCLUSIONS The overall experience with MR is positive. The main benefits of MR seem to be related to improved efficiency. Limitations primarily seem to be related to constraints associated with head-mounted display. Future directions should be aimed at improving head-mounted display technology as well as incorporation of MR within surgical microscopes, robots, and design of trials to prove superiority

Endoscopy

Endoscopy is a minimally invasive diagnostic medical procedure that is used to assess the interior surfaces of an organ by inserting a tube into the body. The instrument may have a rigid or flexible tube and not only provide an image for visual inspection and photography, but also enable taking biopsies and retrieval of foreign objects. Endoscopy is the vehicle for minimally invasive surgery and patients may receive conscious sedation so they do not have to be consciously aware of the discomfort. Many endoscopic procedures are considered to be relatively painless and, at worst, associated with moderate discomfort

Virtual and Augmented Reality in Medical Education

Virtual reality (VR) and augmented reality (AR) are two contemporary simulation models that are currently upgrading medical education. VR provides a 3D and dynamic view of structures and the ability of the user to interact with them. The recent technological advances in haptics, display systems, and motion detection allow the user to have a realistic and interactive experience, enabling VR to be ideal for training in hands-on procedures. Consequently, surgical and other interventional procedures are the main fields of application of VR. AR provides the ability of projecting virtual information and structures over physical objects, thus enhancing or altering the real environment. The integration of AR applications in the understanding of anatomical structures and physiological mechanisms seems to be beneficial. Studies have tried to demonstrate the validity and educational effect of many VR and AR applications, in many different areas, employed via various hardware platforms. Some of them even propose a curriculum that integrates these methods. This chapter provides a brief history of VR and AR in medicine, as well as the principles and standards of their function. Finally, the studies that show the effect of the implementation of these methods in different fields of medical training are summarized and presented

Tactile Sensing System for Lung Tumour Localization during Minimally Invasive Surgery

Video-assisted thoracoscopie surgery (VATS) is becoming a prevalent method for lung cancer treatment. However, VATS suffers from the inability to accurately relay haptic information to the surgeon, often making tumour localization difficult. This limitation was addressed by the design of a tactile sensing system (TSS) consisting of a probe with a tactile sensor and interfacing visualization software. In this thesis, TSS performance was tested to determine the feasibility of implementing the system in VATS. This was accomplished through a series of ex vivo experiments in which the tactile sensor was calibrated and the visualization software was modified to provide haptic information visually to the user, and TSS performance was compared using human and robot palpation methods, and conventional VATS instruments. It was concluded that the device offers the possibility of providing to the surgeon the haptic information lost during surgery, thereby mitigating one of the current limitations of VATS

The Role and Future of Endoscopic Spine Surgery: A Narrative Review

Many types of surgeries are changing from conventional to minimally invasive techniques. Techniques in spine surgery have also changed, with endoscopic spine surgery (ESS) becoming a major surgical technique. Although ESS has advantages such as less soft tissue dissection and normal structure damage, reduced blood loss, less epidural scarring, reduced hospital stay, and earlier functional recovery, it is not possible to replace all spine surgery techniques with ESS. ESS was first used for discectomy in the lumbar spine, but the range of ESS has expanded to cover the entire spine, including the cervical and thoracic spine. With improvements in ESS instruments (optics, endoscope, endoscopic drill and shaver, irrigation pump, and multiportal endoscopic), limitations of ESS have gradually decreased, and it is possible to apply ESS to more spine pathologies. ESS currently incorporates new technologies, such as navigation, augmented and virtual reality, robotics, and 3-dimentional and ultraresolution visualization, to innovate and improve outcomes. In this article, we review the history and current status of ESS, and discuss future goals and possibilities for ESS through comparisons with conventional surgical techniques.ope

AUGMENTED REALITY AND INTRAOPERATIVE C-ARM CONE-BEAM COMPUTED TOMOGRAPHY FOR IMAGE-GUIDED ROBOTIC SURGERY

Minimally-invasive robotic-assisted surgery is a rapidly-growing alternative to traditionally open and laparoscopic procedures; nevertheless, challenges remain. Standard of care derives surgical strategies from preoperative volumetric data (i.e., computed tomography (CT) and magnetic resonance (MR) images) that benefit from the ability of multiple modalities to delineate different anatomical boundaries. However, preoperative images may not reflect a possibly highly deformed perioperative setup or intraoperative deformation. Additionally, in current clinical practice, the correspondence of preoperative plans to the surgical scene is conducted as a mental exercise; thus, the accuracy of this practice is highly dependent on the surgeon’s experience and therefore subject to inconsistencies. In order to address these fundamental limitations in minimally-invasive robotic surgery, this dissertation combines a high-end robotic C-arm imaging system and a modern robotic surgical platform as an integrated intraoperative image-guided system. We performed deformable registration of preoperative plans to a perioperative cone-beam computed tomography (CBCT), acquired after the patient is positioned for intervention. From the registered surgical plans, we overlaid critical information onto the primary intraoperative visual source, the robotic endoscope, by using augmented reality. Guidance afforded by this system not only uses augmented reality to fuse virtual medical information, but also provides tool localization and other dynamic intraoperative updated behavior in order to present enhanced depth feedback and information to the surgeon. These techniques in guided robotic surgery required a streamlined approach to creating intuitive and effective human-machine interferences, especially in visualization. Our software design principles create an inherently information-driven modular architecture incorporating robotics and intraoperative imaging through augmented reality. The system's performance is evaluated using phantoms and preclinical in-vivo experiments for multiple applications, including transoral robotic surgery, robot-assisted thoracic interventions, and cocheostomy for cochlear implantation. The resulting functionality, proposed architecture, and implemented methodologies can be further generalized to other C-arm-based image guidance for additional extensions in robotic surgery

Expanded Indications of Full Endoscopic Spine Sugery

Treatment of spine surgeries has evolved from traditional surgeries to open surgeries. Endoscopic spine surgeries (ESS) and endoscope assisted surgeries along with microscopic and tubular surgeries has developed significantly over the last three decades. With improvement in the diagnostic methods it is now possible to find and differentiate the spinal pathologies. ESS was initially limited to the lumbar disc herniations (DH). But, now it can be used for cervical and thoracic DH. Minimized technical problems has been brought by evolutions in endoscopy, better optics, instruments, access, and safety. Similarly acquired knowledge and skills are being extrapolated to advanced indications in different spinal pathologies. Due to the further advantages of ESS within the ambit of minimal invasive spine surgeries, many misnomers are as well getting added. This confuses the new learners and potential patients as well. ESS should be classified for uniformity in reporting and common nomenclature like FESS (Full endoscopic spine surgery) should be used. It specifically refers to surgery through one working channel under irrigation with incorporated optics. This will make easier understanding for novice surgeons and general population. It will lead to standardised reporting of high quality clinical studies, trials, and meta-analysis for the publications. Rising misnomers and complex nomenclature of endoscopy is suggesting along with the exponential publications in last decade that ESS is entering into its golden era. This review is undertaken to throw light on the techniques, advances and literature review of only FESS and clear the misnomers. This review also describes the evolution of different techniques and goals that led to impeccable advances in the field of FESS. Further improvement of technologies and techniques in future will soon establish FESS as the Gold Standard in spine surgery

Construction d'un modèle per-opératoire 3D du rachis pour la navigation en thoracoscopie.

RÉSUMÉ: Lors de discectomie par thoracoscopie, les outils de visualisation procurent peu d’information de profondeur et le champ de visualisation de la caméra miniature insérée dans le patient est relativement restreint. Aussi, le mouvement simultané de la caméra et des instruments chirurgicaux peut provoquer une désorientation. Ainsi, la courbe d’apprentissage pour l’utilisation de cette technologie est très abrupte et un nombre restreint de chirurgiens choisissent l’intervention minimalement invasive malgré les avantages qu’elle peut procurer aux patients. En effet la discectomie par thoracoscopie réduit les pertes sanguines, le traumatisme des tissus entourant le disque afin d’accéder à la zone d’intérêt et le temps d’hospitalisation. Les discectomies sont prescrites à certains patients scoliotiques afin de redonner de la flexibilité à la colonne avant l’instrumentation (pose de vis et tige pour corriger la déformation). La résection du disque intervertébral est faite partiellement et la quantité du disque réséqué dépend du degré de flexibilité que le chirurgien désire redonner au patient. En effectuant la discectomie par thoracoscopie, il est impossible pour le chirurgien de visualiser rapidement la quantité de disque restant en plus d’avoir les désavantages de désorientation et de petit champ de vision de la caméra miniature insérée dans le patient. Il est donc pertinent de tenter de réduire les problèmes de visualisation rencontrés lors des thoracoscopies en procurant au chirurgien la possibilité d’examiner en 3D les structures anatomiques du patient pendant la chirurgie sans ajouter de radiations supplémentaires au patient. Ce système d’assistance permettrait également d’accroître la sécurité du patient et la qualité de la chirurgie en donnant aux chirurgiens la possibilité de localiser en 3D la moelle épinière et en leur donnant également la possibilité de visualiser la quantité de disque restant. Ainsi, l’intérêt de fusionner les images vidéo avec un modèle pré-opératoire 3D est alors tout indiqué.---------- ABSTRACT: Visualization tools available while doing thoracoscopic diskectomy do not show depth information and the field of view of the miniaturized camera inserted into the patient is small. Also, simultaneous movement of the camera and surgical tools may result in disorientation. The learning curve for the use of this technology is very steep and numbers of surgeons choose not to use minimally invasive surgery despite important advantages for the patients. Indeed, thoracoscopic diskectomy reduce blood loss, trauma of surrounding soft tissues to access intervertebral disks and hospitalization time. Diskectomy are prescribed to specific scoliotic patients to gain flexibility of the spine before instrumentation surgery (fixation of screws and rod to correct the deformation). The intervertebral disk is partly resected depending on the level of flexibility the patient has to gain according to the surgeon. During thoracoscopic diskectomy, it is impossible for the surgeon to rapidly visualize the remaining disk tissue and this further increase the disadvantages for the surgeons. Hence, it is relevant to try to reduce visualization problems encountered during thoracoscopic diskectomy by providing to the surgeons a 3D view of the whole spine during the surgery, without adding supplementary radiation to the patient. The computer assisted surgery system would also increase the security of the patient by allowing the surgeons to localize rapidly in 3D the spinal canal as well as the remaining disk. The fusion of the video images with 3D spine of the patient is of great interest for the surgeons