Image overlay surgery based on augmented reality : a systematic review

Acknowledgements We thank the staff of the Medical Library of the University of Aberdeen for their advice and Prof. Jennifer Cleland and Dr Jenny Gregory for discussion and support. This work was funded by the Roland Sutton Academic Trust (0053/R/17) and an Elphinstone PhD Scholarship from the University of Aberdeen.Postprin

Design and clinical evaluation of an image-guided surgical microscope with an integrated tracking system

A new image-guided microscope system using augmented reality image overlays has been developed. With this system, CT cut-views and segmented objects such as tumors that have been previously extracted from preoperative tomographic images can be directly displayed as augmented reality overlays on the microscope image. The novelty of this design stems from the inclusion of a precise mini-tracker directly on the microscope. This device, which is rigidly mounted to the microscope, is used to track the movements of surgical tools and the patient. In addition to an accuracy gain, this setup offers improved ergonomics since it is much easier for the surgeon to keep an unobstructed line of sight to tracked objects. We describe the components of the system: microscope calibration, image registration, tracker assembly and registration, tool tracking, and augmented reality display. The accuracy of the system has been measured by validation on plastic skulls and cadaver heads, obtaining an overlay error of 0.7mm. In addition, a numerical simulation of the system has been done in order to complement the accuracy study, showing that the integration of the tracker onto the microscope could lead to an improvement of the accuracy to the order of 0.5mm. Finally, we describe our clinical experience using the system in the operation room, where three operations have been performed to dat

Review on Augmented Reality in Oral and Cranio-Maxillofacial Surgery: Toward 'Surgery-Specific' Head-Up Displays

In recent years, there has been an increasing interest towards the augmented reality as applied to the surgical field. We conducted a systematic review of literature classifying the augmented reality applications in oral and cranio-maxillofacial surgery (OCMS) in order to pave the way to future solutions that may ease the adoption of AR guidance in surgical practice. Publications containing the terms 'augmented reality' AND 'maxillofacial surgery', and the terms 'augmented reality' AND 'oral surgery' were searched in the PubMed database. Through the selected studies, we performed a preliminary breakdown according to general aspects, such as surgical subspecialty, year of publication and country of research; then, a more specific breakdown was provided according to technical features of AR-based devices, such as virtual data source, visualization processing mode, tracking mode, registration technique and AR display type. The systematic search identified 30 eligible publications. Most studies (14) were in orthognatic surgery, the minority (2) concerned traumatology, while 6 studies were in oncology and 8 in general OCMS. In 8 of 30 studies the AR systems were based on a head-mounted approach using smart glasses or headsets. In most of these cases (7), a video-see-through mode was implemented, while only 1 study described an optical-see-through mode. In the remaining 22 studies, the AR content was displayed on 2D displays (10), full-parallax 3D displays (6) and projectors (5). In 1 case the AR display type is not specified. AR applications are of increasing interest and adoption in oral and cranio-maxillofacial surgery, however, the quality of the AR experience represents the key requisite for a successful result. Widespread use of AR systems in the operating room may be encouraged by the availability of 'surgery-specific' head-mounted devices that should guarantee the accuracy required for surgical tasks and the optimal ergonomics

An Augmented Reality Platform for Preoperative Surgical Planning

Researching in new technologies for diagnosis, planning and medical treatment have allowed the development of computer tools that provide new ways of representing data obtained from patient's medical images such as computed tomography (CT) and magnetic resonance imaging (MRI). In this sense, augmented reality (AR) technologies provide a new form of data representation by combining the common analysis using images and the ability to superimpose virtual 3D representations of the organs of the human body in the real environment. In this paper the development of a generic computer platform based on augmented reality technology for surgical preoperative planning is presented. In particular, the surgeon can navigate in the 3D models of the patient's organs in order to have the possibility to perfectly understand the anatomy and plan in the best way the surgical procedure. In addition, a touchless interaction with the virtual organs is available thanks to the use of an armband provided of electromiographic muscle sensors. To validate the system, we focused in a navigation through aorta artery for mitral valve repair surgery

Current and future use of virtual and augmented reality in minimal invasive oral surgery

Not availabl

Augmented reality for dental implantology: a pilot clinical report of two cases

Background: Despite the limited number of articles dedicated to its use, augmented reality (AR) is an emerging technology that has shown to have increasing applications in multiple different medical sectors. These include, but are not limited to, the Maxillo-facial and Dentistry disciplines of medicine. In these medical specialties, the focus of AR technology is to achieve a more visible surgical field during an operation. Currently, this goal is brought about by an accurate display of either static or dynamic diagnostic images via the use of a visor or specific glasses. The objective of this study is to evaluate the feasibility of using a virtual display for dynamic navigation via AR. The secondary outcome is to evaluate if the use of this technology could affect the accuracy of dynamic navigation. Case presentation: Two patients, both needing implant rehabilitation in the upper premolar area, were treated with flapless surgery. Prior to the procedure itself, the position of the implant was virtually planned and placed for each of the patients using their previous scans. This placement preparation contributed to a dynamic navigation system that was displayed on AR glasses. This, in turn, allowed for the use of a computer-aided/image-guided procedure to occur. Dedicated software for surface superimposition was then used to match the planned position of the implant and the real one obtained from the postoperative scan. Accuracies, using this procedure were evaluated by way of measuring the deviation between real and planned positions of the implants. For both surgeries it was possible to proceed using the AR technology as planned. The deviations for the first implant were 0.53\u2009mm at the entry point and 0.50\u2009mm at the apical point and for the second implant were 0.46\u2009mm at the entry point and 0.48\u2009mm at the apical point. The angular deviations were respectively 3.05\ub0 and 2.19\ub0. Conclusions: From the results of this pilot study, it seems that AR can be useful in dental implantology for displaying dynamic navigation systems. While this technology did not seem to noticeably affect the accuracy of the procedure, specific software applications should further optimize the results

Computer-assisted dental implant placement following free flap reconstruction: virtual planning, CAD/CAM templates, dynamic navigation and augmented reality

Image-guided surgery, prosthetic-based virtual planning, 3D printing, and CAD/CAM technology are changing head and neck ablative and reconstructive surgical oncology. Due to quality-of-life improvement, dental implant rehabilitation could be considered in every patient treated with curative intent. Accurate implant placement is mandatory for prosthesis long-term stability and success in oncologic patients. We present a prospective study, with a novel workflow, comprising 11 patients reconstructed with free flaps and 56 osseointegrated implants placed in bone flaps or remnant jaws (iliac crest, fibula, radial forearm, anterolateral thigh). Starting from CT data and jaw plaster model scanning, virtual dental prosthesis was designed. Then prosthetically driven dental implacement was also virtually planned and transferred to the patient by means of intraoperative infrared optical navigation (first four patients), and a combination of conventional static teeth supported 3D-printed acrylic guide stent, intraoperative dynamic navigation, and augmented reality for final intraoperative verification (last 7 patients). Coronal, apical, and angular deviation between virtual surgical planning and final guided intraoperative position was measured on each implant. There is a clear learning curve for surgeons when applying guided methods. Initial only-navigated cases achieved low accuracy but were comparable to non-guided freehand positioning due to jig registration instability. Subsequent dynamic navigation cases combining highly stable acrylic static guides as reference and registration markers result in the highest accuracy with a 1-1.5-mm deviation at the insertion point. Smartphone-based augmented reality visualization is a valuable tool for intraoperative visualization and final verification, although it is still a difficult technique for guiding surgery. A fixed screw-retained ideal dental prosthesis was achieved in every case as virtually planned. Implant placement, the final step in free flap oncological reconstruction, could be accurately planned and placed with image-guided surgery, 3D printing, and CAD/CAM technology. The learning curve could be overcome with preclinical laboratory training, but virtually designed and 3D-printed tracer registration stability is crucial for accurate and predictable results. Applying these concepts to our difficult oncologic patient subgroup with deep anatomic alterations ended in comparable results as those reported in non-oncologic patients.This work was supported by grant PI18/01625 (Ministerio de Ciencia e Innovación-Instituto de Salud Carlos III and European Regional Development Fund "Una manera de hacer Europa"). This study was also supported by Ticare® implants (Mozo-Grau, Valladolid, Spain). The funder was not involved in the study design, collection, analysis, interpretation of data, the writing of this article or the decision to submit it for publication

Recent Developments and Future Challenges in Medical Mixed Reality

As AR technology matures, we have seen many applicationsemerge in entertainment, education and training. However, the useof AR is not yet common in medical practice, despite the great po-tential of this technology to help not only learning and training inmedicine, but also in assisting diagnosis and surgical guidance. Inthis paper, we present recent trends in the use of AR across all med-ical specialties and identify challenges that must be overcome tonarrow the gap between academic research and practical use of ARin medicine. A database of 1403 relevant research papers publishedover the last two decades has been reviewed by using a novel re-search trend analysis method based on text mining algorithm. Wesemantically identified 10 topics including varies of technologiesand applications based on the non-biased and in-personal cluster-ing results from the Latent Dirichlet Allocatio (LDA) model andanalysed the trend of each topic from 1995 to 2015. The statisticresults reveal a taxonomy that can best describes the developmentof the medical AR research during the two decades. And the trendanalysis provide a higher level of view of how the taxonomy haschanged and where the focus will goes. Finally, based on the valu-able results, we provide a insightful discussion to the current limi-tations, challenges and future directions in the field. Our objectiveis to aid researchers to focus on the application areas in medicalAR that are most needed, as well as providing medical practitioners with latest technology advancements