Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

Autonomous Camera Movement for Robotic-Assisted Surgery: A Survey

In the past decade, Robotic-Assisted Surgery (RAS) has become a widely accepted technique as an alternative to traditional open surgery procedures. The best robotic assistant system should combine both human and robot capabilities under the human control. As a matter of fact robot should collaborate with surgeons in a natural and autonomous way, thus requiring less of the surgeons\u27 attention. In this survey, we provide a comprehensive and structured review of the robotic-assisted surgery and autonomous camera movement for RAS operation. We also discuss several topics, including but not limited to task and gesture recognition, that are closely related to robotic-assisted surgery automation and illustrate several successful applications in various real-world application domains. We hope that this paper will provide a more thorough understanding of the recent advances in camera automation in RSA and offer some future research directions

Towards markerless orthopaedic navigation with intuitive Optical See-through Head-mounted displays

The potential of image-guided orthopaedic navigation to improve surgical outcomes has been well-recognised during the last two decades. According to the tracked pose of target bone, the anatomical information and preoperative plans are updated and displayed to surgeons, so that they can follow the guidance to reach the goal with higher accuracy, efficiency and reproducibility. Despite their success, current orthopaedic navigation systems have two main limitations: for target tracking, artificial markers have to be drilled into the bone and calibrated manually to the bone, which introduces the risk of additional harm to patients and increases operating complexity; for guidance visualisation, surgeons have to shift their attention from the patient to an external 2D monitor, which is disruptive and can be mentally stressful. Motivated by these limitations, this thesis explores the development of an intuitive, compact and reliable navigation system for orthopaedic surgery. To this end, conventional marker-based tracking is replaced by a novel markerless tracking algorithm, and the 2D display is replaced by a 3D holographic Optical see-through (OST) Head-mounted display (HMD) precisely calibrated to a user's perspective. Our markerless tracking, facilitated by a commercial RGBD camera, is achieved through deep learning-based bone segmentation followed by real-time pose registration. For robust segmentation, a new network is designed and efficiently augmented by a synthetic dataset. Our segmentation network outperforms the state-of-the-art regarding occlusion-robustness, device-agnostic behaviour, and target generalisability. For reliable pose registration, a novel Bounded Iterative Closest Point (BICP) workflow is proposed. The improved markerless tracking can achieve a clinically acceptable error of 0.95 deg and 2.17 mm according to a phantom test. OST displays allow ubiquitous enrichment of perceived real world with contextually blended virtual aids through semi-transparent glasses. They have been recognised as a suitable visual tool for surgical assistance, since they do not hinder the surgeon's natural eyesight and require no attention shift or perspective conversion. The OST calibration is crucial to ensure locational-coherent surgical guidance. Current calibration methods are either human error-prone or hardly applicable to commercial devices. To this end, we propose an offline camera-based calibration method that is highly accurate yet easy to implement in commercial products, and an online alignment-based refinement that is user-centric and robust against user error. The proposed methods are proven to be superior to other similar State-of- the-art (SOTA)s regarding calibration convenience and display accuracy. Motivated by the ambition to develop the world's first markerless OST navigation system, we integrated the developed markerless tracking and calibration scheme into a complete navigation workflow designed for femur drilling tasks during knee replacement surgery. We verify the usability of our designed OST system with an experienced orthopaedic surgeon by a cadaver study. Our test validates the potential of the proposed markerless navigation system for surgical assistance, although further improvement is required for clinical acceptance.Open Acces

Medical Robotics

The first generation of surgical robots are already being installed in a number of operating rooms around the world. Robotics is being introduced to medicine because it allows for unprecedented control and precision of surgical instruments in minimally invasive procedures. So far, robots have been used to position an endoscope, perform gallbladder surgery and correct gastroesophogeal reflux and heartburn. The ultimate goal of the robotic surgery field is to design a robot that can be used to perform closed-chest, beating-heart surgery. The use of robotics in surgery will expand over the next decades without any doubt. Minimally Invasive Surgery (MIS) is a revolutionary approach in surgery. In MIS, the operation is performed with instruments and viewing equipment inserted into the body through small incisions created by the surgeon, in contrast to open surgery with large incisions. This minimizes surgical trauma and damage to healthy tissue, resulting in shorter patient recovery time. The aim of this book is to provide an overview of the state-of-art, to present new ideas, original results and practical experiences in this expanding area. Nevertheless, many chapters in the book concern advanced research on this growing area. The book provides critical analysis of clinical trials, assessment of the benefits and risks of the application of these technologies. This book is certainly a small sample of the research activity on Medical Robotics going on around the globe as you read it, but it surely covers a good deal of what has been done in the field recently, and as such it works as a valuable source for researchers interested in the involved subjects, whether they are currently “medical roboticists” or not

The beginning of a new era in bone surgery Effectiveness and clinical application of a cold-ablation and robot-guided laser osteotome

Most industrial laser applications utilize computer and robot assistance, for guidance, safety, repeatability, and precision. For industrial applications, the increase in throughput and the processing speed are in the foreground. Nevertheless, these tools cannot just be transferred into clinical and surgical use because the focus in surgical interventions is on the exact implementation of a unique plan. The patient, as an inaccurately defined workpiece, with its individual anatomy and pathology, ultimately needs a single lot planning. Nowadays, medical laser systems are hand driven. The possibility of working precision, as used in industry lasers, is not exhausted. Therefore, medical laser beams have to be coupled to robot guidance. But due to the over-size of commercially available tools, efficient and ergonomic work in an operating room is impossible. Integration of the systems such as the laser source, and the robot arm are needed. Another key issue for the accuracy of the robotic arm is the inclusion of a tracking system. All these issues were encountered developing CARLO®: a Cold-Ablation and Robot-guided Laser Osteotome. This PhD thesis is divided in three parts: - an in-vivo study in sheep, - an in-vitro / wetlab study on human cadavers, and - a theoretical-experimental study to evaluate biomechanical changes in different osteotomy pattern. To test the applicability of the system in an operation theatre similar environment, an in-vivo animal trial was performed. Additionally, we wanted to demonstrate that bone healing after laser osteotomy is not impaired compared to the standard tool the piezo-osteotome. In terms of new mineralized bone formation, histological and micro-CT analysis showed clearly a higher tendency towards the acceleration of the healing process in the laser group. Additionally, no signs of bone necrosis were seen. In addition to the pure functioning of the device, the applicability in the clinic is important for technology to prevail. Therefore, dummy tests for the ergonomics and cadaver tests for the simulation of "real" operations in the cranio-maxillofacial field were performed. Wetlab tests were conducted on human cadavers where different macro-retentive osteotomy patterns were performed. It could be demonstrated that our prototype shows advantages over the current state of the art cutting tools, e.g. reduced bone loss, precise and real-time navigated execution of predefined geometries of freely selected osteotomy patterns. This advantage can be implemented in another indication of our prototype in the cranio-maxillofacial field: in craniosynostosis surgery. We performed a study using finite element analysis to simulate incomplete osteotomies on the inner side of the bone flap to facilitate the re-shaping (skull molding). This biomechanical analysis intended to create basic knowledge in terms of the best stress vs. force relation to obtain the largest projected bone surface. Moreover, a human multicenter study is ready to start for the clinical introduction of the cold-ablation and robot-guided laser osteotome and to gain more experience and information for future work

Image overlay surgery based on augmented reality : a systematic review

Acknowledgements We thank the staff of the Medical Library of the University of Aberdeen for their advice and Prof. Jennifer Cleland and Dr Jenny Gregory for discussion and support. This work was funded by the Roland Sutton Academic Trust (0053/R/17) and an Elphinstone PhD Scholarship from the University of Aberdeen.Postprin