Panorama imaging for image-to-physical registration of narrow drill holes inside spongy bones

Image-to-physical registration based on volumetric data like computed tomography on the one side and intraoperative endoscopic images on the other side is an important method for various surgical applications. In this contribution, we present methods to generate panoramic views from endoscopic recordings for image-to-physical registration of narrow drill holes inside spongy bone. One core application is the registration of drill poses inside the mastoid during minimally invasive cochlear implantations. Besides the development of image processing software for registration, investigations are performed on a miniaturized optical system, achieving 360° radial imaging with one shot by extending a conventional, small, rigid, rod lens endoscope. A reflective cone geometry is used to deflect radially incoming light rays into the endoscope optics. Therefore, a cone mirror is mounted in front of a conventional 0° endoscope. Furthermore, panoramic images of inner drill hole surfaces in artificial bone material are created. Prior to drilling, cone beam computed tomography data is acquired from this artificial bone and simulated endoscopic views are generated from this data. A qualitative and quantitative image comparison of resulting views in terms of image-to-image registration is performed. First results show that downsizing of panoramic optics to a diameter of 3mm is possible. Conventional rigid rod lens endoscopes can be extended to produce suitable panoramic one-shot image data. Using unrolling and stitching methods, images of the inner drill hole surface similar to computed tomography image data of the same surface were created. Registration is performed on ten perturbations of the search space and results in target registration errors of (0:487 ± 0:438)mm at the entry point and (0:957 ± 0:948)mm at the exit as well as an angular error of (1:763 ± 1:536)°. The results show suitability of this image data for image-to-image registration. Analysis of the error components in different directions reveals a strong influence of the pattern structure, meaning higher diversity results into smaller errors. © 2017 SPIE

Endoscopic versus Microscopic Transsphenoidal Hypophysectomy: Comparison of the Endocrine Outcome – An Institutional Experience

BACKGROUND: The transnasal transsphenoidal endoscopic approach to the sella turcica is an overwhelming alternative to the microscopic approach for the past few decades assuming into prominence as a new technique, reaching nearly gold standard for this pathology. The endoscopic approach to the pituitary has redefined accurate visualization of the sella. The panoramic view afforded by the endoscope is unparalleled as compared with the traditional conical view of the microscope. AIMS: This study aims to compare both endoscopic and microscopic technologies, including advantages and disadvantages through the results of endocrine outcome. SETTINGS AND DESIGN: Our retrospective/prospective study included 46 microscopically and 39 endoscopically treated patients during the period of 2010–2018. Tumors were classified according to the diameter and clinical outcomes were evaluated. RESULTS: Our retrospective/prospective study included 46 microscopically and 39 endoscopically treated patients during the period of 2010–2018. Tumors were classified according to the diameter, hormone activity and clinical outcomes were evaluated. Comparison results revealed more efficacious and effective endocrine control and reestablishing the endocrine homeostasis utilizing the endoscopic technique, especially in secretory active macroadenomas. Further, the extension of the resection, which was better in endoscopic approach undouptedly contributed to better endocrine control of the disease. Complication rate, including endocrine, was lower following endoscopy compared with microsurgery. CONCLUSION: This technique evidenced to have a statistically significant reduction in operative time and length of hospital stay, as well as more radical safe resection and complication control. There is also a trend toward improved endocrine outcomes and rate of return of visual defects. These two approaches are still comparable with eloquent advantages and disadvantages, formulated as balanced dialectics. In addition, the use of endoscopes, including multilocular polifilament 3D endoscope, facilitates extended approaches, reaching a delicate skull base lesions that are suprasellar, retrosellar, and parasellar, which permits visualization beyond the abilities of the microscope

Frontiers of robotic endoscopic capsules: a review

Digestive diseases are a major burden for society and healthcare systems, and with an aging population, the importance of their effective management will become critical. Healthcare systems worldwide already struggle to insure quality and affordability of healthcare delivery and this will be a significant challenge in the midterm future. Wireless capsule endoscopy (WCE), introduced in 2000 by Given Imaging Ltd., is an example of disruptive technology and represents an attractive alternative to traditional diagnostic techniques. WCE overcomes conventional endoscopy enabling inspection of the digestive system without discomfort or the need for sedation. Thus, it has the advantage of encouraging patients to undergo gastrointestinal (GI) tract examinations and of facilitating mass screening programmes. With the integration of further capabilities based on microrobotics, e.g. active locomotion and embedded therapeutic modules, WCE could become the key-technology for GI diagnosis and treatment. This review presents a research update on WCE and describes the state-of-the-art of current endoscopic devices with a focus on research-oriented robotic capsule endoscopes enabled by microsystem technologies. The article also presents a visionary perspective on WCE potential for screening, diagnostic and therapeutic endoscopic procedures

Accuracy and feasibility of a dedicated image guidance solution for endoscopic lateral skull base surgery.

OBJECTIVE We aimed to design, build and validate a surgical navigation system which fulfills the accuracy requirements for surgical procedures on the ear and the lateral skull base, and which integrates with the endoscopic workflow and operating room setup. MATERIALS AND METHODS The navigation system consists of portable tablet computer (iPad Pro, Apple Computer, USA) and an optical tracking system (Cambar B1, Axios3D, Germany), both connected via a wireless Bluetooth link and attached directly to the OR table. Active optical tracking references are rigidly fixed to both the patient and surgical tools. Software to support image import, registration and 2D/3D visualization has been developed. Two models were used for targeting accuracy assessment: a technical phantom model and an ex vivo temporal bone model. Additionally, workflow integration and usability of the navigation system during endoscopic lateral skull base procedures was investigated in ex vivo experiments on 12 sides of cadaver head specimens. RESULTS The accuracy experiments revealed a target registration error in the technical phantom model of 0.20 ± 0.10 mm (n = 36) and during the ex vivo assessment of 0.28 ± 0.10 mm (n = 21). Navigation was successfully carried out in n = 36 procedures (infracochlear, suprageniculate and transpromontorial approach), with navigated instruments usable without interference with the endoscope. The system aided in the successful and accurate identification of vital anatomical structures. CONCLUSIONS Useful surgical navigation is, to a large extent, a result of sufficiently accurate tracking technology. We have demonstrated sufficient accuracy and a potentially suitable integration for surgical application within endoscopic lateral skull base procedures

Endoscope-assisted resection of brainstem cavernous malformations

Targeted surgical precision and minimally invasive techniques are of utmost importance for resectioning cavernous malformations involving the brainstem region. Minimisation of the surgical corridor is desirable but should not compromise the extent of resection. This study provides detailed information on the role of endoscopy in this challenging surgical task. A retrospective analysis of medical documentation, radiologic studies and detailed intraoperative video documentation was performed for all consecutive patients who underwent surgical resection of brainstem cavernous malformations between 2010 and 2020 at the authors’ institution. A case-based volumetry of the corticotomy was performed and compared to cavernoma dimensions. A total of 20 procedures have been performed in 19 patients. Neuroendoscopy was implemented in all cases. The mean size of the lesion was 5.4 (± 5) mm3. The average size of the brainstem corticotomy was 4.5 × 3.7 (± 1.0 × 1.1) mm, with a median relation to the cavernoma’s dimension of 9.99% (1.2–31.39%). Endoscopic 360° inspection of the resection cavity was feasible in all cases. There were no endoscopy-related complications. Mean follow-up was 27.8 (12–89) months. Gross-total resection was achieved in all but one case (95%). Sixteen procedures (80%) resulted in an improved or stable medical condition. Eleven patients (61.1%) showed further improvement 12 months after the initial surgery. With the experience provided, endoscopic techniques can be safely implemented in surgery for BSCM. A combination of neuroendoscopic visualisation and neuronavigation might enable a targeted size of brainstem corticotomy. Endoscopy can currently be considered a valuable additive tool to facilitate the preparation and resection of BSCM

Appearance Modelling and Reconstruction for Navigation in Minimally Invasive Surgery

Minimally invasive surgery is playing an increasingly important role for patient care. Whilst its direct patient benefit in terms of reduced trauma, improved recovery and shortened hospitalisation has been well established, there is a sustained need for improved training of the existing procedures and the development of new smart instruments to tackle the issue of visualisation, ergonomic control, haptic and tactile feedback. For endoscopic intervention, the small field of view in the presence of a complex anatomy can easily introduce disorientation to the operator as the tortuous access pathway is not always easy to predict and control with standard endoscopes. Effective training through simulation devices, based on either virtual reality or mixed-reality simulators, can help to improve the spatial awareness, consistency and safety of these procedures. This thesis examines the use of endoscopic videos for both simulation and navigation purposes. More specifically, it addresses the challenging problem of how to build high-fidelity subject-specific simulation environments for improved training and skills assessment. Issues related to mesh parameterisation and texture blending are investigated. With the maturity of computer vision in terms of both 3D shape reconstruction and localisation and mapping, vision-based techniques have enjoyed significant interest in recent years for surgical navigation. The thesis also tackles the problem of how to use vision-based techniques for providing a detailed 3D map and dynamically expanded field of view to improve spatial awareness and avoid operator disorientation. The key advantage of this approach is that it does not require additional hardware, and thus introduces minimal interference to the existing surgical workflow. The derived 3D map can be effectively integrated with pre-operative data, allowing both global and local 3D navigation by taking into account tissue structural and appearance changes. Both simulation and laboratory-based experiments are conducted throughout this research to assess the practical value of the method proposed

Towards tactile sensing active capsule endoscopy

Examination of the gastrointestinal(GI) tract has traditionally been performed using tethered endoscopy tools with limited reach and more recently with passive untethered capsule endoscopy with limited capability. Inspection of small intestines is only possible using the latter capsule endoscopy with on board camera system. Limited to visual means it cannot detect features beneath the lumen wall if they have not affected the lumen structure or colour. This work presents an improved capsule endoscopy system with locomotion for active exploration of the small intestines and tactile sensing to detect deformation of the capsule outer surface when it follows the intestinal wall. In laboratory conditions this system is capable of identifying sub-lumen features such as submucosal tumours.Through an extensive literary review the current state of GI tract inspection in particular using remote operated miniature robotics, was investigated, concluding no solution currently exists that utilises tactile sensing with a capsule endoscopy. In order to achieve such a platform, further investigation was made in to tactile sensing technologies, methods of locomotion through the gut, and methods to support an increased power requirement for additional electronics and actuation. A set of detailed criteria were compiled for a soft formed sensor and flexible bodied locomotion system. The sensing system is built on the biomimetic tactile sensing device, Tactip, \cite{Chorley2008, Chorley2010, Winstone2012, Winstone2013} which has been redesigned to fit the form of a capsule endoscopy. These modifications have required a $360^{o}$ cylindrical sensing surface with $360^{o}$ panoramic optical system. Multi-material 3D printing has been used to build an almost complete sensor assembly with a combination of hard and soft materials, presenting a soft compliant tactile sensing system that mimics the tactile sensing methods of the human finger. The cylindrical Tactip has been validated using artificial submucosal tumours in laboratory conditions. The first experiment has explored the new form factor and measured the device's ability to detect surface deformation when travelling through a pipe like structure with varying lump obstructions. Sensor data was analysed and used to reconstruct the test environment as a 3D rendered structure. A second tactile sensing experiment has explored the use of classifier algorithms to successfully discriminate between three tumour characteristics; shape, size and material hardness. Locomotion of the capsule endoscopy has explored further bio-inspiration from earthworm's peristaltic locomotion, which share operating environment similarities. A soft bodied peristaltic worm robot has been developed that uses a tuned planetary gearbox mechanism to displace tendons that contract each worm segment. Methods have been identified to optimise the gearbox parameter to a pipe like structure of a given diameter. The locomotion system has been tested within a laboratory constructed pipe environment, showing that using only one actuator, three independent worm segments can be controlled. This configuration achieves comparable locomotion capabilities to that of an identical robot with an actuator dedicated to each individual worm segment. This system can be miniaturised more easily due to reduced parts and number of actuators, and so is more suitable for capsule endoscopy. Finally, these two developments have been integrated to demonstrate successful simultaneous locomotion and sensing to detect an artificial submucosal tumour embedded within the test environment. The addition of both tactile sensing and locomotion have created a need for additional power beyond what is available from current battery technology. Early stage work has reviewed wireless power transfer (WPT) as a potential solution to this problem. Methods for optimisation and miniaturisation to implement WPT on a capsule endoscopy have been identified with a laboratory built system that validates the methods found. Future work would see this combined with a miniaturised development of the robot presented. This thesis has developed a novel method for sub-lumen examination. With further efforts to miniaturise the robot it could provide a comfortable and non-invasive procedure to GI tract inspection reducing the need for surgical procedures and accessibility for earlier stage of examination. Furthermore, these developments have applicability in other domains such as veterinary medicine, industrial pipe inspection and exploration of hazardous environments

Medical needs related to the endoscopic technology and colonoscopy for colorectal cancer diagnosis

Background. The high incidence and mortality rate of colorectal cancer require new technologies to improve its early diagnosis. This study aims at extracting the medical needs related to the endoscopic technology and the colonoscopy procedure currently used for colorectal cancer diagnosis, essential for designing these demanded technologies. Methods. Semi-structured interviews and an online survey were used. Results. Six endoscopists were interviewed and 103 were surveyed, obtaining the demanded needs that can be divided into: a) clinical needs, for better polyp detection and classification (especially flat polyps), location, size, margins and penetration depth; b) computer-aided diagnosis (CAD) system needs, for additional visual information supporting polyp characterization and diagnosis; and c) operational/physical needs, related to limitations of image quality, colon lighting, flexibility of the endoscope tip, and even poor bowel preparation.This work is part of the PICCOLO project, which has received funding from the European Union’s Horizon 2020 research and innovation Programme under grant agreement No. 732111. GR18199, funded by “Consejería de Economía, Ciencia y Agenda Digital, Junta de Extremadura” and co-funded by European Union (ERDF “A way to make Europe”). The funding bodies did not play any roles in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript

Patient Specific Systems for Computer Assisted Robotic Surgery Simulation, Planning, and Navigation

The evolving scenario of surgery: starting from modern surgery, to the birth of medical imaging and the introduction of minimally invasive techniques, has seen in these last years the advent of surgical robotics. These systems, making possible to get through the difficulties of endoscopic surgery, allow an improved surgical performance and a better quality of the intervention. Information technology contributed to this evolution since the beginning of the digital revolution: providing innovative medical imaging devices and computer assisted surgical systems. Afterwards, the progresses in computer graphics brought innovative visualization modalities for medical datasets, and later the birth virtual reality has paved the way for virtual surgery. Although many surgical simulators already exist, there are no patient specific solutions. This thesis presents the development of patient specific software systems for preoperative planning, simulation and intraoperative assistance, designed for robotic surgery: in particular for bimanual robots that are becoming the future of single port interventions. The first software application is a virtual reality simulator for this kind of surgical robots. The system has been designed to validate the initial port placement and the operative workspace for the potential application of this surgical device. Given a bimanual robot with its own geometry and kinematics, and a patient specific 3D virtual anatomy, the surgical simulator allows the surgeon to choose the optimal positioning of the robot and the access port in the abdominal wall. Additionally, it makes possible to evaluate in a virtual environment if a dexterous movability of the robot is achievable, avoiding unwanted collisions with the surrounding anatomy to prevent potential damages in the real surgical procedure. Even if the software has been designed for a specific bimanual surgical robot, it supports any open kinematic chain structure: as far as it can be described in our custom format. The robot capabilities to accomplish specific tasks can be virtually tested using the deformable models: interacting directly with the target virtual organs, trying to avoid unwanted collisions with the surrounding anatomy not involved in the intervention. Moreover, the surgical simulator has been enhanced with algorithms and data structures to integrate biomechanical parameters into virtual deformable models (based on mass-spring-damper network) of target solid organs, in order to properly reproduce the physical behaviour of the patient anatomy during the interactions. The main biomechanical parameters (Young's modulus and density) have been integrated, allowing the automatic tuning of some model network elements, such as: the node mass and the spring stiffness. The spring damping coefficient has been modeled using the Rayleigh approach. Furthermore, the developed method automatically detect the external layer, allowing the usage of both the surface and internal Young's moduli, in order to model the main parts of dense organs: the stroma and the parenchyma. Finally the model can be manually tuned to represent lesion with specific biomechanical properties. Additionally, some software modules of the simulator have been properly extended to be integrated in a patient specific computer guidance system for intraoperative navigation and assistance in robotic single port interventions. This application provides guidance functionalities working in three different modalities: passive as a surgical navigator, assistive as a guide for the single port placement and active as a tutor preventing unwanted collision during the intervention. The simulation system has beed tested by five surgeons: simulating the robot access port placemen, and evaluating the robot movability and workspace inside the patient abdomen. The tested functionalities, rated by expert surgeons, have shown good quality and performance of the simulation. Moreover, the integration of biomechanical parameters into deformable models has beed tested with various material samples. The results have shown a good visual realism ensuring the performance required by an interactive simulation. Finally, the intraoperative navigator has been tested performing a cholecystectomy on a synthetic patient mannequin, in order to evaluate: the intraoperative navigation accuracy, the network communications latency and the overall usability of the system. The tests performed demonstrated the effectiveness and the usability of the software systems developed: encouraging the introduction of the proposed solution in the clinical practice, and the implementation of further improvements. Surgical robotics will be enhanced by an advanced integration of medical images into software systems: allowing the detailed planning of surgical interventions by means of virtual surgery simulation based on patient specific biomechanical parameters. Furthermore, the advanced functionalities offered by these systems, enable surgical robots to improve the intraoperative surgical assistance: benefitting of the knowledge of the virtual patient anatomy