Observation-driven adaptive differential evolution and its application to accurate and smooth bronchoscope three-dimensional motion tracking

© 2015 Elsevier B.V. This paper proposes an observation-driven adaptive differential evolution algorithm that fuses bronchoscopic video sequences, electromagnetic sensor measurements, and computed tomography images for accurate and smooth bronchoscope three-dimensional motion tracking. Currently an electromagnetic tracker with a position sensor fixed at the bronchoscope tip is commonly used to estimate bronchoscope movements. The large tracking error from directly using sensor measurements, which may be deteriorated heavily by patient respiratory motion and the magnetic field distortion of the tracker, limits clinical applications. How to effectively use sensor measurements for precise and stable bronchoscope electromagnetic tracking remains challenging. We here exploit an observation-driven adaptive differential evolution framework to address such a challenge and boost the tracking accuracy and smoothness. In our framework, two advantageous points are distinguished from other adaptive differential evolution methods: (1) the current observation including sensor measurements and bronchoscopic video images is used in the mutation equation and the fitness computation, respectively and (2) the mutation factor and the crossover rate are determined adaptively on the basis of the current image observation. The experimental results demonstrate that our framework provides much more accurate and smooth bronchoscope tracking than the state-of-the-art methods. Our approach reduces the tracking error from 3.96 to 2.89. mm, improves the tracking smoothness from 4.08 to 1.62. mm, and increases the visual quality from 0.707 to 0.741

Electromagnetic Tracking for Medical Imaging

This thesis explores the novel use of a wireless electromagnetic: EM) tracking device in a Computed Tomography: CT) environment. The sources of electromagnetic interference inside a Philips Brilliant Big Bore CT scanner are analyzed. A research version of the Calypso wireless tracking system was set up inside the CT suite, and a set of three Beacon transponders was bonded to a plastic fixture. The tracking system was tested under different working parameters including orientation of tracking beacons, the gain level of the frontend amplifier, the distance between the transponders and the sensor array, the rotation speed of the CT gantry, and the presence/absence of the CT X-ray source. The performance of the tracking system reveals two obvious factors which bring in electromagnetic interference: 1) metal like effect brought in by carbon fiber patient couch and 2) electromagnetic disturbance due to spinning metal inside the CT gantry. The accuracy requirements for electromagnetic tracking in the CT environment are a Root Mean Square: RMS) error of \u3c2 mm in stationary position tracking. Within a working volume of 120×120×120 mm3 centered 200 mm below the sensor array, the tracking system achieves the desired clinical goal

Novel design and concepts for biopsy in navigated bronchoscopy

Bronchoscopy is a minimally invasive intervention with a low risk of complications. If CT-images show suspicious lesions, there may be a need to take a sample of tissue for a definitive diagnosis with a biopsy tool, such as biopsy forceps, cytology brush or transbronchial needles. The success rate of biopsy procedures performed with bronchoscopy is low, i.e. they do not provide a decisive diagnosis and there is often a need for repetitive biopsies. Using an ultrathin bronchoscope and navigation systems will increase the diagnostic yield of a biopsy, from approximately 63% to 73-80% for solitary peripheral lesion > 2 cm. This thesis presents a novel design concept for a biopsy tool and a new step for the biopsy procedure in order to solve some of the challenges and limitations of sampling lung lesions, particularly in the peripheral parts of the lung

Open-source virtual bronchoscopy for image guided navigation

This thesis describes the development of an open-source system for virtual bronchoscopy used in combination with electromagnetic instrument tracking. The end application is virtual navigation of the lung for biopsy of early stage cancer nodules. The open-source platform 3D Slicer was used for creating freely available algorithms for virtual bronchscopy. Firstly, the development of an open-source semi-automatic algorithm for prediction of solitary pulmonary nodule malignancy is presented. This approach may help the physician decide whether to proceed with biopsy of the nodule. The user-selected nodule is segmented in order to extract radiological characteristics (i.e., size, location, edge smoothness, calcification presence, cavity wall thickness) which are combined with patient information to calculate likelihood of malignancy. The overall accuracy of the algorithm is shown to be high compared to independent experts' assessment of malignancy. The algorithm is also compared with two different predictors, and our approach is shown to provide the best overall prediction accuracy. The development of an airway segmentation algorithm which extracts the airway tree from surrounding structures on chest Computed Tomography (CT) images is then described. This represents the first fundamental step toward the creation of a virtual bronchoscopy system. Clinical and ex-vivo images are used to evaluate performance of the algorithm. Different CT scan parameters are investigated and parameters for successful airway segmentation are optimized. Slice thickness is the most affecting parameter, while variation of reconstruction kernel and radiation dose is shown to be less critical. Airway segmentation is used to create a 3D rendered model of the airway tree for virtual navigation. Finally, the first open-source virtual bronchoscopy system was combined with electromagnetic tracking of the bronchoscope for the development of a GPS-like system for navigating within the lungs. Tools for pre-procedural planning and for helping with navigation are provided. Registration between the lungs of the patient and the virtually reconstructed airway tree is achieved using a landmark-based approach. In an attempt to reduce difficulties with registration errors, we also implemented a landmark-free registration method based on a balanced airway survey. In-vitro and in-vivo testing showed good accuracy for this registration approach. The centreline of the 3D airway model is extracted and used to compensate for possible registration errors. Tools are provided to select a target for biopsy on the patient CT image, and pathways from the trachea towards the selected targets are automatically created. The pathways guide the physician during navigation, while distance to target information is updated in real-time and presented to the user. During navigation, video from the bronchoscope is streamed and presented to the physician next to the 3D rendered image. The electromagnetic tracking is implemented with 5 DOF sensing that does not provide roll rotation information. An intensity-based image registration approach is implemented to rotate the virtual image according to the bronchoscope's rotations. The virtual bronchoscopy system is shown to be easy to use and accurate in replicating the clinical setting, as demonstrated in the pre-clinical environment of a breathing lung method. Animal studies were performed to evaluate the overall system performance

Tracking and Mapping in Medical Computer Vision: A Review

As computer vision algorithms are becoming more capable, their applications in clinical systems will become more pervasive. These applications include diagnostics such as colonoscopy and bronchoscopy, guiding biopsies and minimally invasive interventions and surgery, automating instrument motion and providing image guidance using pre-operative scans. Many of these applications depend on the specific visual nature of medical scenes and require designing and applying algorithms to perform in this environment. In this review, we provide an update to the field of camera-based tracking and scene mapping in surgery and diagnostics in medical computer vision. We begin with describing our review process, which results in a final list of 515 papers that we cover. We then give a high-level summary of the state of the art and provide relevant background for those who need tracking and mapping for their clinical applications. We then review datasets provided in the field and the clinical needs therein. Then, we delve in depth into the algorithmic side, and summarize recent developments, which should be especially useful for algorithm designers and to those looking to understand the capability of off-the-shelf methods. We focus on algorithms for deformable environments while also reviewing the essential building blocks in rigid tracking and mapping since there is a large amount of crossover in methods. Finally, we discuss the current state of the tracking and mapping methods along with needs for future algorithms, needs for quantification, and the viability of clinical applications in the field. We conclude that new methods need to be designed or combined to support clinical applications in deformable environments, and more focus needs to be put into collecting datasets for training and evaluation.Comment: 31 pages, 17 figure

ELECTROMAGNETIC TRACKER CHARACTERIZATION AND OPTIMAL TOOL DESIGN (WITH APPLICATIONS TO ENT SURGERY)

Electromagnetic tracking systems prove to have great potential for serving as the tracking component of image guided surgery (IGS) systems. However, despite their major advantage over other trackers in that they do not require line-of-sight to the sensors, their use has been limited primarily due to their inherent measurement distortion problem. Presented here are methods of mapping the measurement field distortion and results describing the distortion present in various environments. Further, a framework for calibration and characterization of the tracking system’s systematic error is presented. The error maps are used to generate polynomial models of the distortion that can be used to dynamically compensate for measurement errors. The other core theme of this work is related to optimal design of electromagnetically tracked tools; presented here are mathematical tools for analytically predicting error propagation and optimally configuring sensors on a tool. A software simulator, using a model of the magnetic field distortion, is used to further design and test these tools in a simulation of actual measurement environments before ever even being built. These tools are used to design and test a set of electromagnetically tracked instruments, specifically for ENT surgical applications

InterNAV3D: A Navigation Tool for Robot-Assisted Needle-Based Intervention for the Lung

Lung cancer is one of the leading causes of cancer deaths in North America. There are recent advances in cancer treatment techniques that can treat cancerous tumors, but require a real-time imaging modality to provide intraoperative assistive feedback. Ultrasound (US) imaging is one such modality. However, while its application to the lungs has been limited because of the deterioration of US image quality (due to the presence of air in the lungs); recent work has shown that appropriate lung deflation can help to improve the quality sufficiently to enable intraoperative, US-guided robotics-assisted techniques to be used. The work described in this thesis focuses on this approach. The thesis describes a project undertaken at Canadian Surgical Technologies and Advanced Robotics (CSTAR) that utilizes the image processing techniques to further enhance US images and implements an advanced 3D virtual visualization software approach. The application considered is that for minimally invasive lung cancer treatment using procedures such as brachytherapy and microwave ablation while taking advantage of the accuracy and teleoperation capabilities of surgical robots, to gain higher dexterity and precise control over the therapy tools (needles and probes). A number of modules and widgets are developed and explained which improve the visibility of the physical features of interest in the treatment and help the clinician to have more reliable and accurate control of the treatment. Finally the developed tools are validated with extensive experimental evaluations and future developments are suggested to enhance the scope of the applications