Observation-driven adaptive differential evolution and its application to accurate and smooth bronchoscope three-dimensional motion tracking

© 2015 Elsevier B.V. This paper proposes an observation-driven adaptive differential evolution algorithm that fuses bronchoscopic video sequences, electromagnetic sensor measurements, and computed tomography images for accurate and smooth bronchoscope three-dimensional motion tracking. Currently an electromagnetic tracker with a position sensor fixed at the bronchoscope tip is commonly used to estimate bronchoscope movements. The large tracking error from directly using sensor measurements, which may be deteriorated heavily by patient respiratory motion and the magnetic field distortion of the tracker, limits clinical applications. How to effectively use sensor measurements for precise and stable bronchoscope electromagnetic tracking remains challenging. We here exploit an observation-driven adaptive differential evolution framework to address such a challenge and boost the tracking accuracy and smoothness. In our framework, two advantageous points are distinguished from other adaptive differential evolution methods: (1) the current observation including sensor measurements and bronchoscopic video images is used in the mutation equation and the fitness computation, respectively and (2) the mutation factor and the crossover rate are determined adaptively on the basis of the current image observation. The experimental results demonstrate that our framework provides much more accurate and smooth bronchoscope tracking than the state-of-the-art methods. Our approach reduces the tracking error from 3.96 to 2.89. mm, improves the tracking smoothness from 4.08 to 1.62. mm, and increases the visual quality from 0.707 to 0.741

Robust electromagnetically guided endoscopic procedure using enhanced particle swarm optimization for multimodal information fusion

© 2015 American Association of Physicists in Medicine. Purpose: Electromagnetically guided endoscopic procedure, which aims at accurately and robustly localizing the endoscope, involves multimodal sensory information during interventions. However, it still remains challenging in how to integrate these information for precise and stable endoscopic guidance. To tackle such a challenge, this paper proposes a new framework on the basis of an enhanced particle swarm optimization method to effectively fuse these information for accurate and continuous endoscope localization. Methods: The authors use the particle swarm optimization method, which is one of stochastic evolutionary computation algorithms, to effectively fuse the multimodal information including preoperative information (i.e., computed tomography images) as a frame of reference, endoscopic camera videos, and positional sensor measurements (i.e., electromagnetic sensor outputs). Since the evolutionary computation method usually limits its possible premature convergence and evolutionary factors, the authors introduce the current (endoscopic camera and electromagnetic sensors) observation to boost the particle swarm optimization and also adaptively update evolutionary parameters in accordance with spatial constraints and the current observation, resulting in advantageous performance in the enhanced algorithm. Results: The experimental results demonstrate that the authors proposed method provides a more accurate and robust endoscopic guidance framework than state-of-the-art methods. The average guidance accuracy of the authors framework was about 3.0 mm and 5.6° while the previous methods show at least 3.9 mm and 7.0°. The average position and orientation smoothness of their method was 1.0 mm and 1.6°, which is significantly better than the other methods at least with (2.0 mm and 2.6°). Additionally, the average visual quality of the endoscopic guidance was improved to 0.29. Conclusions: A robust electromagnetically guided endoscopy framework was proposed on the basis of an enhanced particle swarm optimization method with using the current observation information and adaptive evolutionary factors. The authors proposed framework greatly reduced the guidance errors from (4.3, 7.8) to (3.0 mm, 5.6°), compared to state-of-the-art methods

Real-Time Quantitative Bronchoscopy

The determination of motion within a sequence of images remains one of the fundamental problems in computer vision after more than 30 years of research. Despite this work, there have been relatively few applications of these techniques to practical problems outside the fields of robotics and video encoding. In this paper, we present the continuing work to apply optical flow and egomotion recovery to the problem of measuring and navigating through the airway using a bronchoscope during a standard procedure, without the need for any additional data, localization systems or other external components. The current implementation uses a number of techniques to provide a range of numerical measurements and estimations to physicians in real time, using standard computer hardware

A New Fiducial Marker for Gated Radiotherapy in the Lung – A Feasibility Study of Bronchoscopy Based Insertion and Removal in Göttingen Mini-Pig

To develop a new prototype fiducial marker (LS-1) that may be used for gated radiotherapy in the lung.  One LS-1 marker was inserted in the lung of each animal under sedation. Animals were kept under observation  for four weeks after insertion. After the observation period the marker was removed. Animals were  CT scanned after insertion and before removal of the LS-1 marker. After the last CT scan animals were  euthanized and lungs excised for pathology. The LS-1 marker was successfully inserted in all fourteen animals. Thirteen of fourteen LS-1 marker’s  were in situ after four weeks. Two cases of pneumothorax were seen in connection with insertion. The LS-1  marker could only be successfully removed from eleven of thirteen animals. Damage to the lung was mainly  local close to the LS-1 marker insertion site. The LS-1 marker has the potential to be a fiducial marker suitable for gated external beam radiotherapy in  the lung. The method still needs some refinement prior to application in humans.

Committee II: Guidelines for cytologic sampling techniques of lung and mediastinal lymph nodes

Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/146505/1/dc23975.pdfhttps://deepblue.lib.umich.edu/bitstream/2027.42/146505/2/dc23975_am.pd

Patient-specific bronchoscope simulation with pq-space-based 2D/3D registration

Objective: The use of patient-specific models for surgical simulation requires photorealistic rendering of 3D structure and surface properties. For bronchoscope simulation, this requires augmenting virtual bronchoscope views generated from 3D tomographic data with patient-specific bronchoscope videos. To facilitate matching of video images to the geometry extracted from 3D tomographic data, this paper presents a new pq-space-based 2D/3D registration method for camera pose estimation in bronchoscope tracking. Methods: The proposed technique involves the extraction of surface normals for each pixel of the video images by using a linear local shape-from-shading algorithm derived from the unique camera/lighting constraints of the endoscopes. The resultant pq-vectors are then matched to those of the 3D model by differentiation of the z-buffer. A similarity measure based on angular deviations of the pq-vectors is used to provide a robust 2D/3D registration framework. Localization of tissue deformation is considered by assessing the temporal variation of the pq-vectors between subsequent frames. Results: The accuracy of the proposed method was assessed by using an electromagnetic tracker and a specially constructed airway phantom. Preliminary in vivo validation of the proposed method was performed on a matched patient bronchoscope video sequence and 3D CT data. Comparison to existing intensity-based techniques was also made. Conclusion: The proposed method does not involve explicit feature extraction and is relatively immune to illumination changes. The temporal variation of the pq distribution also permits the identification of localized deformation, which offers an effective way of excluding such areas from the registration process

Advanced Endoscopic Navigation:Surgical Big Data,Methodology,and Applications

随着科学技术的飞速发展，健康与环境问题日益成为人类面临的最重大问题之一。信息科学、计算机技术、电子工程与生物医学工程等学科的综合应用交叉前沿课题，研究现代工程技术方法，探索肿瘤癌症等疾病早期诊断、治疗和康复手段。本论文综述了计算机辅助微创外科手术导航、多模态医疗大数据、方法论及其临床应用：从引入微创外科手术导航概念出发，介绍了医疗大数据的术前与术中多模态医学成像方法、阐述了先进微创外科手术导航的核心流程包括计算解剖模型、术中实时导航方案、三维可视化方法及交互式软件技术，归纳了各类微创外科手术方法的临床应用。同时，重点讨论了全球各种手术导航技术在临床应用中的优缺点，分析了目前手术导航领域内的最新技术方法。在此基础上，提出了微创外科手术方法正向数字化、个性化、精准化、诊疗一体化、机器人化以及高度智能化的发展趋势。【Abstract】Interventional endoscopy (e.g., bronchoscopy, colonoscopy, laparoscopy, cystoscopy) is a widely performed procedure that involves either diagnosis of suspicious lesions or guidance for minimally invasive surgery in a variety of organs within the body cavity. Endoscopy may also be used to guide the introduction of certain items (e.g., stents) into the body. Endoscopic navigation systems seek to integrate big data with multimodal information (e.g., computed tomography, magnetic resonance images, endoscopic video sequences, ultrasound images, external trackers) relative to the patient's anatomy, control the movement of medical endoscopes and surgical tools, and guide the surgeon's actions during endoscopic interventions. Nevertheless, it remains challenging to realize the next generation of context-aware navigated endoscopy. This review presents a broad survey of various aspects of endoscopic navigation, particularly with respect to the development of endoscopic navigation techniques. First, we investigate big data with multimodal information involved in endoscopic navigation. Next, we focus on numerous methodologies used for endoscopic navigation. We then review different endoscopic procedures in clinical applications. Finally, we discuss novel techniques and promising directions for the development of endoscopic navigation.X.L. acknowledges funding from the Fundamental Research Funds for the Central Universities. T.M.P. acknowledges funding from the Canadian Foundation for Innovation, the Canadian Institutes for Health Research, the National Sciences and Engineering Research Council of Canada, and a grant from Intuitive Surgical Inc

Automated catheter navigation with electromagnetic image guidance

This paper describes a novel method of controlling an endoscopic catheter by using an automated catheter tensioning system with the objective of providing clinicians with improved manipulation capabilities within the patient. Catheters are used in many clinical procedures to provide access to the cardiopulmonary system. Control of such catheters is performed manually by the clinicians using a handle, typically actuating a single or opposing set of pull wires. Such catheters are generally actuated in a single plane, requiring the clinician to rotate the catheter handle to navigate the system. The automation system described here allows closed-loop control of a custom bronchial catheter in tandem with an electromagnetic tracking of the catheter tip and image guidance by using a 3D Slicer. An electromechanical drive train applies tension to four pull wires to steer the catheter tip, with the applied force constantly monitored through force sensing load cells. The applied tension is controlled through a PC connected joystick. An electromagnetic sensor embedded in the catheter tip enables constant real-time position tracking, whereas a working channel provides a route for endoscopic instruments. The system is demonstrated and tested in both a breathing lung model and a preclinical animal study. Navigation to predefined targets in the subject's airways by using the joystick while using virtual image guidance and electromagnetic tracking was demonstrated. Average targeting times were 29 and 10 s, respectively, for the breathing lung and live animal studies. This paper presents the first reported remote controlled bronchial working channel catheter utilizing electromagnetic tracking and has many implications for future development in endoscopic and catheter-based procedures