Vision Sensors and Edge Detection

Vision Sensors and Edge Detection book reflects a selection of recent developments within the area of vision sensors and edge detection. There are two sections in this book. The first section presents vision sensors with applications to panoramic vision sensors, wireless vision sensors, and automated vision sensor inspection, and the second one shows image processing techniques, such as, image measurements, image transformations, filtering, and parallel computing

Visual Tracking in Robotic Minimally Invasive Surgery

Intra-operative imaging and robotics are some of the technologies driving forward better and more effective minimally invasive surgical procedures. To advance surgical practice and capabilities further, one of the key requirements for computationally enhanced interventions is to know how instruments and tissues move during the operation. While endoscopic video captures motion, the complex appearance dynamic effects of surgical scenes are challenging for computer vision algorithms to handle with robustness. Tackling both tissue and instrument motion estimation, this thesis proposes a combined non-rigid surface deformation estimation method to track tissue surfaces robustly and in conditions with poor illumination. For instrument tracking, a keypoint based 2D tracker that relies on the Generalized Hough Transform is developed to initialize a 3D tracker in order to robustly track surgical instruments through long sequences that contain complex motions. To handle appearance changes and occlusion a patch-based adaptive weighting with segmentation and scale tracking framework is developed. It takes a tracking-by-detection approach and a segmentation model is used to assigns weights to template patches in order to suppress back- ground information. The performance of the method is thoroughly evaluated showing that without any offline-training, the tracker works well even in complex environments. Finally, the thesis proposes a novel 2D articulated instrument pose estimation framework, which includes detection-regression fully convolutional network and a multiple instrument parsing component. The framework achieves compelling performance and illustrates interesting properties includ- ing transfer between different instrument types and between ex vivo and in vivo data. In summary, the thesis advances the state-of-the art in visual tracking for surgical applications for both tissue and instrument motion estimation. It contributes to developing the technological capability of full surgical scene understanding from endoscopic video

Development of an image guidance system for laparoscopic liver surgery and evaluation of optical and computer vision techniques for the assessment of liver tissue

Introduction: Liver resection is increasingly being carried out via the laparoscopic approach (keyhole surgery) because there is mounting evidence that it benefits patients by reducing pain and length of hospitalisation. There are however ongoing concerns about oncological radicality (i.e. ability to completely remove cancer) and an inability to control massive haemorrhage. These issues can partially be attributed to a loss of sensation such as depth perception, tactile feedback and a reduced field of view. Utilisation of optical imaging and computer vision may be able to compensate for some of the lost sensory input because these modalities can facilitate visualisation of liver tissue and structural anatomy. Their use in laparoscopy is attractive because it is easy to adapt or integrate with existing technology. The aim of this thesis is to explore to what extent this technology can aid in the detection of normal and abnormal liver tissue and structures. / Methods: The current state of the art for optical imaging and computer vision in laparoscopic liver surgery is assessed in a systematic review. Evaluation of confocal laser endomicroscopy is carried out on a murine and porcine model of liver disease. Multispectral near infrared imaging is evaluated on ex-vivo liver specimen. Video magnification is assessed on a mechanical flow phantom and a porcine model of liver disease. The latter model was also employed to develop a computer vision based image guidance system for laparoscopic liver surgery. This image guidance system is further evaluated in a clinical feasibility study. Where appropriate, experimental findings are substantiated with statistical analysis. / Results: Use of confocal laser endomicroscopy enabled discrimination between cancer and normal liver tissue with a sub-millimetre precision. This technology also made it possible to verify the adequacy of thermal liver ablation. Multispectral imaging, at specific wavelengths was shown to have the potential to highlight the presence of colorectal and hepatocellular cancer. An image reprocessing algorithm is proposed to simplify visual interpretation of the resulting images. It is shown that video magnification can determine the presence of pulsatile motion but that it cannot reliably determine the extent of motion. Development and performance metrics of an image guidance system for laparoscopic liver surgery are outlined. The system was found to improve intraoperative orientation more development work is however required to enable reliable prediction of oncological margins. / Discussion: The results in this thesis indicate that confocal laser endomicroscopy and image guidance systems have reached a development stage where their intraoperative use may benefit surgeons by visualising features of liver anatomy and tissue characteristics. Video magnification and multispectral imaging require more development and suggestions are made to direct this work. It is also highlighted that it is crucial to standardise assessment methods for these technologies which will allow a more direct comparison between the outcomes of different groups. Limited imaging depth is a major restriction of these technologies but this may be overcome by combining them with preoperatively obtained imaging data. Just like laparoscopy, optical imaging and computer vision use functions of light, a shared characteristic that makes their combined use complementary

Engineering precision surgery: Design and implementation of surgical guidance technologies

In the quest for precision surgery, this thesis introduces several novel detection and navigation modalities for the localization of cancer-related tissues in the operating room. The engineering efforts have focused on image-guided surgery modalities that use the complementary tracer signatures of nuclear and fluorescence radiation. The first part of the thesis covers the use of “GPS-like” navigation concepts to navigate fluorescence cameras during surgery, based on SPECT images of the patient. The second part of the thesis introduces several new imaging modalities such as a hybrid 3D freehand Fluorescence and freehand SPECT imaging and navigation device. Furthermore, to improve the detection of radioactive tracer-emissions during robot-assisted laparoscopic surgery, a tethered DROP-IN gamma probe is introduced. The clinical indications that are used to evaluate the new technologies were all focused on sentinel lymph node procedures in urology (i.e. prostate and penile cancer). Nevertheless, all presented techniques are of such a nature, that they can be applied to different surgical indications, including sentinel lymph node and tumor-receptor-targeted procedures, localization the primary tumor and metastatic spread. This will hopefully contribute towards more precise, less invasive and more effective surgical procedures in the field of oncology. Crystal Photonics GmbH Eurorad S.A. Intuitive Surgical Inc. KARL STORZ Endoscopie Nederland B.V. MILabs B.V. PI Medical Diagnostic Equipment B.V. SurgicEye GmbH Verb Surgical Inc.LUMC / Geneeskund

Bio-Inspired Multi-Spectral and Polarization Imaging Sensors for Image-Guided Surgery

Image-guided surgery (IGS) can enhance cancer treatment by decreasing, and ideally eliminating, positive tumor margins and iatrogenic damage to healthy tissue. Current state-of-the-art near-infrared fluorescence imaging systems are bulky, costly, lack sensitivity under surgical illumination, and lack co-registration accuracy between multimodal images. As a result, an overwhelming majority of physicians still rely on their unaided eyes and palpation as the primary sensing modalities to distinguish cancerous from healthy tissue. In my thesis, I have addressed these challenges in IGC by mimicking the visual systems of several animals to construct low power, compact and highly sensitive multi-spectral and color-polarization sensors. I have realized single-chip multi-spectral imagers with 1000-fold higher sensitivity and 7-fold better spatial co-registration accuracy compared to clinical imaging systems in current use by monolithically integrating spectral tapetal and polarization filters with an array of vertically stacked photodetectors. These imaging sensors yield the unique capabilities of imaging simultaneously color, polarization, and multiple fluorophores for near-infrared fluorescence imaging. Preclinical and clinical data demonstrate seamless integration of this technologies in the surgical work flow while providing surgeons with real-time information on the location of cancerous tissue and sentinel lymph nodes, respectively. Due to its low cost, the bio-inspired sensors will provide resource-limited hospitals with much-needed technology to enable more accurate value-based health care

MEMS Technology for Biomedical Imaging Applications

Biomedical imaging is the key technique and process to create informative images of the human body or other organic structures for clinical purposes or medical science. Micro-electro-mechanical systems (MEMS) technology has demonstrated enormous potential in biomedical imaging applications due to its outstanding advantages of, for instance, miniaturization, high speed, higher resolution, and convenience of batch fabrication. There are many advancements and breakthroughs developing in the academic community, and there are a few challenges raised accordingly upon the designs, structures, fabrication, integration, and applications of MEMS for all kinds of biomedical imaging. This Special Issue aims to collate and showcase research papers, short commutations, perspectives, and insightful review articles from esteemed colleagues that demonstrate: (1) original works on the topic of MEMS components or devices based on various kinds of mechanisms for biomedical imaging; and (2) new developments and potentials of applying MEMS technology of any kind in biomedical imaging. The objective of this special session is to provide insightful information regarding the technological advancements for the researchers in the community