Panorama imaging for image-to-physical registration of narrow drill holes inside spongy bones

Image-to-physical registration based on volumetric data like computed tomography on the one side and intraoperative endoscopic images on the other side is an important method for various surgical applications. In this contribution, we present methods to generate panoramic views from endoscopic recordings for image-to-physical registration of narrow drill holes inside spongy bone. One core application is the registration of drill poses inside the mastoid during minimally invasive cochlear implantations. Besides the development of image processing software for registration, investigations are performed on a miniaturized optical system, achieving 360° radial imaging with one shot by extending a conventional, small, rigid, rod lens endoscope. A reflective cone geometry is used to deflect radially incoming light rays into the endoscope optics. Therefore, a cone mirror is mounted in front of a conventional 0° endoscope. Furthermore, panoramic images of inner drill hole surfaces in artificial bone material are created. Prior to drilling, cone beam computed tomography data is acquired from this artificial bone and simulated endoscopic views are generated from this data. A qualitative and quantitative image comparison of resulting views in terms of image-to-image registration is performed. First results show that downsizing of panoramic optics to a diameter of 3mm is possible. Conventional rigid rod lens endoscopes can be extended to produce suitable panoramic one-shot image data. Using unrolling and stitching methods, images of the inner drill hole surface similar to computed tomography image data of the same surface were created. Registration is performed on ten perturbations of the search space and results in target registration errors of (0:487 ± 0:438)mm at the entry point and (0:957 ± 0:948)mm at the exit as well as an angular error of (1:763 ± 1:536)°. The results show suitability of this image data for image-to-image registration. Analysis of the error components in different directions reveals a strong influence of the pattern structure, meaning higher diversity results into smaller errors. © 2017 SPIE

Virtual viewpoint three-dimensional panorama

Conventional panoramic images are known to provide for an enhanced field of view in which the scene always has a fixed appearance. The idea presented in this paper focuses on the use of the concept of virtual viewpoint creation to generate different panoramic images of the same scene with three-dimensional component. Three-dimensional effect in a resultant panorama is realized by superimposing a stereo-pair of panoramic images

2D Reconstruction of Small Intestine's Interior Wall

Examining and interpreting of a large number of wireless endoscopic images from the gastrointestinal tract is a tiresome task for physicians. A practical solution is to automatically construct a two dimensional representation of the gastrointestinal tract for easy inspection. However, little has been done on wireless endoscopic image stitching, let alone systematic investigation. The proposed new wireless endoscopic image stitching method consists of two main steps to improve the accuracy and efficiency of image registration. First, the keypoints are extracted by Principle Component Analysis and Scale Invariant Feature Transform (PCA-SIFT) algorithm and refined with Maximum Likelihood Estimation SAmple Consensus (MLESAC) outlier removal to find the most reliable keypoints. Second, the optimal transformation parameters obtained from first step are fed to the Normalised Mutual Information (NMI) algorithm as an initial solution. With modified Marquardt-Levenberg search strategy in a multiscale framework, the NMI can find the optimal transformation parameters in the shortest time. The proposed methodology has been tested on two different datasets - one with real wireless endoscopic images and another with images obtained from Micro-Ball (a new wireless cubic endoscopy system with six image sensors). The results have demonstrated the accuracy and robustness of the proposed methodology both visually and quantitatively.Comment: Journal draf

An optimized algorithm of image stitching in the case of a multi-modal probe for monitoring the evolution of scars

International audienceWe propose a new system that makes possible to monitor the evolution of scars after the excision of a tumorous dermatosis. The hardware part of this system is composed of a new optical innovative probe with which two types of images can be acquired simultaneously: an anatomic image acquired under a white light and a functional one based on autofluorescence from the protoporphyrin within the cancer cells. For technical reasons related to the maximum size of the area covered by the probe, acquired images are too small to cover the whole scar. That is why a sequence of overlapping images is taken in order to cover the required area. The main goal of this paper is to describe the creation of two panoramic images (anatomic and functional). Fluorescence images do not have enough salient information for matching the images; stitching algorithms are applied over each couple of successive white light images to produce an anatomic panorama of the entire scar. The same transformations obtained from this step are used to register and stitch the functional images. Several experiments have been implemented using different stitching algorithms (SIFT, ASIFT and SURF), with various transformation parameters (angles of rotation, projection, scaling, etc...) and different types of skin images. We present the results of these experiments that propose the best solution. Thus, clinician has two panoramic images superimposed and usable for diagnostic support. A collaborative layer is added to the system to allow sharing panoramas among several practitioners over different places

High-quality Panorama Stitching based on Asymmetric Bidirectional Optical Flow

In this paper, we propose a panorama stitching algorithm based on asymmetric bidirectional optical flow. This algorithm expects multiple photos captured by fisheye lens cameras as input, and then, through the proposed algorithm, these photos can be merged into a high-quality 360-degree spherical panoramic image. For photos taken from a distant perspective, the parallax among them is relatively small, and the obtained panoramic image can be nearly seamless and undistorted. For photos taken from a close perspective or with a relatively large parallax, a seamless though partially distorted panoramic image can also be obtained. Besides, with the help of Graphics Processing Unit (GPU), this algorithm can complete the whole stitching process at a very fast speed: typically, it only takes less than 30s to obtain a panoramic image of 9000-by-4000 pixels, which means our panorama stitching algorithm is of high value in many real-time applications. Our code is available at https://github.com/MungoMeng/Panorama-OpticalFlow.Comment: Published at the 5th International Conference on Computational Intelligence and Applications (ICCIA 2020

A Stronger Stitching Algorithm for Fisheye Images based on Deblurring and Registration

Fisheye lens, which is suitable for panoramic imaging, has the prominent advantage of a large field of view and low cost. However, the fisheye image has a severe geometric distortion which may interfere with the stage of image registration and stitching. Aiming to resolve this drawback, we devise a stronger stitching algorithm for fisheye images by combining the traditional image processing method with deep learning. In the stage of fisheye image correction, we propose the Attention-based Nonlinear Activation Free Network (ANAFNet) to deblur fisheye images corrected by Zhang calibration method. Specifically, ANAFNet adopts the classical single-stage U-shaped architecture based on convolutional neural networks with soft-attention technique and it can restore a sharp image from a blurred image effectively. In the part of image registration, we propose the ORB-FREAK-GMS (OFG), a comprehensive image matching algorithm, to improve the accuracy of image registration. Experimental results demonstrate that panoramic images of superior quality stitching by fisheye images can be obtained through our method.Comment: 6 pages, 5 figure

Image Stitching Based on Corner Detection

An image stitching is a method of combining multiple images which are overlapping images of the same scene into a larger image. Mostly used methods are Harris corner detection method and SIFTS (Scale Invariant Feature Transform) method. In this paper, a study of Harris corner detection algorithm and SIFT algorithm is done by comparatively in image stitching using similarity matrix matching scheme. Total 30 pairs of different images have been used for their simulation and comparison. The algorithms have been compared with more number of corners detected in images, number of matching pairs and number of matching time. From the results of simulation it has been observed that SIFT corner detection method is most efficient in image stitching

Scars collaborative telediagnosis platform using adaptive image flow

International audienceTelemedicine has been developed to allow practitioners to remotely connect with patients and with other medical staff.We propose a new system (hardware and software), named DICODERM (COllaborative DIagnosis of DERMatosis), which makes it possible to monitor the evolution of scars after the excision of a tumorous dermatosis (like melanoma). The hardware part of this system is composed of a new optical innovative probe with which two types of images can be acquired simultaneously: anatomic with a white light image and functional with a fluorescence image (using autofluorescence from the protoporphyrin within the cancer cell). The software part is composed of two components: the image stitching component, and the collaborative/adaptive layer component. Our system creates a panoramic view of these scars obtained by stitching a sequence of small images. We conducted experiments for different image stitching algorithms to define the best solution. We also deployed a second component: a collaborative system layer which allows to remotely share images of scars and to adapt these images. We also made the system adaptive to communicate across different client platforms. We conducted experiments to compare the exchange of images with or without adaptation: these tests showed the efficiency of our layer