Motion Segmentation by New Three-View Constraint from a Moving Camera

We propose a new method for the motion segmentation using a moving camera. The proposed method classifies each image pixel in the image sequence as the background or the motion regions by applying a novel three-view constraint called the “parallax-based multiplanar constraint.” This new three-view constraint, being the main contribution of this paper, is derived from the relative projective structure of two points in three different views and implemented within the “Plane + Parallax” framework. The parallax-based multiplanar constraint overcomes the problem of the previous geometry constraint and does not require the reference plane to be constant across multiple views. Unlike the epipolar constraint, the parallax-based multiplanar constraint modifies the surface degradation to the line degradation to detect the motion objects followed by a moving camera in the same direction. We evaluate the proposed method with several video sequences to demonstrate the effectiveness and robustness of the parallax-based multiplanar constraint

Interactive Building and Augmentation of Piecewise Planar Environments Using the Intersection Lines

International audienceThis paper describes a method for online interactive building of piecewise planar environments for immediate use in augmented reality. This system combines user interaction from a camera-mouse and automated tracking / reconstruction methods to recover planar structures of the scene that are relevant for the augmentation task. An important contribution of our algorithm is that the process of tracking and reconstructing planar structures is decomposed into three steps - tracking, computation of the intersection lines of the planes, reconstruction - that can each be visually assessed by the user, making the interactive modeling procedure really robust and accurate with intuitive interaction. Videos illustrating our system both on synthetic and long real-size experiments are available at http://www.loria.fr/~gsimon/v

Real time registration of known or recovered multi-planar structures: application to AR

Colloque avec actes et comité de lecture. internationale.International audienceThis paper describes an efficient and reliable method for real time camera tracking. Our algorithm makes use of a multi-planar model of the scene to achieve fast and accurate registration. In this paper, we also propose an automatic method for recovering the multi-planar structure of the scene directly from the homographies induced by the planes in the images. Results are presented, demonstrating tracking and reconstruction for indoor scenes

Intraoperative Endoscopic Augmented Reality in Third Ventriculostomy

In neurosurgery, as a result of the brain-shift, the preoperative patient models used as a intraoperative reference change. A meaningful use of the preoperative virtual models during the operation requires for a model update. The NEAR project, Neuroendoscopy towards Augmented Reality, describes a new camera calibration model for high distorted lenses and introduces the concept of active endoscopes endowed with with navigation, camera calibration, augmented reality and triangulation modules

Longitudinal visualization for exploratory analysis of multiple sclerosis lesions

In multiple sclerosis (MS), the amount of brain damage, anatomical location, shape, and changes are important aspects that help medical researchers and clinicians to understand the temporal patterns of the disease. Interactive visualization for longitudinal MS data can support studies aimed at exploratory analysis of lesion and healthy tissue topology. Existing visualizations in this context comprise bar charts and summary measures, such as absolute numbers and volumes to summarize lesion trajectories over time, as well as summary measures such as volume changes. These techniques can work well for datasets having dual time point comparisons. For frequent follow-up scans, understanding patterns from multimodal data is difficult without suitable visualization approaches. As a solution, we propose a visualization application, wherein we present lesion exploration tools through interactive visualizations that are suitable for large time-series data. In addition to various volumetric and temporal exploration facilities, we include an interactive stacked area graph with other integrated features that enable comparison of lesion features, such as intensity or volume change. We derive the input data for the longitudinal visualizations from automated lesion tracking. For cases with a larger number of follow-ups, our visualization design can provide useful summary information while allowing medical researchers and clinicians to study features at lower granularities. We demonstrate the utility of our visualization on simulated datasets through an evaluation with domain experts.publishedVersio

Synthetic movies derived from multi-dimensional image sensors

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Architecture, 1989.Includes bibliographical references (leaves 139-152).by V. Michael Bove, Jr.Ph.D

A 3D computer assisted Orthopedic Surgery Planning approach based on planar radiography

Dissertação de mestrado integrado em Engenharia Biomédica (área de especialização em Informática Médica)The main goal of this work consisted in develop a system to perform the 3D reconstruction of bone models from radiographic images. This system can be then integrated with a commercial software that performs pre-operative planning of orthopedic surgeries. The benefit of performing this 3D reconstruction from planar radiography is that this modality has some advantages over other modalities that perform this reconstruction directly, like CT and MRI. To develop the system it was used radiographic images of the femur obtained from medical image databases online. It was also used a generic model of the femur available in the online repository BEL. This generic model completes the information missing in the radiographic images. It was developed two methods to perform the 3D reconstruction through the deformation of the generic model, one uses triangulation of extracted edge points and the other don't. The first method was not successful, the final model had very low thickness, possibly because the triangulation process was not performed correctly. With the second method it was obtained a 3D bone model of the femur aligned with the radiographic images of the patient and with the same size as the patient's bone. However, the obtained model still needs some adjustment to coincide fully with reality. To perform this is necessary to enhance the deformation step of the model so that it will have the same shape as the patient's bone. The second method is more advantageous because it doesn't need the parameters of the x-ray imaging system. However, it's necessary to enhance the step deformation of this method so that the final model matches patient's anatomy.O principal objetivo deste trabalho consistiu em desenvolver um sistema capaz de realizar a reconstrução 3D de modelos ósseos a partir de imagens radiográficas. Este sistema pode posteriormente ser integrado num produto comercial que realiza o planeamento pré-operativo de cirurgias ortopédicas. O benefício de realizar esta reconstrução 3D a partir de radiografias está relacionado com o facto desta modalidade ter vantagens em relação às outras modalidades que fazem esta reconstrução diretamente, como as modalidades CT e MRI. Para desenvolver este sistema foram usadas imagens radiográficas do fémur obtidas através de bases de dados online de imagens médicas. Também foi usado um modelo genérico do fémur disponível no repositório online BEL. Este modelo genérico completa a informação que está em falta nas imagens radiográficas. Foram desenvolvidos dois métodos, que realizam a reconstrução 3D através da deformação do modelo genérico sendo que num é feita a triangulação de pontos dos contornos e noutro não. O primeiro método não foi bem sucedido, visto que o modelo final tinha uma espessura muito pequena, possivelmente devido ao facto do processo de triangulação não ter sido executado corretamente. Com o segundo método foi obtido um modelo 3D do fémur alinhado com as imagens radiográficas do paciente e com o mesmo tamanho do osso do paciente. No entanto, o modelo obtido carece ainda de alguma afinação de modo a coincidir na íntegra com a realidade. Para fazer isto é necessário melhorar o passo de deformação do modelo, para que este fique com a mesma forma do osso do paciente. O segundo método é mais vantajoso porque não necessita dos parâmetros dos sistema de raios- X. No entanto, é necessário melhorar o passo de deformação deste método para que o modelo final coincida com a anatomia do paciente

Specular reflection removal and bloodless vessel segmentation for 3-D heart model reconstruction from single view images

Three Dimensional (3D) human heart model is attracting attention for its role in medical images for education and clinical purposes. Analysing 2D images to obtain meaningful information requires a certain level of expertise. Moreover, it is time consuming and requires special devices to obtain aforementioned images. In contrary, a 3D model conveys much more information. 3D human heart model reconstruction from medical imaging devices requires several input images, while reconstruction from a single view image is challenging due to the colour property of the heart image, light reflections, and its featureless surface. Lights and illumination condition of the operating room cause specular reflections on the wet heart surface that result in noises forming of the reconstruction process. Image-based technique is used for the proposed human heart surface reconstruction. It is important the reflection is eliminated to allow for proper 3D reconstruction and avoid imperfect final output. Specular reflections detection and correction process examine the surface properties. This was implemented as a first step to detect reflections using the standard deviation of RGB colour channel and the maximum value of blue channel to establish colour, devoid of specularities. The result shows the accurate and efficient performance of the specularities removing process with 88.7% similarity with the ground truth. Realistic 3D heart model reconstruction was developed based on extraction of pixel information from digital images to allow novice surgeons to reduce the time for cardiac surgery training and enhancing their perception of the Operating Theatre (OT). Cardiac medical imaging devices such as Magnetic Resonance Imaging (MRI), Computed Tomography (CT) images, or Echocardiography provide cardiac information. However,these images from medical modalities are not adequate, to precisely simulate the real environment and to be used in the training simulator for cardiac surgery. The propose method exploits and develops techniques based on analysing real coloured images taken during cardiac surgery in order to obtain meaningful information of the heart anatomical structures. Another issue is the different human heart surface vessels. The most important vessel region is the bloodless, lack of blood, vessels. Surgeon faces some difficulties in locating the bloodless vessel region during surgery. The thesis suggests a technique of identifying the vessels’ Region of Interest (ROI) to avoid surgical injuries by examining an enhanced input image. The proposed method locates vessels’ ROI by using Decorrelation Stretch technique. This Decorrelation Stretch can clearly enhance the heart’s surface image. Through this enhancement, the surgeon become enables effectively identifying the vessels ROI to perform the surgery from textured and coloured surface images. In addition, after enhancement and segmentation of the vessels ROI, a 3D reconstruction of this ROI takes place and then visualize it over the 3D heart model. Experiments for each phase in the research framework were qualitatively and quantitatively evaluated. Two hundred and thirteen real human heart images are the dataset collected during cardiac surgery using a digital camera. The experimental results of the proposed methods were compared with manual hand-labelling ground truth data. The cost reduction of false positive and false negative of specular detection and correction processes of the proposed method was less than 24% compared to other methods. In addition, the efficient results of Root Mean Square Error (RMSE) to measure the correctness of the z-axis values to reconstruction of the 3D model accurately compared to other method. Finally, the 94.42% accuracy rate of the proposed vessels segmentation method using RGB colour space achieved is comparable to other colour spaces. Experimental results show that there is significant efficiency and robustness compared to existing state of the art methods

Intraoperative Navigation Systems for Image-Guided Surgery

Recent technological advancements in medical imaging equipment have resulted in a dramatic improvement of image accuracy, now capable of providing useful information previously not available to clinicians. In the surgical context, intraoperative imaging provides a crucial value for the success of the operation. Many nontrivial scientific and technical problems need to be addressed in order to efficiently exploit the different information sources nowadays available in advanced operating rooms. In particular, it is necessary to provide: (i) accurate tracking of surgical instruments, (ii) real-time matching of images from different modalities, and (iii) reliable guidance toward the surgical target. Satisfying all of these requisites is needed to realize effective intraoperative navigation systems for image-guided surgery. Various solutions have been proposed and successfully tested in the field of image navigation systems in the last ten years; nevertheless several problems still arise in most of the applications regarding precision, usability and capabilities of the existing systems. Identifying and solving these issues represents an urgent scientific challenge. This thesis investigates the current state of the art in the field of intraoperative navigation systems, focusing in particular on the challenges related to efficient and effective usage of ultrasound imaging during surgery. The main contribution of this thesis to the state of the art are related to: Techniques for automatic motion compensation and therapy monitoring applied to a novel ultrasound-guided surgical robotic platform in the context of abdominal tumor thermoablation. Novel image-fusion based navigation systems for ultrasound-guided neurosurgery in the context of brain tumor resection, highlighting their applicability as off-line surgical training instruments. The proposed systems, which were designed and developed in the framework of two international research projects, have been tested in real or simulated surgical scenarios, showing promising results toward their application in clinical practice