Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

Computer- and robot-assisted Medical Intervention

Medical robotics includes assistive devices used by the physician in order to make his/her diagnostic or therapeutic practices easier and more efficient. This chapter focuses on such systems. It introduces the general field of Computer-Assisted Medical Interventions, its aims, its different components and describes the place of robots in that context. The evolutions in terms of general design and control paradigms in the development of medical robots are presented and issues specific to that application domain are discussed. A view of existing systems, on-going developments and future trends is given. A case-study is detailed. Other types of robotic help in the medical environment (such as for assisting a handicapped person, for rehabilitation of a patient or for replacement of some damaged/suppressed limbs or organs) are out of the scope of this chapter.Comment: Handbook of Automation, Shimon Nof (Ed.) (2009) 000-00

A Fetal Brain magnetic resonance Acquisition Numerical phantom (FaBiAN)

Accurate characterization of in utero human brain maturation is critical as it involves complex and interconnected structural and functional processes that may influence health later in life. Magnetic resonance imaging is a powerful tool to investigate equivocal neurological patterns during fetal development. However, the number of acquisitions of satisfactory quality available in this cohort of sensitive subjects remains scarce, thus hindering the validation of advanced image processing techniques. Numerical phantoms can mitigate these limitations by providing a controlled environment with a known ground truth. In this work, we present FaBiAN, an open-source Fetal Brain magnetic resonance Acquisition Numerical phantom that simulates clinical T2-weighted fast spin echo sequences of the fetal brain. This unique tool is based on a general, flexible and realistic setup that includes stochastic fetal movements, thus providing images of the fetal brain throughout maturation comparable to clinical acquisitions. We demonstrate its value to evaluate the robustness and optimize the accuracy of an algorithm for super-resolution fetal brain magnetic resonance imaging from simulated motion-corrupted 2D low-resolution series compared to a synthetic high-resolution reference volume. We also show that the images generated can complement clinical datasets to support data-intensive deep learning methods for fetal brain tissue segmentation

Advanced Algorithms for 3D Medical Image Data Fusion in Specific Medical Problems

Fúze obrazu je dnes jednou z nejběžnějších avšak stále velmi diskutovanou oblastí v lékařském zobrazování a hraje důležitou roli ve všech oblastech lékařské péče jako je diagnóza, léčba a chirurgie. V této dizertační práci jsou představeny tři projekty, které jsou velmi úzce spojeny s oblastí fúze medicínských dat. První projekt pojednává o 3D CT subtrakční angiografii dolních končetin. V práci je využito kombinace kontrastních a nekontrastních dat pro získání kompletního cévního stromu. Druhý projekt se zabývá fúzí DTI a T1 váhovaných MRI dat mozku. Cílem tohoto projektu je zkombinovat stukturální a funkční informace, které umožňují zlepšit znalosti konektivity v mozkové tkáni. Třetí projekt se zabývá metastázemi v CT časových datech páteře. Tento projekt je zaměřen na studium vývoje metastáz uvnitř obratlů ve fúzované časové řadě snímků. Tato dizertační práce představuje novou metodologii pro klasifikaci těchto metastáz. Všechny projekty zmíněné v této dizertační práci byly řešeny v rámci pracovní skupiny zabývající se analýzou lékařských dat, kterou vedl pan Prof. Jiří Jan. Tato dizertační práce obsahuje registrační část prvního a klasifikační část třetího projektu. Druhý projekt je představen kompletně. Další část prvního a třetího projektu, obsahující specifické předzpracování dat, jsou obsaženy v disertační práci mého kolegy Ing. Romana Petera.Image fusion is one of today´s most common and still challenging tasks in medical imaging and it plays crucial role in all areas of medical care such as diagnosis, treatment and surgery. Three projects crucially dependent on image fusion are introduced in this thesis. The first project deals with the 3D CT subtraction angiography of lower limbs. It combines pre-contrast and contrast enhanced data to extract the blood vessel tree. The second project fuses the DTI and T1-weighted MRI brain data. The aim of this project is to combine the brain structural and functional information that purvey improved knowledge about intrinsic brain connectivity. The third project deals with the time series of CT spine data where the metastases occur. In this project the progression of metastases within the vertebrae is studied based on fusion of the successive elements of the image series. This thesis introduces new methodology of classifying metastatic tissue. All the projects mentioned in this thesis have been solved by the medical image analysis group led by Prof. Jiří Jan. This dissertation concerns primarily the registration part of the first project and the classification part of the third project. The second project is described completely. The other parts of the first and third project, including the specific preprocessing of the data, are introduced in detail in the dissertation thesis of my colleague Roman Peter, M.Sc.

On the Real-Time Performance, Robustness and Accuracy of Medical Image Non-Rigid Registration

Three critical issues about medical image non-rigid registration are performance, robustness and accuracy. A registration method, which is capable of responding timely with an accurate alignment, robust against the variation of the image intensity and the missing data, is desirable for its clinical use. This work addresses all three of these issues. Unacceptable execution time of Non-rigid registration (NRR) often presents a major obstacle to its routine clinical use. We present a hybrid data partitioning method to parallelize a NRR method on a cooperative architecture, which enables us to get closer to the goal: accelerating using architecture rather than designing a parallel algorithm from scratch. to further accelerate the performance for the GPU part, a GPU optimization tool is provided to automatically optimize GPU execution configuration.;Missing data and variation of the intensity are two severe challenges for the robustness of the registration method. A novel point-based NRR method is presented to resolve mapping function (deformation field) with the point correspondence missing. The novelty of this method lies in incorporating a finite element biomechanical model into an Expectation and Maximization (EM) framework to resolve the correspondence and mapping function simultaneously. This method is extended to deal with the deformation induced by tumor resection, which imposes another challenge, i.e. incomplete intra-operative MRI. The registration is formulated as a three variable (Correspondence, Deformation Field, and Resection Region) functional minimization problem and resolved by a Nested Expectation and Maximization framework. The experimental results show the effectiveness of this method in correcting the deformation in the vicinity of the tumor. to deal with the variation of the intensity, two different methods are developed depending on the specific application. For the mono-modality registration on delayed enhanced cardiac MRI and cine MRI, a hybrid registration method is designed by unifying both intensity- and feature point-based metrics into one cost function. The experiment on the moving propagation of suspicious myocardial infarction shows effectiveness of this hybrid method. For the multi-modality registration on MRI and CT, a Mutual Information (MI)-based NRR is developed by modeling the underlying deformation as a Free-Form Deformation (FFD). MI is sensitive to the variation of the intensity due to equidistant bins. We overcome this disadvantage by designing a Top-to-Down K-means clustering method to naturally group similar intensities into one bin. The experiment shows this method can increase the accuracy of the MI-based registration.;In image registration, a finite element biomechanical model is usually employed to simulate the underlying movement of the soft tissue. We develop a multi-tissue mesh generation method to build a heterogeneous biomechanical model to realistically simulate the underlying movement of the brain. We focus on the following four critical mesh properties: tissue-dependent resolution, fidelity to tissue boundaries, smoothness of mesh surfaces, and element quality. Each mesh property can be controlled on a tissue level. The experiments on comparing the homogeneous model with the heterogeneous model demonstrate the effectiveness of the heterogeneous model in improving the registration accuracy

Automated template-based brain localization and extraction for fetal brain MRI reconstruction.

Most fetal brain MRI reconstruction algorithms rely only on brain tissue-relevant voxels of low-resolution (LR) images to enhance the quality of inter-slice motion correction and image reconstruction. Consequently the fetal brain needs to be localized and extracted as a first step, which is usually a laborious and time consuming manual or semi-automatic task. We have proposed in this work to use age-matched template images as prior knowledge to automatize brain localization and extraction. This has been achieved through a novel automatic brain localization and extraction method based on robust template-to-slice block matching and deformable slice-to-template registration. Our template-based approach has also enabled the reconstruction of fetal brain images in standard radiological anatomical planes in a common coordinate space. We have integrated this approach into our new reconstruction pipeline that involves intensity normalization, inter-slice motion correction, and super-resolution (SR) reconstruction. To this end we have adopted a novel approach based on projection of every slice of the LR brain masks into the template space using a fusion strategy. This has enabled the refinement of brain masks in the LR images at each motion correction iteration. The overall brain localization and extraction algorithm has shown to produce brain masks that are very close to manually drawn brain masks, showing an average Dice overlap measure of 94.5%. We have also demonstrated that adopting a slice-to-template registration and propagation of the brain mask slice-by-slice leads to a significant improvement in brain extraction performance compared to global rigid brain extraction and consequently in the quality of the final reconstructed images. Ratings performed by two expert observers show that the proposed pipeline can achieve similar reconstruction quality to reference reconstruction based on manual slice-by-slice brain extraction. The proposed brain mask refinement and reconstruction method has shown to provide promising results in automatic fetal brain MRI segmentation and volumetry in 26 fetuses with gestational age range of 23 to 38 weeks

Innovative techniques to devise 3D-printed anatomical brain phantoms for morpho-functional medical imaging

Introduction. The Ph.D. thesis addresses the development of innovative techniques to create 3D-printed anatomical brain phantoms, which can be used for quantitative technical assessments on morpho-functional imaging devices, providing simulation accuracy not obtainable with currently available phantoms. 3D printing (3DP) technology is paving the way for advanced anatomical modelling in biomedical applications. Despite the potential already expressed by 3DP in this field, it is still little used for the realization of anthropomorphic phantoms of human organs with complex internal structures. Making an anthropomorphic phantom is very different from making a simple anatomical model and 3DP is still far from being plug-and-print. Hence, the need to develop ad-hoc techniques providing innovative solutions for the realization of anatomical phantoms with unique characteristics, and greater ease-of-use. Aim. The thesis explores the entire workflow (brain MRI images segmentation, 3D modelling and materialization) developed to prototype a new complex anthropomorphic brain phantom, which can simulate three brain compartments simultaneously: grey matter (GM), white matter (WM) and striatum (caudate nucleus and putamen, known to show a high uptake in nuclear medicine studies). The three separate chambers of the phantom will be filled with tissue-appropriate solutions characterized by different concentrations of radioisotope for PET/SPECT, para-/ferro-magnetic metals for MRI, and iodine for CT imaging. Methods. First, to design a 3D model of the brain phantom, it is necessary to segment MRI images and to extract an error-less STL (Standard Tessellation Language) description. Then, it is possible to materialize the prototype and test its functionality. - Image segmentation. Segmentation is one of the most critical steps in modelling. To this end, after demonstrating the proof-of-concept, a multi-parametric segmentation approach based on brain relaxometry was proposed. It includes a pre-processing step to estimate relaxation parameter maps (R1 = longitudinal relaxation rate, R2 = transverse relaxation rate, PD = proton density) from the signal intensities provided by MRI sequences of routine clinical protocols (3D-GrE T1-weighted, FLAIR and fast-T2-weighted sequences with ≤ 3 mm slice thickness). In the past, maps of R1, R2, and PD were obtained from Conventional Spin Echo (CSE) sequences, which are no longer suitable for clinical practice due to long acquisition times. Rehabilitating the multi-parametric segmentation based on relaxometry, the estimation of pseudo-relaxation maps allowed developing an innovative method for the simultaneous automatic segmentation of most of the brain structures (GM, WM, cerebrospinal fluid, thalamus, caudate nucleus, putamen, pallidus, nigra, red nucleus and dentate). This method allows the segmentation of higher resolution brain images for future brain phantom enhancements. - STL extraction. After segmentation, the 3D model of phantom is described in STL format, which represents the shapes through the approximation in manifold mesh (i.e., collection of triangles, which is continuous, without holes and with a positive – not zero – volume). For this purpose, we developed an automatic procedure to extract a single voxelized surface, tracing the anatomical interface between the phantom's compartments directly on the segmented images. Two tubes were designed for each compartment (one for filling and the other to facilitate the escape of air). The procedure automatically checks the continuity of the surface, ensuring that the 3D model could be exported in STL format, without errors, using a common image-to-STL conversion software. Threaded junctions were added to the phantom (for the hermetic closure) using a mesh processing software. The phantom's 3D model resulted correct and ready for 3DP. Prototyping. Finally, the most suitable 3DP technology is identified for the materialization. We investigated the material extrusion technology, named Fused Deposition Modeling (FDM), and the material jetting technology, named PolyJet. FDM resulted the best candidate for our purposes. It allowed materializing the phantom's hollow compartments in a single print, without having to print them in several parts to be reassembled later. FDM soluble internal support structures were completely removable after the materialization, unlike PolyJet supports. A critical aspect, which required a considerable effort to optimize the printing parameters, was the submillimetre thickness of the phantom walls, necessary to avoid distorting the imaging simulation. However, 3D printer manufacturers recommend maintaining a uniform wall thickness of at least 1 mm. The optimization of printing path made it possible to obtain strong, but not completely waterproof walls, approximately 0.5 mm thick. A sophisticated technique, based on the use of a polyvinyl-acetate solution, was developed to waterproof the internal and external phantom walls (necessary requirement for filling). A filling system was also designed to minimize the residual air bubbles, which could result in unwanted hypo-intensity (dark) areas in phantom-based imaging simulation. Discussions and conclusions. The phantom prototype was scanned trough CT and PET/CT to evaluate the realism of the brain simulation. None of the state-of-the-art brain phantoms allow such anatomical rendering of three brain compartments. Some represent only GM and WM, others only the striatum. Moreover, they typically have a poor anatomical yield, showing a reduced depth of the sulci and a not very faithful reproduction of the cerebral convolutions. The ability to simulate the three brain compartments simultaneously with greater accuracy, as well as the possibility of carrying out multimodality studies (PET/CT, PET/MRI), which represent the frontier of diagnostic imaging, give this device cutting-edge prospective characteristics. The effort to further customize 3DP technology for these applications is expected to increase significantly in the coming years