Rendering techniques for multimodal data

Many different direct volume rendering methods have been developed to visualize 3D scalar fields on uniform rectilinear grids. However, little work has been done on rendering simultaneously various properties of the same 3D region measured with different registration devices or at different instants of time. The demand for this type of visualization is rapidly increasing in scientific applications such as medicine in which the visual integration of multiple modalities allows a better comprehension of the anatomy and a perception of its relationships with activity. This paper presents different strategies of Direct Multimodal Volume Rendering (DMVR). It is restricted to voxel models with a known 3D rigid alignment transformation. The paper evaluates at which steps of the render-ing pipeline must the data fusion be realized in order to accomplish the desired visual integration and to provide fast re-renders when some fusion parameters are modified. In addition, it analyzes how existing monomodal visualization al-gorithms can be extended to multiple datasets and it compares their efficiency and their computational cost.Postprint (published version

Design of a multimodal rendering system

This paper addresses the rendering of aligned regular multimodal datasets. It presents a general framework of multimodal data fusion that includes several data merging methods. We also analyze the requirements of a rendering system able to provide these different fusion methods. On the basis of these requirements, we propose a novel design for a multimodal rendering system. The design has been implemented and proved showing to be efficient and flexible.Postprint (published version

Segmentation of pelvic structures from preoperative images for surgical planning and guidance

Prostate cancer is one of the most frequently diagnosed malignancies globally and the second leading cause of cancer-related mortality in males in the developed world. In recent decades, many techniques have been proposed for prostate cancer diagnosis and treatment. With the development of imaging technologies such as CT and MRI, image-guided procedures have become increasingly important as a means to improve clinical outcomes. Analysis of the preoperative images and construction of 3D models prior to treatment would help doctors to better localize and visualize the structures of interest, plan the procedure, diagnose disease and guide the surgery or therapy. This requires efficient and robust medical image analysis and segmentation technologies to be developed. The thesis mainly focuses on the development of segmentation techniques in pelvic MRI for image-guided robotic-assisted laparoscopic radical prostatectomy and external-beam radiation therapy. A fully automated multi-atlas framework is proposed for bony pelvis segmentation in MRI, using the guidance of MRI AE-SDM. With the guidance of the AE-SDM, a multi-atlas segmentation algorithm is used to delineate the bony pelvis in a new \ac{MRI} where there is no CT available. The proposed technique outperforms state-of-the-art algorithms for MRI bony pelvis segmentation. With the SDM of pelvis and its segmented surface, an accurate 3D pelvimetry system is designed and implemented to measure a comprehensive set of pelvic geometric parameters for the examination of the relationship between these parameters and the difficulty of robotic-assisted laparoscopic radical prostatectomy. This system can be used in both manual and automated manner with a user-friendly interface. A fully automated and robust multi-atlas based segmentation has also been developed to delineate the prostate in diagnostic MR scans, which have large variation in both intensity and shape of prostate. Two image analysis techniques are proposed, including patch-based label fusion with local appearance-specific atlases and multi-atlas propagation via a manifold graph on a database of both labeled and unlabeled images when limited labeled atlases are available. The proposed techniques can achieve more robust and accurate segmentation results than other multi-atlas based methods. The seminal vesicles are also an interesting structure for therapy planning, particularly for external-beam radiation therapy. As existing methods fail for the very onerous task of segmenting the seminal vesicles, a multi-atlas learning framework via random decision forests with graph cuts refinement has further been proposed to solve this difficult problem. Motivated by the performance of this technique, I further extend the multi-atlas learning to segment the prostate fully automatically using multispectral (T1 and T2-weighted) MR images via hybrid \ac{RF} classifiers and a multi-image graph cuts technique. The proposed method compares favorably to the previously proposed multi-atlas based prostate segmentation. The work in this thesis covers different techniques for pelvic image segmentation in MRI. These techniques have been continually developed and refined, and their application to different specific problems shows ever more promising results.Open Acces

Accomplishments and challenges in stem cell imaging in vivo.

Stem cell therapies have demonstrated promising preclinical results, but very few applications have reached the clinic owing to safety and efficacy concerns. Translation would benefit greatly if stem cell survival, distribution and function could be assessed in vivo post-transplantation, particularly in patients. Advances in molecular imaging have led to extraordinary progress, with several strategies being deployed to understand the fate of stem cells in vivo using magnetic resonance, scintigraphy, PET, ultrasound and optical imaging. Here, we review the recent advances, challenges and future perspectives and opportunities in stem cell tracking and functional assessment, as well as the advantages and challenges of each imaging approach

A robust coregistration method for in vivo studies using a first generation simultaneous PET/MR scanner

Purpose: Hybrid positron emission tomography (PET)/magnetic resonance (MR) imaging systems have recently been built that allow functional and anatomical information obtained from PET and MR to be acquired simultaneously. The authors have developed a robust coregistration scheme for a first generation small animal PET/MR imaging system and illustrated the potential of this system to study intratumoral heterogeneity in a mouse model. Methods: An alignment strategy to fuse simultaneously acquired PET and MR data, using the MR imaging gradient coordinate system as the reference basis, was developed. The fidelity of the alignment was evaluated over multiple study sessions. In order to explore its robustness in vivo, the alignment strategy was applied to explore the heterogeneity of glucose metabolism in a xenograft tumor model, using ^(18)F-FDG-PET to guide the acquisition of localized ^1H MR spectra within a single imaging session. Results: The alignment method consistently fused the PET/MR data sets with subvoxel accuracy (registration error mean=0.55 voxels, <0.28 mm); this was independent of location within the field of view. When the system was used to study intratumoral heterogeneity within xenograft tumors, a correlation of high ^(18)F-FDG-PET signal with high choline/creatine ratio was observed. Conclusions: The authors present an implementation of an efficient and robust coregistration scheme for multimodal noninvasive imaging using PET and MR. This setup allows time-sensitive, multimodal studies of physiology to be conducted in an efficient manner