Facilitating the design of multidimensional and local transfer functions for volume visualization

The importance of volume visualization is increasing since the sizes of the datasets that need to be inspected grow with every new version of medical scanners (e.g., CT and MR). Direct volume rendering is a 3D visualization technique that has, in many cases, clear benefits over 2D views. It is able to show 3D information, facilitating mental reconstruction of the 3D shape of objects and their spatial relation. The complexity of the settings required in order to generate a 3D rendering is, however, one of the main reasons for this technique not being used more widely in practice. Transfer functions play an important role in the appearance of volume rendered images by determining the optical properties of each piece of the data. The transfer function determines what will be seen and how. The goal of the project on which this PhD thesis reports was to develop and investigate new approaches that would facilitate the setting of transfer functions. As shown in the state of the art overview in Chapter 2, there are two main aspects that influence the effectiveness of a TF: the choice of the TF domain and the process of defining the shape of the TF. The choice of a TF domain, i.e., the choice of the data properties used, directly determines which aspects of the volume data can be visualized. In many approaches, special attention is given to TF domains that would enable an easier selection and visualization of boundaries between materials. The boundaries are an important aspect of the volume data since they reveal the shapes and sizes of objects. Our research in improving the TF definition focused on introducing new user interaction methods and automation techniques that shield the user from the complex process of manually defining the shape and color properties of TFs. Our research dealt with both the TF domain and the TF definition since they are closely related. A suitable TF domain cannot only greatly improve the manual definition, but also, more importantly, increases the possibilities of using automated techniques. Chapter 3 presents a new TF domain. We have used the LH space and the associated LH histogram for TFs based on material boundaries. We showed that the LH space reduces the ambiguity when selecting boundaries compared to the commonly used space of the data value and gradient magnitude. Fur- thermore, boundaries appear as blobs in the LH histogram that make them easier to select. Its compactness and easier selectivity of the boundaries makes the LH histogram suitable for the introduction of clustering-based automation. The mirrored extension of the LH space differentiates between both sides of the boundary. The mirrored LH histogram shows interesting properties of this space, allowing the selection of all boundaries belonging to one material in an easy way. We have also shown that segmentation techniques, such as region growing methods, can benefit from the properties of LH space. Standard cost functions based on the data value and/or the gradient magnitude may experience problems at the boundaries due to the partial volume effect. However, our cost function that is based on the LH space is, however, capable of handling the region growing of boundaries better. Chapter 4 presents an interaction framework for the TF definition based on hierarchical clustering of material boundaries. Our framework aims at an easy combination of various similarity measures that reflect requirements of the user. One of the main benefits of the framework is the absence of similarity-weighting coefficients that are usually hard to define. Further, the framework enables the user to visualize objects that may exist at different levels of the hierarchy. We also introduced two similarity measures that illustrate the functionality of the framework. The main contribution is the first similarity measure that takes advantage of properties of the LH histogram from Chapter 3. We assumed that the shapes of the peaks in the LH histogram can guide the grouping of clusters. The second similarity measure is based on the spatial relationships of clusters. In Chapter 5, we presented part of our research that focused on one of the main issues encountered in the TFs in general. Standard TFs, as they are applied everywhere in the volume in the same way, become difficult to use when the data properties (measurements) of the same material vary over the volume, for example, due to the acquisition inaccuracies. We address this problem by introducing the concept and framework of local transfer functions (LTFs). Local transfer functions are based on using locally applicable TFs in cases where it might be difficult or impossible to define a globally applicable TF. We discussed a number of reasons that hamper the global TF and illustrated how the LTFs may help to alleviate these problems. We have also discussed how multiple TFs can be combined and automatically adapted. One of our contributions is the use of the similarity of local histograms and their correlation for the combination and adaptation of LTFs

Time-varying volume visualization

Volume rendering is a very active research field in Computer Graphics because of its wide range of applications in various sciences, from medicine to flow mechanics. In this report, we survey a state-of-the-art on time-varying volume rendering. We state several basic concepts and then we establish several criteria to classify the studied works: IVR versus DVR, 4D versus 3D+time, compression techniques, involved architectures, use of parallelism and image-space versus object-space coherence. We also address other related problems as transfer functions and 2D cross-sections computation of time-varying volume data. All the papers reviewed are classified into several tables based on the mentioned classification and, finally, several conclusions are presented.Preprin

Fast widefield techniques for fluorescence and phase endomicroscopy

Thesis (Ph.D.)--Boston UniversityEndomicroscopy is a recent development in biomedical optics which gives researchers and physicians microscope-resolution views of intact tissue to complement macroscopic visualization during endoscopy screening. This thesis presents HiLo endomicroscopy and oblique back-illumination endomicroscopy, fast widefield imaging techniques with fluorescence and phase contrast, respectively. Fluorescence imaging in thick tissue is often hampered by strong out-of-focus background signal. Laser scanning confocal endomicroscopy has been developed for optically-sectioned imaging free from background, but reliance on mechanical scanning fundamentally limits the frame rate and represents significant complexity and expense. HiLo is a fast, simple, widefield fluorescence imaging technique which rejects out-of-focus background signal without the need for scanning. It works by acquiring two images of the sample under uniform and structured illumination and synthesizing an optically sectioned result with real-time image processing. Oblique back-illumination microscopy (OBM) is a label-free technique which allows, for the first time, phase gradient imaging of sub-surface morphology in thick scattering tissue with a reflection geometry. OBM works by back-illuminating the sample with the oblique diffuse reflectance from light delivered via off-axis optical fibers. The use of two diametrically opposed illumination fibers allows simultaneous and independent measurement of phase gradients and absorption contrast. Video-rate single-exposure operation using wavelength multiplexing is demonstrated

A multi-technique hierarchical X-ray phase-based approach for the characterization and quantification of the effects of novel radiotherapies

Cancer is the first or second leading cause of premature deaths worldwide with an overall rapidly growing burden. Standard cancer therapies include surgery, chemotherapy and radiotherapy (RT) and often a combination of the three is applied to improve the probability of tumour control. Standard therapy protocols have been established for many types of cancers and new approaches are under study especially for treating radio-resistant tumours associated to an overall poor prognosis, as for brain and lung cancers. Follow up techniques able to monitor and investigate the effects of therapies are important for surveying the efficacy of conventionally applied treatments and are key for accessing the curing capabilities and the onset of acute and late adverse effects of new therapies. In this framework, this doctoral Thesis proposes the X-ray Phase Contrast Im-aging - Computed Tomography (XPCI-CT) technique as an imaging-based tool to study and quantify the effects of novel RTs, namely Microbeam and Minibeam Radiation therapy (MRT and MB), and to compare them to the standard Broad Beam (BB) induced effects on brain and lungs. MRT and MB are novel radiotherapies that deliver an array of spatially fractionated X-ray beamlets issued from a synchrotron radiation source, with widths of tens or hundreds of micrometres, respectively. MRT and MB exploit the so-called dose-volume effect: hundreds of Grays are well tolerated by healthy tissues and show a preferential effect on tumour cells and vasculature when delivered in a micrometric sized micro-plane, while induce lethal effects if applied over larger uniform irradiation fields. Such highly collimated X-ray beams need a high-resolution and a full-organ approach that can visualize, with high sensitivity, the effects of the treatment along and outside the beamlets path. XPCI-CT is here suggested and proven as a powerful imaging technique able to determine and quantify the effects of the radiation on normal and tumour-bearing tissues. Moreover, it is shown as an effective technique to complement, with 3D information, the histology findings in the follow-up of the RT treatments. Using a multi-scale and multi-technique X-ray-based approach, I have visualized and analysed the effects of RT delivery on healthy and glioblastoma multiforme (GBM)-bearing rat brains as well as on healthy rat lungs. Ex-vivo XPCI-CT datasets acquired with isotropic voxel sizes in the range 3.253 – 0.653 μm3 could distinguish, with high sensitivity, the idiopathic effects of MRT, MB and BB therapies. Histology, immunohistochemistry, Small- and Wide-Angle X-ray Scattering and X-ray Fluorescence experiments were also carried out to accurately interpret and complement the XPCI-CT findings as well as to obtain a detailed structural and chemical characterization of the detected pathological features. Overall, this multi-technique approach could detect: i) a different radio-sensitivity for the MRT-treated brain areas; ii) Ca and Fe deposits, hydroxyapatite crystals formation; iii) extended and isolated fibrotic contents. Full-organ XPCI-CT datasets allowed for the quantification of tumour and mi-crocalcifications’ volumes in treated brains and the amount of scarring tissue in irradiated lungs. Herein, the role of XPCI-CT as a 3D virtual histology technique for the follow-up of ex-vivo RT effects has been assessed as a complementary method for an accurate volumetric investigation of normal and pathological states in brains and lungs, in a small animal model. Moreover, the technique is proposed as a guidance and auxiliary tool for conventional histology, which is the gold standard for pathological evaluations, owing to its 3D capabilities and the possibility of virtually navigating within samples. This puts a landmark for XPCI-CT inclusion in the pre-clinical studies pipeline and for advancing towards in-vivo XPCI-CT imaging of treated organs.Weltweit gilt Krebs als häufigste bzw. zweithäufigste Ursache eines zu früh erfolgenden Todes, wobei die Zahlen rasch ansteigen. Standardmäßige Krebstherapien umfassen chirurgische Eingriffe, Chemotherapie und Strahlentherapie (radiotherapy, RT); oft kommt eine Kombination daraus zur Anwendung, um die Wahrscheinlichkeit der Tumorkontrolle zu erhöhen. Es wurden Standardtherapieprotokolle für zahlreiche Krebsarten eingerichtet und es wird vor allem in der Behandlung von strahlenresistenten Tumoren mit allgemein schlechter Prognose wie bei Hirn- und Lungentumoren an neuen Ansätzen geforscht. Nachverfolgungstechniken, welche die Auswirkungen von Therapien überwachen und ermitteln, sind zur Überwachung der Wirksamkeit herkömmlich angewandter Behandlungen wichtig und auch maßgeblich am Zugang zu den Fähigkeiten zur Heilung sowie zum Auftreten akuter und verzögerter Nebenwirkungen neuer Therapien beteiligt. In diesem Rahmenwerk unterbreitet diese Doktorarbeit die Technik der Röntgen-Phasenkontrast-Bildgebung über Computertomographie (X-ray Phase Contrast Imaging - Computed Tomography, XPCI‑CT) als bildverarbeitungs-basiertes Tool zur Untersuchung und Quantifizierung der Auswirkungen neuartiger Strahlentherapien, nämlich der Mikrobeam- und Minibeam-Strahlentherapie (MRT und MB), sowie zum Vergleich derselben mit den herkömmlichen durch Breitstrahlen (Broad Beam, BB) erzielten Auswirkungen auf Gehirn und Lunge. MRT und MB sind neuartige Strahlentherapien, die ein Array räumlich aufgeteilter Röntgenstrahlenbeamlets aus einer synchrotronen Strahlenquelle mit einer Breite von Zehnteln bzw. Hundersteln Mikrometern abgeben. MRT und MB nutzen den sogenannten Dosis-Volumen-Effekt: Hunderte Gray werden von gesundem Gewebe gut vertragen und wirken bei der Abgabe in einer Mikroebene im Mikrometerbereich vorrangig auf Tumorzellen und Blutgefäße, während sie bei einer Anwendung über größere gleichförmige Strahlungsfelder letale Auswirkungen aufweisen. Solche hoch kollimierten Röntgenstrahlen erfordern eine hohe Auflösung und einen Zugang zum gesamten Organ, bei dem die Auswirkungen der Behandlung entlang und außerhalb der Beamletpfade mit hoher Empfindlichkeit visualisiert werden können. Hier empfiehlt und bewährt sich die XPCI‑CT als leistungsstarke Bildverarbeitungstechnik, welche die Auswirkungen der Strahlung auf normale und tumortragende Gewebe feststellen und quantifizieren kann. Außerdem hat sich gezeigt, dass sie durch 3‑D-Informationen eine effektive Technik zur Ergänzung der histologischen Erkenntnisse in der Nachverfolgung der Strahlenbehandlung ist. Anhand eines mehrstufigen und multitechnischen röntgenbasierten Ansatzes habe ich die Auswirkungen der Strahlentherapie auf gesunde und von Glioblastomen (GBM) befallene Rattenhirne sowie auf gesunde Rattenlungen visualisiert und analysiert. Mit isotropen Voxelgrößen im Bereich von 3,53 bis 0,653 μm3 erfasste Ex-vivo-XPCI-CT-Datensätze konnten die idiopathischen Auswirkungen der MRT-, MB- und BB‑Behandlung mit hoher Empfindlichkeit unterscheiden. Es wurden auch Experimente zu Histologie, Immunhistochemie, Röntgenklein- und ‑weitwinkelstreuung und Röntgenfluoreszenz durchgeführt, um die XPCI‑CT-Erkenntnisse präzise zu interpretieren und zu ergänzen sowie eine detaillierte strukturelle und chemische Charakterisierung der nachgewiesenen pathologischen Merkmale zu erhalten. Im Allgemeinen wurde durch diesen multitechnischen Ansatz Folgendes ermittelt: i) eine un-terschiedliche Strahlenempfindlichkeit der mit MRT behandelten Gehirnbereiche; ii) Ca- und Fe-Ablagerungen und die Bildung von Hydroxylapatitkristallen; iii) ein ausgedehnter und isolierter Fibrosegehalt. XPCI‑CT-Datensätze des gesamten Organs ermöglichten die Quantifizierung der Volume von Tumoren und Mikroverkalkungen in den behandelten Gehirnen und der Menge des Narbengewebes in bestrahlten Lungen. Dabei wurde die Rolle der XPCI‑CT als virtuelle 3‑D-Histologietechnik für die Nachverfolgung von Ex-vivo-RT‑Auswirkungen als ergänzende Methode für eine präzise volumetrische Untersuchung des normalen und pathologischen Zustands von Gehirnen und Lungen im Kleintiermodell untersucht. Darüber hinaus wird die Technik aufgrund ihrer 3‑D-Fähigkeiten und der Möglichkeit zur virtuellen Navigation in den Proben als Leitfaden und Hilfstool für die herkömmliche Histologie vorgeschlagen, die der Goldstandard für die pathologische Evaluierung ist. Dies markiert einen Meilenstein für die Übernahme der XPCI‑CT in die Pipeline präklinischer Studien und für den Übergang zur In-vivo-XPCI‑CT von behandelten Organen

Feature-driven Volume Visualization of Medical Imaging Data

Direct volume rendering (DVR) is a volume visualization technique that has been proved to be a very powerful tool in many scientific visualization domains. Diagnostic medical imaging is one such domain in which DVR provides new capabilities for the analysis of complex cases and improves the efficiency of image interpretation workflows. However, the full potential of DVR in the medical domain has not yet been realized. A major obstacle for a better integration of DVR in the medical domain is the time-consuming process to optimize the rendering parameters that are needed to generate diagnostically relevant visualizations in which the important features that are hidden in image volumes are clearly displayed, such as shape and spatial localization of tumors, its relationship with adjacent structures, and temporal changes in the tumors. In current workflows, clinicians must manually specify the transfer function (TF), view-point (camera), clipping planes, and other visual parameters. Another obstacle for the adoption of DVR to the medical domain is the ever increasing volume of imaging data. The advancement of imaging acquisition techniques has led to a rapid expansion in the size of the data, in the forms of higher resolutions, temporal imaging acquisition to track treatment responses over time, and an increase in the number of imaging modalities that are used for a single procedure. The manual specification of the rendering parameters under these circumstances is very challenging. This thesis proposes a set of innovative methods that visualize important features in multi-dimensional and multi-modality medical images by automatically or semi-automatically optimizing the rendering parameters. Our methods enable visualizations necessary for the diagnostic procedure in which 2D slice of interest (SOI) can be augmented with 3D anatomical contextual information to provide accurate spatial localization of 2D features in the SOI; the rendering parameters are automatically computed to guarantee the visibility of 3D features; and changes in 3D features can be tracked in temporal data under the constraint of consistent contextual information. We also present a method for the efficient computation of visibility histograms (VHs) using adaptive binning, which allows our optimal DVR to be automated and visualized in real-time. We evaluated our methods by producing visualizations for a variety of clinically relevant scenarios and imaging data sets. We also examined the computational performance of our methods for these scenarios

Feature-driven Volume Visualization of Medical Imaging Data

Direct volume rendering (DVR) is a volume visualization technique that has been proved to be a very powerful tool in many scientific visualization domains. Diagnostic medical imaging is one such domain in which DVR provides new capabilities for the analysis of complex cases and improves the efficiency of image interpretation workflows. However, the full potential of DVR in the medical domain has not yet been realized. A major obstacle for a better integration of DVR in the medical domain is the time-consuming process to optimize the rendering parameters that are needed to generate diagnostically relevant visualizations in which the important features that are hidden in image volumes are clearly displayed, such as shape and spatial localization of tumors, its relationship with adjacent structures, and temporal changes in the tumors. In current workflows, clinicians must manually specify the transfer function (TF), view-point (camera), clipping planes, and other visual parameters. Another obstacle for the adoption of DVR to the medical domain is the ever increasing volume of imaging data. The advancement of imaging acquisition techniques has led to a rapid expansion in the size of the data, in the forms of higher resolutions, temporal imaging acquisition to track treatment responses over time, and an increase in the number of imaging modalities that are used for a single procedure. The manual specification of the rendering parameters under these circumstances is very challenging. This thesis proposes a set of innovative methods that visualize important features in multi-dimensional and multi-modality medical images by automatically or semi-automatically optimizing the rendering parameters. Our methods enable visualizations necessary for the diagnostic procedure in which 2D slice of interest (SOI) can be augmented with 3D anatomical contextual information to provide accurate spatial localization of 2D features in the SOI; the rendering parameters are automatically computed to guarantee the visibility of 3D features; and changes in 3D features can be tracked in temporal data under the constraint of consistent contextual information. We also present a method for the efficient computation of visibility histograms (VHs) using adaptive binning, which allows our optimal DVR to be automated and visualized in real-time. We evaluated our methods by producing visualizations for a variety of clinically relevant scenarios and imaging data sets. We also examined the computational performance of our methods for these scenarios

Compression of 4D medical image and spatial segmentation using deformable models

Ph.DDOCTOR OF PHILOSOPH

Synergistic Visualization And Quantitative Analysis Of Volumetric Medical Images

The medical diagnosis process starts with an interview with the patient, and continues with the physical exam. In practice, the medical professional may require additional screenings to precisely diagnose. Medical imaging is one of the most frequently used non-invasive screening methods to acquire insight of human body. Medical imaging is not only essential for accurate diagnosis, but also it can enable early prevention. Medical data visualization refers to projecting the medical data into a human understandable format at mediums such as 2D or head-mounted displays without causing any interpretation which may lead to clinical intervention. In contrast to the medical visualization, quantification refers to extracting the information in the medical scan to enable the clinicians to make fast and accurate decisions. Despite the extraordinary process both in medical visualization and quantitative radiology, efforts to improve these two complementary fields are often performed independently and synergistic combination is under-studied. Existing image-based software platforms mostly fail to be used in routine clinics due to lack of a unified strategy that guides clinicians both visually and quan- titatively. Hence, there is an urgent need for a bridge connecting the medical visualization and automatic quantification algorithms in the same software platform. In this thesis, we aim to fill this research gap by visualizing medical images interactively from anywhere, and performing a fast, accurate and fully-automatic quantification of the medical imaging data. To end this, we propose several innovative and novel methods. Specifically, we solve the following sub-problems of the ul- timate goal: (1) direct web-based out-of-core volume rendering, (2) robust, accurate, and efficient learning based algorithms to segment highly pathological medical data, (3) automatic landmark- ing for aiding diagnosis and surgical planning and (4) novel artificial intelligence algorithms to determine the sufficient and necessary data to derive large-scale problems