Multi-dimensional volume rendering for PC- based medical simulation

Ph.DDOCTOR OF PHILOSOPH

Visualisation of multi-dimensional medical images with application to brain electrical impedance tomography

Medical imaging plays an important role in modem medicine. With the increasing complexity and information presented by medical images, visualisation is vital for medical research and clinical applications to interpret the information presented in these images. The aim of this research is to investigate improvements to medical image visualisation, particularly for multi-dimensional medical image datasets. A recently developed medical imaging technique known as Electrical Impedance Tomography (EIT) is presented as a demonstration. To fulfil the aim, three main efforts are included in this work. First, a novel scheme for the processmg of brain EIT data with SPM (Statistical Parametric Mapping) to detect ROI (Regions of Interest) in the data is proposed based on a theoretical analysis. To evaluate the feasibility of this scheme, two types of experiments are carried out: one is implemented with simulated EIT data, and the other is performed with human brain EIT data under visual stimulation. The experimental results demonstrate that: SPM is able to localise the expected ROI in EIT data correctly; and it is reasonable to use the balloon hemodynamic change model to simulate the impedance change during brain function activity. Secondly, to deal with the absence of human morphology information in EIT visualisation, an innovative landmark-based registration scheme is developed to register brain EIT image with a standard anatomical brain atlas. Finally, a new task typology model is derived for task exploration in medical image visualisation, and a task-based system development methodology is proposed for the visualisation of multi-dimensional medical images. As a case study, a prototype visualisation system, named EIT5DVis, has been developed, following this methodology. to visualise five-dimensional brain EIT data. The EIT5DVis system is able to accept visualisation tasks through a graphical user interface; apply appropriate methods to analyse tasks, which include the ROI detection approach and registration scheme mentioned in the preceding paragraphs; and produce various visualisations

Visualisierung zweidimensionaler Volumen

In dieser Arbeit wird ein neues Verfahren zur Visualisierung zweidimensionaler Volumen vorgestellt. Der Begriff multidimensionales Volumen wird dabei definiert als eine Menge von räumlich dreidimensionalen Datensätzen, die jeder eine andere Eigenschaft (eine physikalische Qualität, z.B. Dichte oder Temperatur) desselben Objekts beschreiben. Zweidimensionale Volumen beschreiben also zwei verschiedene Eigenschaften eines Objekts. Sie entstehen z.B. in biomedizinischen Anwendungen, wenn gleichzeitig funktionale und anatomische Datensätze untersucht werden. Zunächst wird der Stand der Technik in der Visualisierung zweidimensionaler Volumen dargelegt. Dabei sind besonders die folgenden Schwächen bestehender Verfahren erkennbar: - Schlechte räumliche Darstellung und schlechte Lokalisierbarkeit von Ausprägungen (bemerkenswerte Quantitäten einer Eigenschaft an einer Stelle). Beschränkung auf Datensätze aus speziellen Quellen oder spezielle Kombinationen von Datensätzen. - Prinzipbedingte Beschränkung einer Eigenschaft auf wenige kleine Regionen innerhalb der anderen Eigenschaft. Basierend auf diesen Defiziten werden die Anforderungen für ein besseres Visualisierungsverfahren herausgearbeitet, anhand derer ein neues Verfahren, dependent rendering genannt, entwickelt wird. Das Verfahren basiert auf der Annahme, dass bei der Visualisierung mehrerer Eigenschaften immer eine Eigenschaft als Referenz zur Lokalisierung dienen kann. Abhängig von der ersten kann eine weitere Eigenschaft visualisiert werden. Es werden drei Implementierungen des Verfahrens vorgestellt, die ersten beiden sind Prototypen, die dritte eine spezialisierte Anwendung für eine biomedizinische Visualisierungsplattform. Die Implementierungen veranschaulichen, dass sich das vorgestellte Verfahren gegenüber bestehenden Ansätzen besonders durch folgende Punkte auszeichnet: - Gute Lokalisierbarkeit von Ausprägungen bei gleichzeitiger guter räumlicher Darstellung des Objekts (z.B.: "Ist es auf der Oberfläche heiss oder innerhalb des Objekts?"). - Gleiche räumliche Ausdehnung beider Datensätze möglich. - Genereller Ansatz: Keine Beschränkung auf Datensätze aus speziellen Quellen oder auf spezielle Kombinationen von Datensätzen. Das vorgestellte Verfahren stellt daher einen bedeutenden Fortschritt in der Technik der Visualisierung zweier Eigenschaften eines Objekts dar.In this thesis, a new technique for the visualisation of two-dimensional volumes is presented. The term multi-dimensional volume is defined as a set of spatially three-dimensional data sets, each of them describing another property (a physical quality, e.g. density or temperature) of the same object. Thus, two-dimensional volumes describe two different properties of an object. They are used e.g. in biomedical imaging, where anatomical and functional data are examined jointly. First, the state of the art in the visualisation of two-dimensional volumes is presented. In the course of this, the following deficiencies of existing approaches become apparent: - Unsatisfactory 3D impression (it is difficult to mentally reconstruct the spatially three-dimensional object from the rendering) and difficult localisation of features (i.e. remarkable characteristics in the quantity of a property at a given location). - Restriction to data sets from particular origins or particular combinations of data sets. - By design, one property is restricted to only a few small regions inside the other property. Starting from these deficiencies, the requirements for a visualisation technique that overcomes these limitations are elaborated. These are then used to develop a new technique, called dependent rendering, which is based on the assumption that, when visualising two properties of an object, there is alway one property that can serve as a spatial reference for the other. The other property is then visualised in dependency on this reference. Three implementations of the technique are presented, the first two are prototypes, the third one is a specialised application for a biomedical visualisation platform. The implementations show that, compared to existing approaches, the presented technique especially stands out because of the following features: - Precise localisation of features combined with good 3D impression of the object (e.g. "Is it hot on the surface or only inside the object?"). - Both data sets can be extended over the same region. - General approach: No restriction to data sets from particular origins or particular combinations of data sets. The presented technique therefore represents an important advancement in the joint visualisation of two properties of an object

Efficient rendering of large 3-D and 4-D scalar fields

Rendering volumetric data, as a compute/communication intensive and highly parallel application, represents the characteristics of future workloads for desktop computers. Interactively rendering volumetric data has been a challenging problem due to its high computational and communication requirements. With the consistent trend toward high resolution data, it has remained a difficult problem despite the continuous increase in processing power, because of the increasing performance gap between computation and communication. On the other hand, the new multi-core architecture trend in computational units in PC, which can be characterized by parallelism and heterogeneity, provides both opportunities and challenges. While the new on-chip parallel architectures offer opportunities for extremely high performance, widespread use of those parallel processors requires extensive changes in previous algorithms to take advantage of the new architectures. In this dissertation, we develop new methods and techniques to support interactive rendering of large volumetric data. In particular, we present a novel method to layout data on disk for efficiently performing an out-of-core axis-aligned slicing of large multidimensional scalar fields. We also present a new method to efficiently build an out-of-core indexing structure for n-dimensional volumetric data. Then, we describe a streaming model for efficiently implementing volume ray casting on a heterogeneous compute resource environment. We describe how we implement the model on SONY/TOSHIBA/IBM Cell Broadband Engine and on NVIDIA CUDA architecture. Our results show that our out-of-core techniques significantly reduce the communication bandwidth requirements and that our streaming model very effectively makes use of the strengths of those heterogeneous parallel compute resource environment for volume rendering. In all cases, we achieve scalability and load balancing, while hiding memory latency

Fast Compressed Segmentation Volumes for Scientific Visualization

Voxel-based segmentation volumes often store a large number of labels and voxels, and the resulting amount of data can make storage, transfer, and interactive visualization difficult. We present a lossless compression technique which addresses these challenges. It processes individual small bricks of a segmentation volume and compactly encodes the labelled regions and their boundaries by an iterative refinement scheme. The result for each brick is a list of labels, and a sequence of operations to reconstruct the brick which is further compressed using rANS-entropy coding. As the relative frequencies of operations are very similar across bricks, the entropy coding can use global frequency tables for an entire data set which enables efficient and effective parallel (de)compression. Our technique achieves high throughput (up to gigabytes per second both for compression and decompression) and strong compression ratios of about 1% to 3% of the original data set size while being applicable to GPU-based rendering. We evaluate our method for various data sets from different fields and demonstrate GPU-based volume visualization with on-the-fly decompression, level-of-detail rendering (with optional on-demand streaming of detail coefficients to the GPU), and a caching strategy for decompressed bricks for further performance improvement.Comment: IEEE Vis 202

Qualifying 4D Deforming Surfaces by Registered Differential Features

Institute of Perception, Action and BehaviourRecent advances in 4D data acquisition systems in the field of Computer Vision have opened up many exciting new possibilities for the interpretation of complex moving surfaces. However, a fundamental problem is that this has also led to a huge increase in the volume of data to be handled. Attempting to make sense of this wealth of information is then a core issue to be addressed if such data can be applied to more complex tasks. Similar problems have been historically encountered in the analysis of 3D static surfaces, leading to the extraction of higher-level features based on analysis of the differential geometry.Our central hypothesis is that there exists a compact set of similarly useful descriptors for the analysis of dynamic 4D surfaces. The primary advantages in considering localised changes are that they provide a naturally useful set of invariant characteristics. We seek a constrained set of terms - a vocabulary - for describing all types of deformation. By using this, we show how to describe what the surface is doing more effectively; and thereby enable better characterisation, and consequently more effective visualisation and comparison.This thesis investigates this claim. We adopt a bottom-up approach of the problem, in which we acquire raw data from a newly constructed commercial 4D data capture system developed by our industrial partners. A crucial first step resolves the temporal non-linear registration between instances of the captured surface. We employ a combined optical/range flow to guide a conformation over a sequence. By extending the use of aligned colour information alongside the depth data we improve this estimation in the case of local surface motion ambiguities. By employing a KLT/thin-plate-spline method we also seek to preserve global deformation for regions with no estimate.We then extend aspects of differential geometry theory for existing static surface analysis to the temporal domain. Our initial formulation considers the possible intrinsic transitions from the set of shapes defined by the variations in the magnitudes of the principal curvatures. This gives rise to a total of 15 basic types of deformation. The change in the combined magnitudes also gives an indication of the extent of change. We then extend this to surface characteristics associated with expanding, rotating and shearing; to derive a full set of differential features.Our experimental results include qualitative assessment of deformations for short episodic registered sequences of both synthetic and real data. The higher-level distinctions extracted are furthermore a useful first step for parsimonious feature extraction, which we then proceed to demonstrate can be used as a basis for further analysis. We ultimately evaluate this approach by considering shape transition features occurring within the human face, and the applicability for identification and expression analysis tasks

A volume filtering and rendering system for an improved visual balance of feature preservation and noise suppression in medical imaging

Preserving or enhancing salient features whilst effectively suppressing noise-derived artifacts and extraneous detail have been two consistent yet competing objectives in volumetric medical image processing. Illustrative techniques (and methods inspired by them) can help to enhance and, if desired, isolate the depiction of specific regions of interest whilst retaining overall context. However, highlighting or enhancing specific features can have the undesirable side-effect of highlighting noise. Second-derivative based methods can be employed effectively in both the rendering and volume filtering stages of a visualisation pipeline to enhance the depiction of feature detail whilst minimising noise-based artifacts. We develop a new 3D anisotropic-diffusion PDE for an improved balance of feature-retention and noise reduction; furthermore, we present a feature-enhancing visualisation pipeline that can be applied to multiple modalities and has been shown to be particularly effective in the context of 3D ultrasound

CT investigations of Australian Devonian fossil fishes, and the application of 3D segmentation and modelling in vertebrate morphology

This thesis by compilation covers four publications, which together demonstrate and apply computed tomography (CT) data, three-dimensional (3D) segmentation, and 3D printing, for the non-destructive high-resolution detection of internal structure in early vertebrate fossils. Paper 1 (published 2019) studied the tetrapodomorph fish, Gogonasus, a three dimensionally well-preserved acid etched Devonian sarcopterygian. 3D modelling and printing were used for the reconstruction of its shoulder girdle and opercular series. A close fit of the opercular series against the upper bones of the shoulder girdle required a more horizontally aligned anocleithrum, supracleithrum and post-temporal than in previous reconstructions. Subopercular bone 2 is partly covered the clavicle, and the ascending process of the clavicle, and the ventral process of the anocleithrum, do not fit closely inside the cleithrum, suggesting they may have functioned for ligamentous attachment. A rugose area on the anocleithral process has a similar relative position to muscle ligament attachements in the shoulder girdle of various living actinopterygians. The manipulation of 3D models demonstrates testing of morphological fit for extremely fragile acid-etched bones. Paper 2 (published 2017) dealt with an articulated Devonian placoderm. Micro CT, 3D segmentation, and enlarged 3D models of a buchanosteid arthrodire, demonstrated a double ethmoid and a palatobasal connection for the palatoquadrate, but no otic connection. A separately ossified cartilage behind the mandibular joint is comparable to the interhyal of osteichthyans. Two articular facets on the braincase associated with the hyomandibular nerve foramen supported a possible epihyal element and a separate opercular cartilage. Reassembling 3D printouts demonstrates the limits of jaw kinetics. New details on the hyoid arch will help to reformulate characters that are key in the heated debate of placoderm monophyly or paraphyly. Paper 3 (published 2019) describes the detailed morphology of the three gnathal elements of the same specimen as on the previous paper, giving insights into the morphology and organization of the dentition in arthrodires. Arthrodire placoderms, as a possible sister group of Chinese maxillate placoderms plus crown gnathostomes, provide important information regarding early evolution of jaws and teeth. In displaying numerous denticle rows, the gnathal element morphology is different from the much-reduced denticulation of higher brachythoracid arthrodires. Ossification centres are anterolateral on the anterior supragnathal (attached to the braincase), anteromesial on the posterior supragnathal (attached to the palatoquadrate), and in the central part of the biting portion of the infragnathal (attached to the meckelian cartilage). The infragnathal shows no evidence of two ossification centres, as has been interpreted for more advanced arthrodires. The new evidence gives insights into the primitive arthrodire condition for comparison with the dermal jaw bones of Chinese 'maxillate' placoderms that have been homologised with the premaxilla, maxilla, and dentary of osteichthyans. Paper 4 (published 2020) summarizes the relationship between different modules of Drishti, an open-source volume exploration, rendering and three-dimensional segmentation software program that was used in the previous three published papers. A new version (Drishti v2.7) is presented, with a new tool for thresholding volume data (i.e. gradient thresholding). A protocol is introduced for performing three-dimensional segmentation using the new 3D Freeform Painter tool. In Drishti Paint, these new tools and workflow enable more accurate and precise digital reconstruction, 3D modelling and three-dimensional printing/modelling results. Scan data from the buchanosteus arthrodire (Paper 2 and 3) was used as a case study but published procedure is widely applicable in biological, medical and industrial research