Integrated navigation and visualisation for skull base surgery

Skull base surgery involves the management of tumours located on the underside of the brain and the base of the skull. Skull base tumours are intricately associated with several critical neurovascular structures making surgery challenging and high risk. Vestibular schwannoma (VS) is a benign nerve sheath tumour arising from one of the vestibular nerves and is the commonest pathology encountered in skull base surgery. The goal of modern VS surgery is maximal tumour removal whilst preserving neurological function and maintaining quality of life but despite advanced neurosurgical techniques, facial nerve paralysis remains a potentially devastating complication of this surgery. This thesis describes the development and integration of various advanced navigation and visualisation techniques to increase the precision and accuracy of skull base surgery. A novel Diffusion Magnetic Resonance Imaging (dMRI) acquisition and processing protocol for imaging the facial nerve in patients with VS was developed to improve delineation of facial nerve preoperatively. An automated Artificial Intelligence (AI)-based framework was developed to segment VS from MRI scans. A user-friendly navigation system capable of integrating dMRI and tractography of the facial nerve, 3D tumour segmentation and intraoperative 3D ultrasound was developed and validated using an anatomically-realistic acoustic phantom model of a head including the skull, brain and VS. The optical properties of five types of human brain tumour (meningioma, pituitary adenoma, schwannoma, low- and high-grade glioma) and nine different types of healthy brain tissue were examined across a wavelength spectrum of 400 nm to 800 nm in order to inform the development of an Intraoperative Hypserpectral Imaging (iHSI) system. Finally, functional and technical requirements of an iHSI were established and a prototype system was developed and tested in a first-in-patient study

Real-time and post-processed georeferencing for hyperpspectral drone remote sensing

The use of drones and photogrammetric technologies are increasing rapidly in different applications. Currently, drone processing workflow is in most cases based on sequential image acquisition and post-processing, but there are great interests towards real-time solutions. Fast and reliable real-time drone data processing can benefit, for instance, environmental monitoring tasks in precision agriculture and in forest. Recent developments in miniaturized and low-cost inertial measurement systems and GNSS sensors, and Real-time kinematic (RTK) position data are offering new perspectives for the comprehensive remote sensing applications. The combination of these sensors and light-weight and low-cost multi- or hyperspectral frame sensors in drones provides the opportunity of creating near real-time or real-time remote sensing data of target object. We have developed a system with direct georeferencing onboard drone to be used combined with hyperspectral frame cameras in real-time remote sensing applications. The objective of this study is to evaluate the real-time georeferencing comparing with post-processing solutions. Experimental data sets were captured in agricultural and forested test sites using the system. The accuracy of onboard georeferencing data were better than 0.5 m. The results showed that the real-time remote sensing is promising and feasible in both test sites. © Authors 2018. CC BY 4.0 License.Peer reviewe

An investigation into the complementary capabilities of X-ray computed tomography and hyperspectral imaging of drill core in geometallurgy

The mining industry is faced with the challenge of mining and processing low grade, heterogeneous, and complex ores, a phenomenon known as ore variability. These ores need to be managed at an early operational stage, ideally during drill core exploration, to avoid risks during the project phase (such as project delays and failure) and operational phases (such as plant instabilities), ultimately affecting the cash flow. The discipline of geometallurgy has arisen to manage the risks associated with ore variability by acquiring upfront knowledge of the mineral assemblage and texture before mining and processing. As we head towards the fourth industrial revolution (4IR), machine learning, intensive and automated data derived from drill cores are becoming more common. In this case, using non-destructive, rapid, and inexpensive automated scanning techniques such as 2D hyperspectral imaging (HSI) and 3D Xray computed tomography (XCT) have the potential to be incorporated into the machine learning dataset. Hyperspectral imaging is a critical component of continuous drill core scanning in geometallurgy for identifying problematic minerals in downstream mineral processing, such as the phyllosilicates (e.g., kaolinite, serpentine and talc). However, it only provides 2D imaging of the core, and its mineral identification is limited to minerals that show a definitive spectral response. On the other hand, XCT provides 3D imaging of drill cores, but is more routinely used in research applications and does not independently give the mineral assemblage. Mineral identification and discrimination for XCT is limited and requires prior mineralogical knowledge and sufficient mineral density and attenuation coefficient variation greater than 6%. No systematic study to date appears to have explored how the results from these two techniques can be integrated using a local South African magmatic nickel-copper-platinum group element (Ni-Cu-PGE) ore case study. This opened an opportunity to couple the two techniques to address and emphasize the image scanning techniques for drill core in geometallurgy and to provide further knowledge on the practicality of the HSI and XCT in drill core from image acquisition to processing. Ultimately, the aim is to investigate how well the techniques complement each other for mineral and texture identifications and, if combined, will produce additional mineralogical and textural information. The objective of this study was achieved by moving HSI cores to smaller samples than standard practice to produce 25 mm diameter mini cores instead of standard cores (e.g., 50 mm in diameter). For accurate mineral assemblage and textural characterisation of the drill cores, manual core logging, quantitative evaluation of minerals by scanning electron microscopy (QEMSCAN) and quantitative X-ray diffraction (QXRD) were used as supporting techniques. The results showed HSI scanning on the magmatic Ni-Cu-PGE drill core to be challenging because of pervasive mineral alteration and the nature of the rock types (mafic and ultra-mafic rocks) - providing limited information on the mineral assemblage and texture due to low scanning resolution and pervasive alteration (serpentinisation and chloritization) in the rocks. The limited mineral identification includes mixed-phases (such as serpentine-olivine in visible-shortwave infrared and plagioclase-chlorite in the longwave infrared) and unclassified minerals in the core. The resultant mineral assemblage was comparable to QEMSCAN and QXRD in terms of minerals present with generally similar abundances. However, useful information on the alteration mineralogy can still be extracted, such as the presence of serpentine, chlorite and talc and their association with other silicate minerals. Other parameters such as mineral grades and grain sizes were quantified on MATLAB using specially developed scripts. The interconnected grains could not be separated due to invisible boundaries on the HSI maps. Therefore, only a small number of grains were generated with larger grain size values, likely underestimating the real grain numbers. XCT provided information on valuable high-density minerals (including possible platinum-group minerals (PGMs)) and mineral texture in the cores. Due to extensive alteration in the rocks, discrimination between grey values was, however, challenging. Grey level segmentation into the different mineral groups was also noted to be dependent on the rock type. For example, plagioclase and orthopyroxene were more easily discriminated in the less altered rocks (feldspathic pyroxenite and anorthosite) than the more altered rocks (altered harzburgite and pegmatoidal pyroxenite). The high scanning resolution allowed for the extraction of mineral texture, such as mineral association and grain size distribution (GSD). The 3D XCT derived GSD was slightly coarser than the 2D QEMSCAN derived GSD. The differences in GSD are attributed to a combination of both stereological and sampling effects. However, sufficient information on ore variability can be obtained when using the pertinent scanning parameters and careful segmentation processes. These two techniques provide variable information on the mineral assemblage and texture, such as the identification of silicate minerals (particularly alteration minerals) in HSI and high-density minerals in XCT and good textural information on XCT than HSI. With the information provided, possible image overlapping scenarios of the two techniques were identified: (1) using XCT for high-density minerals, and HSI for silicate identification, (2) using XCT data with good mineral and texture discrimination (silicate associated with sulphides) to map unclassified areas in HSI, (3) is the opposite of the second scenario. Ultimately, the two scanning techniques will likely offer complementary information, although the application of this combined technique for routine work will be limited in practicality. Additionally, more work needs to be carried out with revised scanning and processing to improve the sustainability of the techniques in geometallurgy

Multisource and Multitemporal Data Fusion in Remote Sensing

The sharp and recent increase in the availability of data captured by different sensors combined with their considerably heterogeneous natures poses a serious challenge for the effective and efficient processing of remotely sensed data. Such an increase in remote sensing and ancillary datasets, however, opens up the possibility of utilizing multimodal datasets in a joint manner to further improve the performance of the processing approaches with respect to the application at hand. Multisource data fusion has, therefore, received enormous attention from researchers worldwide for a wide variety of applications. Moreover, thanks to the revisit capability of several spaceborne sensors, the integration of the temporal information with the spatial and/or spectral/backscattering information of the remotely sensed data is possible and helps to move from a representation of 2D/3D data to 4D data structures, where the time variable adds new information as well as challenges for the information extraction algorithms. There are a huge number of research works dedicated to multisource and multitemporal data fusion, but the methods for the fusion of different modalities have expanded in different paths according to each research community. This paper brings together the advances of multisource and multitemporal data fusion approaches with respect to different research communities and provides a thorough and discipline-specific starting point for researchers at different levels (i.e., students, researchers, and senior researchers) willing to conduct novel investigations on this challenging topic by supplying sufficient detail and references

Advances in automated tongue diagnosis techniques

This paper reviews the recent advances in a significant constituent of traditional oriental medicinal technology, called tongue diagnosis. Tongue diagnosis can be an effective, noninvasive method to perform an auxiliary diagnosis any time anywhere, which can support the global need in the primary healthcare system. This work explores the literature to evaluate the works done on the various aspects of computerized tongue diagnosis, namely preprocessing, tongue detection, segmentation, feature extraction, tongue analysis, especially in traditional Chinese medicine (TCM). In spite of huge volume of work done on automatic tongue diagnosis (ATD), there is a lack of adequate survey, especially to combine it with the current diagnosis trends. This paper studies the merits, capabilities, and associated research gaps in current works on ATD systems. After exploring the algorithms used in tongue diagnosis, the current trend and global requirements in health domain motivates us to propose a conceptual framework for the automated tongue diagnostic system on mobile enabled platform. This framework will be able to connect tongue diagnosis with the future point-of-care health system

Advancing fluorescent contrast agent recovery methods for surgical guidance applications

Fluorescence-guided surgery (FGS) utilizes fluorescent contrast agents and specialized optical instruments to assist surgeons in intraoperatively identifying tissue-specific characteristics, such as perfusion, malignancy, and molecular function. In doing so, FGS represents a powerful surgical navigation tool for solving clinical challenges not easily addressed by other conventional imaging methods. With growing translational efforts, major hurdles within the FGS field include: insufficient tools for understanding contrast agent uptake behaviors, the inability to image tissue beyond a couple millimeters, and lastly, performance limitations of currently-approved contrast agents in accurately and rapidly labeling disease. The developments presented within this thesis aim to address such shortcomings. Current preclinical fluorescence imaging tools often sacrifice either 3D scale or spatial resolution. To address this gap in high-resolution, whole-body preclinical imaging tools available, the crux of this work lays on the development of a hyperspectral cryo-imaging system and image-processing techniques to accurately recapitulate high-resolution, 3D biodistributions in whole-animal experiments. Specifically, the goal is to correct each cryo-imaging dataset such that it becomes a useful reporter for whole-body biodistributions in relevant disease models. To investigate potential benefits of seeing deeper during FGS, we investigated short-wave infrared imaging (SWIR) for recovering fluorescence beyond the conventional top few millimeters. Through phantom, preclinical, and clinical SWIR imaging, we were able to 1) validate the capability of SWIR imaging with conventional NIR-I fluorophores, 2) demonstrate the translational benefits of SWIR-ICG angiography in a large animal model, and 3) detect micro-dose levels of an EGFR-targeted NIR-I probe during a Phase 0 clinical trial. Lastly, we evaluated contrast agent performances for FGS glioma resection and breast cancer margin assessment. To evaluate glioma-labeling performance of untargeted contrast agents, 3D agent biodistributions were compared voxel-by-voxel to gold-standard Gd-MRI and pathology slides. Finally, building on expertise in dual-probe ratiometric imaging at Dartmouth, a 10-pt clinical pilot study was carried out to assess the technique’s efficacy for rapid margin assessment. In summary, this thesis serves to advance FGS by introducing novel fluorescence imaging devices, techniques, and agents which overcome challenges in understanding whole-body agent biodistributions, recovering agent distributions at greater depths, and verifying agents’ performance for specific FGS applications

Hyperspectral benthic mapping from underwater robotic platforms

We live on a planet of vast oceans; 70% of the Earth's surface is covered in water. They are integral to supporting life, providing 99% of the inhabitable space on Earth. Our oceans and the habitats within them are under threat due to a variety of factors. To understand the impacts and possible solutions, the monitoring of marine habitats is critically important. Optical imaging as a method for monitoring can provide a vast array of information however imaging through water is complex. To compensate for the selective attenuation of light in water, this thesis presents a novel light propagation model and illustrates how it can improve optical imaging performance. An in-situ hyperspectral system is designed which comprised of two upward looking spectrometers at different positions in the water column. The downwelling light in the water column is continuously sampled by the system which allows for the generation of a dynamic water model. In addition to the two upward looking spectrometers the in-situ system contains an imaging module which can be used for imaging of the seafloor. It consists of a hyperspectral sensor and a trichromatic stereo camera. New calibration methods are presented for the spatial and spectral co-registration of the two optical sensors. The water model is used to create image data which is invariant to the changing optical properties of the water and changing environmental conditions. In this thesis the in-situ optical system is mounted onboard an Autonomous Underwater Vehicle. Data from the imaging module is also used to classify seafloor materials. The classified seafloor patches are integrated into a high resolution 3D benthic map of the surveyed site. Given the limited imaging resolution of the hyperspectral sensor used in this work, a new method is also presented that uses information from the co-registered colour images to inform a new spectral unmixing method to resolve subpixel materials