Imaging through obscurants using time-correlated single-photon counting in the short-wave infrared

Single-photon time-of-flight (ToF) light detection and ranging (LiDAR) systems have emerged in recent years as a candidate technology for high-resolution depth imaging in challenging environments, such as long-range imaging and imaging in scattering media. This Thesis investigates the potential of two ToF single-photon depth imaging systems based on the time-correlated single-photon (TCSPC) technique for imaging targets in highly scattering environments. The high sensitivity and picosecond timing resolution afforded by the TCSPC technique offers high-resolution depth profiling of remote targets while maintaining low optical power levels. Both systems comprised a pulsed picosecond laser source with an operating wavelength of 1550 nm, and employed InGaAs/InP SPAD detectors. The main benefits of operating in the shortwave infrared (SWIR) band include improved atmospheric transmission, reduced solar background, as well as increased laser eye-safety thresholds over visible band sensors. Firstly, a monostatic scanning transceiver unit was used in conjunction with a single-element Peltier-cooled InGaAs/InP SPAD detector to attain sub-centimetre resolution three-dimensional images of long-range targets obscured by camouflage netting or in high levels of scattering media. Secondly, a bistatic system, which employed a 32 × 32 pixel format InGaAs/InP SPAD array was used to obtain rapid depth profiles of targets which were flood-illuminated by a higher power pulsed laser source. The performance of this system was assessed in indoor and outdoor scenarios in the presence of obscurants and high ambient background levels. Bespoke image processing algorithms were developed to reconstruct both the depth and intensity images for data with very low signal returns and short data acquisition times, illustrating the practicality of TCSPC-based LiDAR systems for real-time image acquisition in the SWIR wavelength region - even in the photon-starved regime.The Defence Science and Technology Laboratory ( Dstl) National PhD Schem

Advancing fluorescent contrast agent recovery methods for surgical guidance applications

Fluorescence-guided surgery (FGS) utilizes fluorescent contrast agents and specialized optical instruments to assist surgeons in intraoperatively identifying tissue-specific characteristics, such as perfusion, malignancy, and molecular function. In doing so, FGS represents a powerful surgical navigation tool for solving clinical challenges not easily addressed by other conventional imaging methods. With growing translational efforts, major hurdles within the FGS field include: insufficient tools for understanding contrast agent uptake behaviors, the inability to image tissue beyond a couple millimeters, and lastly, performance limitations of currently-approved contrast agents in accurately and rapidly labeling disease. The developments presented within this thesis aim to address such shortcomings. Current preclinical fluorescence imaging tools often sacrifice either 3D scale or spatial resolution. To address this gap in high-resolution, whole-body preclinical imaging tools available, the crux of this work lays on the development of a hyperspectral cryo-imaging system and image-processing techniques to accurately recapitulate high-resolution, 3D biodistributions in whole-animal experiments. Specifically, the goal is to correct each cryo-imaging dataset such that it becomes a useful reporter for whole-body biodistributions in relevant disease models. To investigate potential benefits of seeing deeper during FGS, we investigated short-wave infrared imaging (SWIR) for recovering fluorescence beyond the conventional top few millimeters. Through phantom, preclinical, and clinical SWIR imaging, we were able to 1) validate the capability of SWIR imaging with conventional NIR-I fluorophores, 2) demonstrate the translational benefits of SWIR-ICG angiography in a large animal model, and 3) detect micro-dose levels of an EGFR-targeted NIR-I probe during a Phase 0 clinical trial. Lastly, we evaluated contrast agent performances for FGS glioma resection and breast cancer margin assessment. To evaluate glioma-labeling performance of untargeted contrast agents, 3D agent biodistributions were compared voxel-by-voxel to gold-standard Gd-MRI and pathology slides. Finally, building on expertise in dual-probe ratiometric imaging at Dartmouth, a 10-pt clinical pilot study was carried out to assess the technique’s efficacy for rapid margin assessment. In summary, this thesis serves to advance FGS by introducing novel fluorescence imaging devices, techniques, and agents which overcome challenges in understanding whole-body agent biodistributions, recovering agent distributions at greater depths, and verifying agents’ performance for specific FGS applications

A Compact, High Resolution Hyperspectral Imager for Remote Sensing of Soil Moisture

Measurement of soil moisture content is a key challenge across a variety of fields, ranging from civil engineering through to defence and agriculture. While dedicated satellite platforms like SMAP and SMOS provide high spatial coverage, their low spatial resolution limits their application to larger regional studies. The advent of compact, high lift capacity UAVs has enabled small scale surveys of specific farmland cites. This thesis presents work on the development of a compact, high spatial and spectral resolution hyperspectral imager, designed for remote measurement of soil moisture content. The optical design of the system incorporates a bespoke freeform blazed diffraction grating, providing higher optical performance at a similar aperture to conventional Offner-Chrisp designs. The key challenges of UAV-borne hyperspectral imaging relate to using only solar illumination, with both intermittent cloud cover and atmospheric water absorption creating challenges in obtaining accurate reflectance measurements. A hardware based calibration channel for mitigating cloud cover effects is introduced, along with a comparison of methods for recovering soil moisture content from reflectance data under varying illumination conditions. The data processing pipeline required to process the raw pushbroom data into georectified images is also discussed. Finally, preliminary work on applying soil moisture techniques to leaf imaging are presented

Investigations into applications of photometric stereo and single-pixel imaging

Computational image reconstruction is generally an inverse procedure which helps to recover the original information in a scene. Various imaging techniques have been developed to extract certain kinds of information for applications in different fields. The focus of this thesis is to improve two elegant and powerful methods among those approaches, namely, photometric stereo and single-pixel imaging, into a more practical and applicable phase. With the advances in modern imaging technology, 3D information is playing an increasingly significant role in real-world applications, from robotic vision, manufacturing, entertainment, and biology to security. While an immense amount of research has been conducted over the last few decades, the requirement of generating a rapid and accurate estimation of scene depth information with a cost-efficient system remains challenging. In the first work, we developed an inexpensive computational camera system allowing fast 3D reconstruction of objects based on the principle of photometric stereo. By analysing the estimated 3D data of various objects, we noticed good quantitative agreement with the known reference object with a wide viewing angle. With a low-cost accessory, our system provides a simplified reconstruction routine alongside a high efficiency, which extends its portability and capability for practical applications. Single-pixel imaging is an emerging paradigm which utilises spatial correlation of light with a single-pixel detector to form an image. It provides an alternative strategy to conventional imaging techniques which reply on a pixelated sensor for spatial resolution. In the second work, we combined photometric stereo with single-pixel imaging to evolve a new 3D imaging system with an efficient realtime sampling scheme. By utilising a high-speed structured illumination and four single-pixel detectors, multiple images of a scene with different shading profiles were able to be reconstructed with perfect pixel registration for depth estimation, empowering 3D imaging of dynamic scene. A compressive strategy, known as evolutionary compressed sensing, was further employed to improve the frame rate of 3D single-pixel video at an expense of only a modest reduction in image quality. This system represents a step-forward towards real-time 3D single-pixel imaging. By using single-pixel imaging technique, it offers a feasible solution for situations that are costly or constrained with conventional pixelated camera sensor, for instance, near-infrared (NIR) imaging and fluorescence imaging through multimode fibres. However, the signal-to-noise ratio (SNR) scales poorly when increasing the single-pixel imaging resolution. In the last work, we developed a NIR single-pixel imaging system with micro-scanning, an optimisation approach that generates a higher-resolution image while maintaining the SNR of the lower-resolution images where it is derived from. With the use of sunlight and an infrared heat lamp as the illumination sources and a set of NIR bandpass filters, our system indicated a well capability of revealing the water absorption underneath the surfaces of plant leaves and fruits compared to an expensive pixelated InGaAs camera. Additional efforts were devoted to further improve the image quality of a modified single-pixel imaging system that allows visible and NIR dual-band detection simultaneously

The development of optical projection tomography instrumentation and its application to in vivo three dimensional imaging of zebrafish

OPT is a three dimensional (3D) imaging technique that can produce 3D reconstructions of transparent samples, requiring only a widefield imaging system and sample rotation. OPT can be readily applied to chemically cleared samples, or to live transparent organisms such as nematodes or zebrafish. For preclinical imaging, there is a trade-off between optical accessibility and biological relevance to humans. Adult Danio rerio (zebrafish) represent a promising compromise, with greater homology to humans than smaller animals, and superior optical accessibility than mice. However, their size and physiology present a number of imaging challenges including non-negligible absorption and optical scattering, and limited time for image data acquisition if the fish are to be recovered for longitudinal studies. A key goal of this PhD thesis research was to develop OPT to address these challenges and improve in vivo imaging capabilities for this model organism. This thesis builds on previous work at Imperial where angularly multiplexed OPT using compressed sensing was developed and applied to in vivo imaging of a cancer-burdened adult zebrafish, with a sufficiently short OPT data acquisition time to allow recovery of the fish after anaesthesia. The previous cross-sectional study of this work was extended to a longitudinal study of cancer progression that I undertook. The volume and quality of data acquired in the longitudinal study presented a number of data processing challenges, which I addressed with improved automation of the data processing pipeline and with the demonstration that convolutional neural networks (CNN) could replace the iterative compressed sensing algorithm previously used to suppress artifacts when reconstructing undersampled OPT data sets. To address the issue of high attenuation through the centre of an adult zebrafish, I developed conformal-high-dynamic-range (C-HDR) OPT and demonstrated that it could provide sufficient dynamic range for brightfield imaging of such optically thick samples, noting that transmitted light images can provide anatomical context for fluorescence image data. To reduce the impact of optical scattering in OPT, I developed a parallelised quasi-confocal version of OPT called slice-illuminated OPT (slice-OPT) to reject scattered photons and demonstrated this with live zebrafish. To enable 3D imaging with short wave infrared (SWIR) light, without the requirement of an expensive Ge or InGaAs camera, I implemented a single pixel camera and demonstrated single-pixel OPT (SP-OPT) for the first time.Open Acces

Photonic Technology for Precision Metrology

Photonics has had a decisive influence on recent scientific and technological achievements. It includes aspects of photon generation and photon–matter interaction. Although it finds many applications in the whole optical range of the wavelengths, most solutions operate in the visible and infrared range. Since the invention of the laser, a source of highly coherent optical radiation, optical measurements have become the perfect tool for highly precise and accurate measurements. Such measurements have the additional advantages of requiring no contact and a fast rate suitable for in-process metrology. However, their extreme precision is ultimately limited by, e.g., the noise of both lasers and photodetectors. The Special Issue of the Applied Science is devoted to the cutting-edge uses of optical sources, detectors, and optoelectronics systems in numerous fields of science and technology (e.g., industry, environment, healthcare, telecommunication, security, and space). The aim is to provide detail on state-of-the-art photonic technology for precision metrology and identify future developmental directions. This issue focuses on metrology principles and measurement instrumentation in optical technology to solve challenging engineering problems

Tomographic measurement of all orthogonal components of three-dimensional displacement fields within scattering materials using wavelength scanning interferometry

Experimental mechanics is currently contemplating tremendous opportunities of further advancements thanks to a combination of powerful computational techniques and also fullfield non-contact methods to measure displacement and strain fields in a wide variety of materials. Identification techniques, aimed to evaluate material mechanical properties given known loads and measured displacement or strain fields, are bound to benefit from increased data availability (both in density and dimensionality) and efficient inversion methods such as finite element updating (FEU) and the virtual fields method (VFM). They work at their best when provided with dense and multicomponent experimental displacement (or strain) data, i.e. when all orthogonal components of displacements (or all components of the strain tensor) are known at points closely spaced within the volume of the material under study. Although a very challenging requirement, an increasing number of techniques are emerging to provide such data. In this Thesis, a novel wavelength scanning interferometry (WSI) system that provides three dimensional (3-D) displacement fields inside the volume of semi-transparent scattering materials is proposed. Sequences of two-dimensional interferograms are recorded whilst tuning the frequency of a laser at a constant rate. A new approach based on frequency multiplexing is used to encode the interference signal corresponding to multiple illumination directions at different spectral bands. Different optical paths along each illumination direction ensure that the signals corresponding to each sensitivity vector do not overlap in the frequency domain. All the information required to reconstruct the location and the 3-D displacement vector of scattering points within the material is thus recorded simultaneously in a single wavelength scan. By comparing phase data volumes obtained for two successive scans, all orthogonal components of the three dimensional displacement field introduced between scans (e.g. by means of loading or moving the sample under study) are readily obtained with high displacement sensitivity. The fundamental principle that describes the technique is presented in detail, including the correspondence between interference signal frequency and its associated depth within the sample, depth range, depth resolution, transverse resolution and displacement sensitivity. Data processing of the interference signal includes Fourier transformation, noise reduction, re-registration of data volumes, measurement of the illumination and sensitivity vectors from experimental data using a datum surface, phase difference evaluation, 3-D phase unwrapping and 3-D displacement field evaluation. Experiments consisting of controlled rigid body rotations and translations of a phantom were performed to validate the results. Both in-plane and the out-of-plane displacement components were measured for each voxel in the resulting data volume, showing an excellent agreement with the expected 3-D displacement

Imaging Cultural Heritage at Different Scales: Part I, the Micro-Scale (Manufacts)

Applications of non-invasive sensing techniques to investigate the internal structure and surface of precious and delicate objects represent a very important and consolidated research field in the scientific domain of cultural heritage knowledge and conservation. The present article is the first of three reviews focused on contact and non-contact imaging techniques applied to surveying cultural heritage at micro- (i.e., manufacts), meso- (sites) and macro-scales (landscapes). The capability to infer variations in geometrical and physical properties across the inspected surfaces or volumes is the unifying factor of these techniques, allowing scientists to discover new historical sites or to image their spatial extent and material features at different scales, from landscape to artifact. This first part concentrates on the micro-scale, i.e., inspection, study and characterization of small objects (ancient papers, paintings, statues, archaeological findings, architectural elements, etc.) from surface to internal properties

RGB-D And Thermal Sensor Fusion: A Systematic Literature Review

In the last decade, the computer vision field has seen significant progress in multimodal data fusion and learning, where multiple sensors, including depth, infrared, and visual, are used to capture the environment across diverse spectral ranges. Despite these advancements, there has been no systematic and comprehensive evaluation of fusing RGB-D and thermal modalities to date. While autonomous driving using LiDAR, radar, RGB, and other sensors has garnered substantial research interest, along with the fusion of RGB and depth modalities, the integration of thermal cameras and, specifically, the fusion of RGB-D and thermal data, has received comparatively less attention. This might be partly due to the limited number of publicly available datasets for such applications. This paper provides a comprehensive review of both, state-of-the-art and traditional methods used in fusing RGB-D and thermal camera data for various applications, such as site inspection, human tracking, fault detection, and others. The reviewed literature has been categorised into technical areas, such as 3D reconstruction, segmentation, object detection, available datasets, and other related topics. Following a brief introduction and an overview of the methodology, the study delves into calibration and registration techniques, then examines thermal visualisation and 3D reconstruction, before discussing the application of classic feature-based techniques as well as modern deep learning approaches. The paper concludes with a discourse on current limitations and potential future research directions. It is hoped that this survey will serve as a valuable reference for researchers looking to familiarise themselves with the latest advancements and contribute to the RGB-DT research field.Comment: 33 pages, 20 figure