Photometric stereo for strong specular highlights

Photometric stereo (PS) is a fundamental technique in computer vision known to produce 3-D shape with high accuracy. The setting of PS is defined by using several input images of a static scene taken from one and the same camera position but under varying illumination. The vast majority of studies in this 3-D reconstruction method assume orthographic projection for the camera model. In addition, they mainly consider the Lambertian reflectance model as the way that light scatters at surfaces. So, providing reliable PS results from real world objects still remains a challenging task. We address 3-D reconstruction by PS using a more realistic set of assumptions combining for the first time the complete Blinn-Phong reflectance model and perspective projection. To this end, we will compare two different methods of incorporating the perspective projection into our model. Experiments are performed on both synthetic and real world images. Note that our real-world experiments do not benefit from laboratory conditions. The results show the high potential of our method even for complex real world applications such as medical endoscopy images which may include high amounts of specular highlights

Flat-top TIRF illumination boosts DNA-PAINT imaging and quantification

Super-resolution (SR) techniques have extended the optical resolution down to a few nanometers. However, quantitative treatment of SR data remains challenging due to its complex dependence on a manifold of experimental parameters. Among the different SR variants, DNA-PAINT is relatively straightforward to implement, since it achieves the necessary 'blinking' without the use of rather complex optical or chemical activation schemes. However, it still suffers from image and quantification artifacts caused by inhomogeneous optical excitation. Here we demonstrate that several experimental challenges can be alleviated by introducing a segment-wise analysis approach and ultimately overcome by implementing a flat-top illumination profile for TIRF microscopy using a commercially-available beam-shaping device. The improvements with regards to homogeneous spatial resolution and precise kinetic information over the whole field-of-view were quantitatively assayed using DNA origami and cell samples. Our findings open the door to high-throughput DNA-PAINT studies with thus far unprecedented accuracy for quantitative data interpretation

State-of-the-art in studies of glacial isostatic adjustment for the British Isles: a literature review

Understanding the effects of glacial isostatic adjustment (GIA) of the British Isles is essential for the assessment of past and future sea-level trends. GIA has been extensively examined in the literature, employing different research methods and observational data types. Geological evidence from palaeo-shorelines and undisturbed sedimentary deposits has been used to reconstruct long-term relative sea-level change since the Last Glacial Maximum. This information derived from sea-level index points has been employed to inform empirical isobase models of the uplift in Scotland using trend surface and Gaussian trend surface analysis, as well as to calibrate more theory-driven GIA models that rely on Earth mantle rheology and ice sheet history. Furthermore, current short-term rates of GIA-induced crustal motion during the past few decades have been measured using different geodetic techniques, mainly continuous GPS (CGPS) and absolute gravimetry (AG). AG-measurements are generally employed to increase the accuracy of the CGPS estimates. Synthetic aperture radar interferometry (InSAR) looks promising as a relatively new technique to measure crustal uplift in the northern parts of Great Britain, where the GIA-induced vertical land deformation has its highest rate. This literature review provides an in-depth comparison and discussion of the development of these different research approaches

NOVEL DENSE STEREO ALGORITHMS FOR HIGH-QUALITY DEPTH ESTIMATION FROM IMAGES

This dissertation addresses the problem of inferring scene depth information from a collection of calibrated images taken from different viewpoints via stereo matching. Although it has been heavily investigated for decades, depth from stereo remains a long-standing challenge and popular research topic for several reasons. First of all, in order to be of practical use for many real-time applications such as autonomous driving, accurate depth estimation in real-time is of great importance and one of the core challenges in stereo. Second, for applications such as 3D reconstruction and view synthesis, high-quality depth estimation is crucial to achieve photo realistic results. However, due to the matching ambiguities, accurate dense depth estimates are difficult to achieve. Last but not least, most stereo algorithms rely on identification of corresponding points among images and only work effectively when scenes are Lambertian. For non-Lambertian surfaces, the brightness constancy assumption is no longer valid. This dissertation contributes three novel stereo algorithms that are motivated by the specific requirements and limitations imposed by different applications. In addressing high speed depth estimation from images, we present a stereo algorithm that achieves high quality results while maintaining real-time performance. We introduce an adaptive aggregation step in a dynamic-programming framework. Matching costs are aggregated in the vertical direction using a computationally expensive weighting scheme based on color and distance proximity. We utilize the vector processing capability and parallelism in commodity graphics hardware to speed up this process over two orders of magnitude. In addressing high accuracy depth estimation, we present a stereo model that makes use of constraints from points with known depths - the Ground Control Points (GCPs) as referred to in stereo literature. Our formulation explicitly models the influences of GCPs in a Markov Random Field. A novel regularization prior is naturally integrated into a global inference framework in a principled way using the Bayes rule. Our probabilistic framework allows GCPs to be obtained from various modalities and provides a natural way to integrate information from various sensors. In addressing non-Lambertian reflectance, we introduce a new invariant for stereo correspondence which allows completely arbitrary scene reflectance (bidirectional reflectance distribution functions - BRDFs). This invariant can be used to formulate a rank constraint on stereo matching when the scene is observed by several lighting configurations in which only the lighting intensity varies

The First Public Release of South Pole Telescope Data: Maps of a 95 deg^2 Field from 2008 Observations

The South Pole Telescope (SPT) has nearly completed a 2500 deg^2 survey of the southern sky in three frequency bands. Here, we present the first public release of SPT maps and associated data products. We present arcminute-resolution maps at 150 GHz and 220 GHz of an approximately 95 deg^2 field centered at R.A. 82°.7, decl. –55°. The field was observed to a depth of approximately 17 μK arcmin at 150 GHz and 41 μK arcmin at 220 GHz during the 2008 austral winter season. Two variations on map filtering and map projection are presented, one tailored for producing catalogs of galaxy clusters detected through their Sunyaev-Zel'dovich effect signature and one tailored for producing catalogs of emissive sources. We describe the data processing pipeline, and we present instrument response functions, filter transfer functions, and map noise properties. All data products described in this paper are available for download at http://pole.uchicago.edu/public/data/maps/ra5h30dec-55 and from the NASA Legacy Archive for Microwave Background Data Analysis server. This is the first step in the eventual release of data from the full 2500 deg^2 SPT survey

MP-MVS: Multi-Scale Windows PatchMatch and Planar Prior Multi-View Stereo

Significant strides have been made in enhancing the accuracy of Multi-View Stereo (MVS)-based 3D reconstruction. However, untextured areas with unstable photometric consistency often remain incompletely reconstructed. In this paper, we propose a resilient and effective multi-view stereo approach (MP-MVS). We design a multi-scale windows PatchMatch (mPM) to obtain reliable depth of untextured areas. In contrast with other multi-scale approaches, which is faster and can be easily extended to PatchMatch-based MVS approaches. Subsequently, we improve the existing checkerboard sampling schemes by limiting our sampling to distant regions, which can effectively improve the efficiency of spatial propagation while mitigating outlier generation. Finally, we introduce and improve planar prior assisted PatchMatch of ACMP. Instead of relying on photometric consistency, we utilize geometric consistency information between multi-views to select reliable triangulated vertices. This strategy can obtain a more accurate planar prior model to rectify photometric consistency measurements. Our approach has been tested on the ETH3D High-res multi-view benchmark with several state-of-the-art approaches. The results demonstrate that our approach can reach the state-of-the-art. The associated codes will be accessible at https://github.com/RongxuanTan/MP-MVS

Phase Aberration Correction: A Deep Learning-Based Aberration to Aberration Approach

One of the primary sources of suboptimal image quality in ultrasound imaging is phase aberration. It is caused by spatial changes in sound speed over a heterogeneous medium, which disturbs the transmitted waves and prevents coherent summation of echo signals. Obtaining non-aberrated ground truths in real-world scenarios can be extremely challenging, if not impossible. This challenge hinders training of deep learning-based techniques' performance due to the presence of domain shift between simulated and experimental data. Here, for the first time, we propose a deep learning-based method that does not require ground truth to correct the phase aberration problem, and as such, can be directly trained on real data. We train a network wherein both the input and target output are randomly aberrated radio frequency (RF) data. Moreover, we demonstrate that a conventional loss function such as mean square error is inadequate for training such a network to achieve optimal performance. Instead, we propose an adaptive mixed loss function that employs both B-mode and RF data, resulting in more efficient convergence and enhanced performance. Finally, we publicly release our dataset, including 161,701 single plane-wave images (RF data). This dataset serves to mitigate the data scarcity problem in the development of deep learning-based techniques for phase aberration correction.Comment: arXiv admin note: text overlap with arXiv:2303.0574

Computational Multispectral Endoscopy

Minimal Access Surgery (MAS) is increasingly regarded as the de-facto approach in interventional medicine for conducting many procedures this is due to the reduced patient trauma and consequently reduced recovery times, complications and costs. However, there are many challenges in MAS that come as a result of viewing the surgical site through an endoscope and interacting with tissue remotely via tools, such as lack of haptic feedback; limited field of view; and variation in imaging hardware. As such, it is important best utilise the imaging data available to provide a clinician with rich data corresponding to the surgical site. Measuring tissue haemoglobin concentrations can give vital information, such as perfusion assessment after transplantation; visualisation of the health of blood supply to organ; and to detect ischaemia. In the area of transplant and bypass procedures measurements of the tissue tissue perfusion/total haemoglobin (THb) and oxygen saturation (SO2) are used as indicators of organ viability, these measurements are often acquired at multiple discrete points across the tissue using with a specialist probe. To acquire measurements across the whole surface of an organ one can use a specialist camera to perform multispectral imaging (MSI), which optically acquires sequential spectrally band limited images of the same scene. This data can be processed to provide maps of the THb and SO2 variation across the tissue surface which could be useful for intra operative evaluation. When capturing MSI data, a trade off often has to be made between spectral sensitivity and capture speed. The work in thesis first explores post processing blurry MSI data from long exposure imaging devices. It is of interest to be able to use these MSI data because the large number of spectral bands that can be captured, the long capture times, however, limit the potential real time uses for clinicians. Recognising the importance to clinicians of real-time data, the main body of this thesis develops methods around estimating oxy- and deoxy-haemoglobin concentrations in tissue using only monocular and stereo RGB imaging data