Physics vs. Learned Priors: Rethinking Camera and Algorithm Design for Task-Specific Imaging

Cameras were originally designed using physics-based heuristics to capture aesthetic images. In recent years, there has been a transformation in camera design from being purely physics-driven to increasingly data-driven and task-specific. In this paper, we present a framework to understand the building blocks of this nascent field of end-to-end design of camera hardware and algorithms. As part of this framework, we show how methods that exploit both physics and data have become prevalent in imaging and computer vision, underscoring a key trend that will continue to dominate the future of task-specific camera design. Finally, we share current barriers to progress in end-to-end design, and hypothesize how these barriers can be overcome

The Structure and Dynamics of the Upper Chromosphere and Lower Transition Region as Revealed by the Subarcsecond VAULT Observations

The Very high Angular resolution ULtraviolet Telescope (VAULT) is a sounding rocket payload built to study the crucial interface between the solar chromosphere and the corona by observing the strongest line in the solar spectrum, the Ly-a line at 1216 {\AA}. In two flights, VAULT succeeded in obtaining the first ever sub-arcsecond (0.5") images of this region with high sensitivity and cadence. Detailed analyses of those observations have contributed significantly to new ideas about the nature of the transition region. Here, we present a broad overview of the Ly-a atmosphere as revealed by the VAULT observations, and bring together past results and new analyses from the second VAULT flight to create a synthesis of our current knowledge of the high-resolution Ly-a Sun. We hope that this work will serve as a good reference for the design of upcoming Ly-a telescopes and observing plans.Comment: 28 pages, 11 figure

Making Thermal Imaging More Equitable and Accurate: Resolving Solar Loading Biases

Thermal cameras and thermal point detectors are used to measure the temperature of human skin. These are important devices that are used everyday in clinical and mass screening settings, particularly in an epidemic. Unfortunately, despite the wide use of thermal sensors, the temperature estimates from thermal sensors do not work well in uncontrolled scene conditions. Previous work has studied the effect of wind and other environment factors on skin temperature, but has not considered the heating effect from sunlight, which is termed solar loading. Existing device manufacturers recommend that a subject who has been outdoors in sun re-acclimate to an indoor environment after a waiting period. The waiting period, up to 30 minutes, is insufficient for a rapid screening tool. Moreover, the error bias from solar loading is greater for darker skin tones since melanin absorbs solar radiation. This paper explores two approaches to address this problem. The first approach uses transient behavior of cooling to more quickly extrapolate the steady state temperature. A second approach explores the spatial modulation of solar loading, to propose single-shot correction with a wide-field thermal camera. A real world dataset comprising of thermal point, thermal image, subjective, and objective measurements of melanin is collected with statistical significance for the effect size observed. The single-shot correction scheme is shown to eliminate solar loading bias in the time of a typical frame exposure (33ms)

Passive Micron-scale Time-of-Flight with Sunlight Interferometry

We introduce an interferometric technique for passive time-of-flight imaging and depth sensing at micrometer axial resolutions. Our technique uses a full-field Michelson interferometer, modified to use sunlight as the only light source. The large spectral bandwidth of sunlight makes it possible to acquire micrometer-resolution time-resolved scene responses, through a simple axial scanning operation. Additionally, the angular bandwidth of sunlight makes it possible to capture time-of-flight measurements insensitive to indirect illumination effects, such as interreflections and subsurface scattering. We build an experimental prototype that we operate outdoors, under direct sunlight, and in adverse environmental conditions such as mechanical vibrations and vehicle traffic. We use this prototype to demonstrate, for the first time, passive imaging capabilities such as micrometer-scale depth sensing robust to indirect illumination, direct-only imaging, and imaging through diffusers

Measuring and simulating haemodynamics due to geometric changes in facial expression

The human brain has evolved to be very adept at recognising imperfections in human skin. In particular, observing someone’s facial skin appearance is important in recognising when someone is ill, or when finding a suitable mate. It is therefore a key goal of computer graphics research to produce highly realistic renderings of skin. However, the optical processes that give rise to skin appearance are complex and subtle. To address this, computer graphics research has incorporated more and more sophisticated models of skin reflectance. These models are generally based on static concentrations of skin chromophores; melanin and haemoglobin. However, haemoglobin concentrations are far from static, as blood flow is directly caused by both changes in facial expression and emotional state. In this thesis, we explore how blood flow changes as a consequence of changing facial expression with the aim of producing more accurate models of skin appearance. To build an accurate model of blood flow, we base it on real-world measurements of blood concentrations over time. We describe, in detail, the steps required to obtain blood concentrations from photographs of a subject. These steps are then used to measure blood concentration maps for a series of expressions that define a wide gamut of human expression. From this, we define a blending algorithm that allows us to interpolate these maps to generate concentrations for other expressions. This technique, however, requires specialist equipment to capture the maps in the first place. We try to rectify this problem by investigating a direct link between changes in facial geometry and haemoglobin concentrations. This requires building a unique capture device that captures both simultaneously. Our analysis hints a direct linear connection between the two, paving the way for further investigatio

Quasi-periodic pulsations in solar flares

For several decades, quasi-periodic pulsations (QPP) in flares have been a signature feature of solar dynamics. In the last fifteen years, the advent of new observational instruments has led to a much-improved scope for studying and understanding such phenomena. These events are particularly relevant to the field of coronal seismology, where impulsive events are used as diagnostic tools to estimate the physical parameters of the solar atmosphere remotely. In this thesis we investigate quasi-periodic pulsations in flares from both a numerical and observational perpective, mostly in terms of magnetohydrodynamic (MHD) waves. It has long been suggested that MHD modes may be the cause of QPP in flares as they are capable of modulating a wide range of observable quantities. We study one such mode in detail: the sausage mode. For a model including significant gas pressure, the characteristic period, the ratio of the mode harmonics and the behaviour of the wavenumber cutoff are all considered. Although the period and wavenumber are only marginally affected by this gas pressure, the density contrast ratio and length are important factors. An observational study of a flaring QPP event was undertaken, where new techniques were developed in an attempt to successfully diagnose the QPP mechanism. Cross-correlation mapping was applied to spatially resolved radio data, showing how the strength and phase relationship of a flaring oscillation can be mapped in space. Using this information, we were able to exclude many mechanisms as possible drivers for this event, and suggest that an MHD sausage mode is the likely candidate. A second flaring QPP event was considered, based on the possibility of multiple harmonic oscillations. A sequential spectral peak filtering method was used to demonstrate the presence of multiple significant periods in the flare. Analysis of the harmonic ratios indicated that an MHD wave such as a kink mode was the probable cause. Finally we explore the potential of a new technique in the context of the solar corona, the combination of empirical mode decomposition (EMD) and the Hilbert spectrum. It was established that, under certain circumstances, this method compared favourably with existing analysis techniques such as the Morlet wavelet, and may lead to significant future observational results