Plenoptic Signal Processing for Robust Vision in Field Robotics

This thesis proposes the use of plenoptic cameras for improving the robustness and simplicity of machine vision in field robotics applications. Dust, rain, fog, snow, murky water and insufficient light can cause even the most sophisticated vision systems to fail. Plenoptic cameras offer an appealing alternative to conventional imagery by gathering significantly more light over a wider depth of field, and capturing a rich 4D light field structure that encodes textural and geometric information. The key contributions of this work lie in exploring the properties of plenoptic signals and developing algorithms for exploiting them. It lays the groundwork for the deployment of plenoptic cameras in field robotics by establishing a decoding, calibration and rectification scheme appropriate to compact, lenslet-based devices. Next, the frequency-domain shape of plenoptic signals is elaborated and exploited by constructing a filter which focuses over a wide depth of field rather than at a single depth. This filter is shown to reject noise, improving contrast in low light and through attenuating media, while mitigating occluders such as snow, rain and underwater particulate matter. Next, a closed-form generalization of optical flow is presented which directly estimates camera motion from first-order derivatives. An elegant adaptation of this "plenoptic flow" to lenslet-based imagery is demonstrated, as well as a simple, additive method for rendering novel views. Finally, the isolation of dynamic elements from a static background is considered, a task complicated by the non-uniform apparent motion caused by a mobile camera. Two elegant closed-form solutions are presented dealing with monocular time-series and light field image pairs. This work emphasizes non-iterative, noise-tolerant, closed-form, linear methods with predictable and constant runtimes, making them suitable for real-time embedded implementation in field robotics applications

Plenoptic Signal Processing for Robust Vision in Field Robotics

This thesis proposes the use of plenoptic cameras for improving the robustness and simplicity of machine vision in field robotics applications. Dust, rain, fog, snow, murky water and insufficient light can cause even the most sophisticated vision systems to fail. Plenoptic cameras offer an appealing alternative to conventional imagery by gathering significantly more light over a wider depth of field, and capturing a rich 4D light field structure that encodes textural and geometric information. The key contributions of this work lie in exploring the properties of plenoptic signals and developing algorithms for exploiting them. It lays the groundwork for the deployment of plenoptic cameras in field robotics by establishing a decoding, calibration and rectification scheme appropriate to compact, lenslet-based devices. Next, the frequency-domain shape of plenoptic signals is elaborated and exploited by constructing a filter which focuses over a wide depth of field rather than at a single depth. This filter is shown to reject noise, improving contrast in low light and through attenuating media, while mitigating occluders such as snow, rain and underwater particulate matter. Next, a closed-form generalization of optical flow is presented which directly estimates camera motion from first-order derivatives. An elegant adaptation of this "plenoptic flow" to lenslet-based imagery is demonstrated, as well as a simple, additive method for rendering novel views. Finally, the isolation of dynamic elements from a static background is considered, a task complicated by the non-uniform apparent motion caused by a mobile camera. Two elegant closed-form solutions are presented dealing with monocular time-series and light field image pairs. This work emphasizes non-iterative, noise-tolerant, closed-form, linear methods with predictable and constant runtimes, making them suitable for real-time embedded implementation in field robotics applications

Supernova / Acceleration Probe: A Satellite Experiment to Study the Nature of the Dark Energy

The Supernova / Acceleration Probe (SNAP) is a proposed space-based experiment designed to study the dark energy and alternative explanations of the acceleration of the Universe's expansion by performing a series of complementary systematics-controlled measurements. We describe a self-consistent reference mission design for building a Type Ia supernova Hubble diagram and for performing a wide-area weak gravitational lensing study. A 2-m wide-field telescope feeds a focal plane consisting of a 0.7 square-degree imager tiled with equal areas of optical CCDs and near infrared sensors, and a high-efficiency low-resolution integral field spectrograph. The SNAP mission will obtain high-signal-to-noise calibrated light-curves and spectra for several thousand supernovae at redshifts between z=0.1 and 1.7. A wide-field survey covering one thousand square degrees resolves ~100 galaxies per square arcminute. If we assume we live in a cosmological-constant-dominated Universe, the matter density, dark energy density, and flatness of space can all be measured with SNAP supernova and weak-lensing measurements to a systematics-limited accuracy of 1%. For a flat universe, the density-to-pressure ratio of dark energy can be similarly measured to 5% for the present value w0 and ~0.1 for the time variation w'. The large survey area, depth, spatial resolution, time-sampling, and nine-band optical to NIR photometry will support additional independent and/or complementary dark-energy measurement approaches as well as a broad range of auxiliary science programs. (Abridged)Comment: 40 pages, 18 figures, submitted to PASP, http://snap.lbl.go

Camera positioning for 3D panoramic image rendering

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University London.Virtual camera realisation and the proposition of trapezoidal camera architecture are the two broad contributions of this thesis. Firstly, multiple camera and their arrangement constitute a critical component which affect the integrity of visual content acquisition for multi-view video. Currently, linear, convergence, and divergence arrays are the prominent camera topologies adopted. However, the large number of cameras required and their synchronisation are two of prominent challenges usually encountered. The use of virtual cameras can significantly reduce the number of physical cameras used with respect to any of the known camera structures, hence adequately reducing some of the other implementation issues. This thesis explores to use image-based rendering with and without geometry in the implementations leading to the realisation of virtual cameras. The virtual camera implementation was carried out from the perspective of depth map (geometry) and use of multiple image samples (no geometry). Prior to the virtual camera realisation, the generation of depth map was investigated using region match measures widely known for solving image point correspondence problem. The constructed depth maps have been compare with the ones generated using the dynamic programming approach. In both the geometry and no geometry approaches, the virtual cameras lead to the rendering of views from a textured depth map, construction of 3D panoramic image of a scene by stitching multiple image samples and performing superposition on them, and computation of virtual scene from a stereo pair of panoramic images. The quality of these rendered images were assessed through the use of either objective or subjective analysis in Imatest software. Further more, metric reconstruction of a scene was performed by re-projection of the pixel points from multiple image samples with a single centre of projection. This was done using sparse bundle adjustment algorithm. The statistical summary obtained after the application of this algorithm provides a gauge for the efficiency of the optimisation step. The optimised data was then visualised in Meshlab software environment, hence providing the reconstructed scene. Secondly, with any of the well-established camera arrangements, all cameras are usually constrained to the same horizontal plane. Therefore, occlusion becomes an extremely challenging problem, and a robust camera set-up is required in order to resolve strongly the hidden part of any scene objects. To adequately meet the visibility condition for scene objects and given that occlusion of the same scene objects can occur, a multi-plane camera structure is highly desirable. Therefore, this thesis also explore trapezoidal camera structure for image acquisition. The approach here is to assess the feasibility and potential of several physical cameras of the same model being sparsely arranged on the edge of an efficient trapezoid graph. This is implemented both Matlab and Maya. The quality of the depth maps rendered in Matlab are better in Quality

The 2-degree Field Lensing Survey: design and clustering measurements

We present the 2-degree Field Lensing Survey (2dFLenS), a new galaxy redshift survey performed at the Anglo-Australian Telescope. 2dFLenS is the first wide-area spectroscopic survey specifically targeting the area mapped by deep-imaging gravitational lensing fields, in this case the Kilo-Degree Survey. 2dFLenS obtained 70 079 redshifts in the range z < 0.9 over an area of 731 deg2, and is designed to extend the data sets available for testing gravitational physics and promote the development of relevant algorithms for joint imaging and spectroscopic analysis. The redshift sample consists first of 40 531 Luminous Red Galaxies (LRGs), which enable analyses of galaxy–galaxy lensing, redshift-space distortion, and the overlapping source redshift distribution by cross-correlation. An additional 28 269 redshifts form a magnitude-limited (r < 19.5) nearly complete subsample, allowing direct source classification and photometric-redshift calibration. In this paper, we describe the motivation, target selection, spectroscopic observations, and clustering analysis of 2dFLenS. We use power spectrum multipole measurements to fit the redshift-space distortion parameter of the LRG sample in two redshift ranges 0.15 < z < 0.43 and 0.43 < z < 0.7 as β = 0.49 ± 0.15 and β = 0.26 ± 0.09, respectively. These values are consistent with those obtained from LRGs in the Baryon Oscillation Spectroscopic Survey. 2dFLenS data products will be released via our website http://2dflens.swin.edu.au

The Astrophysics Science Division Annual Report 2009

The Astrophysics Science Division (ASD) at Goddard Space Flight Center (GSFC) is one of the largest and most diverse astrophysical organizations in the world, with activities spanning a broad range of topics in theory, observation, and mission and technology development. Scientific research is carried out over the entire electromagnetic spectrum - from gamma rays to radio wavelengths - as well as particle physics and gravitational radiation. Members of ASD also provide the scientific operations for three orbiting astrophysics missions - WMAP, RXTE, and Swift, as well as the Science Support Center for the Fermi Gamma-ray Space Telescope. A number of key technologies for future missions are also under development in the Division, including X-ray mirrors, space-based interferometry, high contrast imaging techniques to search for exoplanets, and new detectors operating at gamma-ray, X-ray, ultraviolet, infrared, and radio wavelengths. The overriding goals of ASD are to carry out cutting-edge scientific research, provide Project Scientist support for spaceflight missions, implement the goals of the NASA Strategic Plan, serve and support the astronomical community, and enable future missions by conceiving new concepts and inventing new technologies

Assessing the performance of Digital Micromirror Devices for use in space-based multi-object spectrometers

A current need in space-based instrumentation is a reconfigurable slit mask. Several techniques for slit masks have been employed for ground-based astronomical spectrographs. These ground-based instruments have used large discrete components, which are impractical for remote operation in space-based deployment. The Texas Instruments\u27 Digital Micromirror Device (DMD) was originally conceived purely for display purposes, but is a viable candidate to be use as a slit mask in a space-based multi-object spectrograph (MOS). The Integrated Circuit (IC) manufacturing industry has enabled the robust integration of both silicon transistors and Micro-Electrical Mechanical Systems (MEMS) optical components into a very reliable monolithic chip (the DMD). The focus of this work was in three areas that addressed the suitability of proposing DMDs for future space missions. The DMDs were optically characterized to assess their utility in a spectrograph. The DMDs were also cooled in a liquid nitrogen dewar to determine their minimum operating temperature. The low temperature tests indicated that the DMD can operate to temperatures as low as 130 K. In addition, several DMDs were irradiated with high-energy protons at the LBNL 88 Cyclotron to determine how robust the devices are to ionizing radiation (protons). The radiation testing results indicate that DMDs would survive medium to long duration space missions with full operability. Based on preliminary tests in these three areas, the DMD should be considered as an excellent candidate for deployment in future space missions

The Multiplexed Squid Tes Array at Ninety Gigahertz (Mustang)

The Multiplexed SQUID/TES Array at Ninety Gigahertz (MUSTANG) is a bolometric continuum imaging camera designed to operate at the Gregorian focus of the 100m Green Bank Telescope (GBT) in Pocahontas county, West Virginia. The combination of the GBT\u27s large collecting area and the 8x8 array of transition edge sensors at the heart of MUSTANG allows for deep imaging at 10\u27\u27 resolution at 90GHz. The MUSTANG receiver is now a facility instrument of the National Radio Astronomy Observatory available to the general astronomical community. The 3.3mm continuum passband is useful to access a large range of Galactic and extra-Galactic astrophysics. Sources with synchrotron, free-free and thermal blackbody emission can be detected at 3.3mm. Of particular interest is the Sunyaev Zel\u27dovich effect in clusters of galaxies, which arises from the inverse Compton scattering of CMB photons off hot electrons in the intra-cluster medium. In the MUSTANG band, the effect is observationally manifested as an artificial decrement in power on the sky in the direction of the cluster. There have been many experiments in the past two decades dedicated to measurements of the SZE, however, nearly all of them were accomplished with angular resolution larger than ~1\u27. The massive primary of the GBT enables measurements of the SZE on 10\u27\u27 scales. This provides a new technique to map pressure substructure in the plasma atmospheres of merging clusters of galaxies. By analyzing MUSTANG data along side X-ray data, spatially resolved measurements of the temperature, density and pressure of the ICM can be performed which can be used to infer the physics governing major mergers. This thesis details the design, commissioning and operation of the various components which comprise the MUSTANG receiver. This includes the sub-kelvin cryogenic cooling, the time domain multiplexed readout electronics and the array of transition edge sensor bolometers. Laboratory characterization of the detector array is thoroughly described, including the measurements of the transition temperatures, thermal conductance, noise properties and time constants. Bolometric cameras measure celestial radiation as time ordered data. To reliably produce images from traces in the time domain, care must be taken to modulate the signal in an optimized fashion during observations. Once the data are taken, algorithms must be developed to isolate the signal of interest from the foregrounds produced by atmospheric emission. This thesis presents the techniques developed to scan, calibrate, filter and produce images from time ordered data taken with MUSTANG and the GBT. From the Fall of 2006 to the winter of 2010, the MUSTANG receiver was commissioned on the GBT and used for a range of astrophysical measurements. The commissioning process and early science results are given in this thesis. This includes the modifications made to the instrument which have resulted in dramatically enhanced sensitivity as well as the images produced from bright extended millimeter sources such as high mass star forming regions, active galactic nuclei and supernova remnants. This thesis presents a sample of merging clusters of galaxies imaged through their Sunyaev Zel\u27dovich signatures at high angular resolution. In the massive cluster RXJ1347, a previously reported pressure enhancement to the south east of the cluster peak was confirmed. This is now interpreted as a parcel of hot shock heated gas (KT~20keV) produced in a recent merger. In the high redshift systems MACS0744 and CL1226, pressure substructure was identified and is believed to be associated with merger activity. Both systems contain peaks in dark matter revealed by gravitational lensing which are not associated with baryonic emission, supportive of a scenario in which an infalling cluster has passed through a main cluster being stripped of its baryons. In MACS0744, the SZE and X-ray morphology is suggestive of a shock wave propagating through the ICM. By fitting the Rankine Hugoniot jump conditions in a simultaneous SZE/X-ray analysis, the likelihood of this interpretation is explored. The system is well described by a mildly supersonic shock wave propagating with a Mach number of ~1.2