Foundations, Inference, and Deconvolution in Image Restoration

Image restoration is a critical preprocessing step in computer vision, producing images with reduced noise, blur, and pixel defects. This enables precise higher-level reasoning as to the scene content in later stages of the vision pipeline (e.g., object segmentation, detection, recognition, and tracking). Restoration techniques have found extensive usage in a broad range of applications from industry, medicine, astronomy, biology, and photography. The recovery of high-grade results requires models of the image degradation process, giving rise to a class of often heavily underconstrained, inverse problems. A further challenge specific to the problem of blur removal is noise amplification, which may cause strong distortion by ringing artifacts. This dissertation presents new insights and problem solving procedures for three areas of image restoration, namely (1) model foundations, (2) Bayesian inference for high-order Markov random fields (MRFs), and (3) blind image deblurring (deconvolution). As basic research on model foundations, we contribute to reconciling the perceived differences between probabilistic MRFs on the one hand, and deterministic variational models on the other. To do so, we restrict the variational functional to locally supported finite elements (FE) and integrate over the domain. This yields a sum of terms depending locally on FE basis coefficients, and by identifying the latter with pixels, the terms resolve to MRF potential functions. In contrast with previous literature, we place special emphasis on robust regularizers used commonly in contemporary computer vision. Moreover, we draw samples from the derived models to further demonstrate the probabilistic connection. Another focal issue is a class of high-order Field of Experts MRFs which are learned generatively from natural image data and yield best quantitative results under Bayesian estimation. This involves minimizing an integral expression, which has no closed form solution in general. However, the MRF class under study has Gaussian mixture potentials, permitting expansion by indicator variables as a technical measure. As approximate inference method, we study Gibbs sampling in the context of non-blind deblurring and obtain excellent results, yet at the cost of high computing effort. In reaction to this, we turn to the mean field algorithm, and show that it scales quadratically in the clique size for a standard restoration setting with linear degradation model. An empirical study of mean field over several restoration scenarios confirms advantageous properties with regard to both image quality and computational runtime. This dissertation further examines the problem of blind deconvolution, beginning with localized blur from fast moving objects in the scene, or from camera defocus. Forgoing dedicated hardware or user labels, we rely only on the image as input and introduce a latent variable model to explain the non-uniform blur. The inference procedure estimates freely varying kernels and we demonstrate its generality by extensive experiments. We further present a discriminative method for blind removal of camera shake. In particular, we interleave discriminative non-blind deconvolution steps with kernel estimation and leverage the error cancellation effects of the Regression Tree Field model to attain a deblurring process with tightly linked sequential stages

A Low-cost Depth Imaging Mobile Platform for Canola Phenotyping

To meet the high demand for supporting and accelerating progress in the breeding of novel traits, plant scientists and breeders have to measure a large number of plants and their characteristics accurately. A variety of imaging methodologies are being deployed to acquire data for quantitative studies of complex traits. When applied to a large number of plants such as canola plants, however, a complete three-dimensional (3D) model is time-consuming and expensive for high-throughput phenotyping with an enormous amount of data. In some contexts, a full rebuild of entire plants may not be necessary. In recent years, many 3D plan phenotyping techniques with high cost and large-scale facilities have been introduced to extract plant phenotypic traits, but these applications may be affected by limited research budgets and cross environments. This thesis proposed a low-cost depth and high-throughput phenotyping mobile platform to measure canola plant traits in cross environments. Methods included detecting and counting canola branches and seedpods, monitoring canola growth stages, and fusing color images to improve images resolution and achieve higher accuracy. Canola plant traits were examined in both controlled environment and field scenarios. These methodologies were enhanced by different imaging techniques. Results revealed that this phenotyping mobile platform can be used to investigate canola plant traits in cross environments with high accuracy. The results also show that algorithms for counting canola branches and seedpods enable crop researchers to analyze the relationship between canola genotypes and phenotypes and estimate crop yields. In addition to counting algorithms, fusing techniques can be helpful for plant breeders with more comfortable access plant characteristics by improving the definition and resolution of color images. These findings add value to the automation, low-cost depth and high-throughput phenotyping for canola plants. These findings also contribute a novel multi-focus image fusion that exhibits a competitive performance with outperforms some other state-of-the-art methods based on the visual saliency maps and gradient domain fast guided filter. This proposed platform and counting algorithms can be applied to not only canola plants but also other closely related species. The proposed fusing technique can be extended to other fields, such as remote sensing and medical image fusion

Multiplexed photography : single-exposure capture of multiple camera settings

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2009.Cataloged from PDF version of thesis.Includes bibliographical references (p. 115-124).The space of camera settings is large and individual settings can vary dramatically from scene to scene. This thesis explores methods for capturing and manipulating multiple camera settings in a single exposure. Multiplexing multiple camera settings in a single exposure can allow post-exposure control and improve the quality of photographs taken in challenging lighting environments (e.g. low light or high motion). We first describe the design and implementation of a prototype optical system and associated algorithms to capture four images of a scene in a single exposure, each taken with a different aperture setting. Our system can be used with commercially available DSLR cameras and photographic lenses without modification to either. We demonstrate several applications of our multi-aperture camera, such as post-exposure depth of field control, synthetic refocusing, and depth-guided deconvolution. Next we describe multiplexed flash illumination to recover both flash and ambient light information as well as extract depth information in a single exposure. Traditional photographic flashes illuminate the scene with a spatially-constant light beam. By adding a mask and optics to a flash, we can project a spatially varying illumination onto the scene which allows us to spatially multiplex the flash and ambient illuminations onto the imager. We apply flash multiplexing to enable single exposure flash/no-flash image fusion, in particular, performing flash/no-flash relighting on dynamic scenes with moving objects. Finally, we propose spatio-temporal multiplexing, a novel image sensor feature that enables simultaneous capture of flash and ambient illumination.(cont.) We describe two possible applications of spatio-temporal multiplexing: single-image flash/no-flash relighting and white balancing scenes containing two distinct illuminants (e.g. flash and fluorescent lighting).by Paul Elijah Green.Ph.D

Inferring Human Pose and Motion from Images

As optical gesture recognition technology advances, touchless human computer interfaces of the future will soon become a reality. One particular technology, markerless motion capture, has gained a large amount of attention, with widespread application in diverse disciplines, including medical science, sports analysis, advanced user interfaces, and virtual arts. However, the complexity of human anatomy makes markerless motion capture a non-trivial problem: I) parameterised pose configuration exhibits high dimensionality, and II) there is considerable ambiguity in surjective inverse mapping from observation to pose configuration spaces with a limited number of camera views. These factors together lead to multimodality in high dimensional space, making markerless motion capture an ill-posed problem. This study challenges these difficulties by introducing a new framework. It begins with automatically modelling specific subject template models and calibrating posture at the initial stage. Subsequent tracking is accomplished by embedding naturally-inspired global optimisation into the sequential Bayesian filtering framework. Tracking is enhanced by several robust evaluation improvements. Sparsity of images is managed by compressive evaluation, further accelerating computational efficiency in high dimensional space

Alpha Matting of Motion-Blurred Objects in Bracket Sequence Images

Abstract. We present a method that utilizes bracket sequence images to automatically extract the alpha matte of a motion-blurred object. This method makes use of a sharp, short-exposure snapshot in the sequence to help overcome major challenges in this task, including blurred object detection, spatially-variant object motion, and foreground/background color ambiguity. A key component of our matte estimation is the infer-ence of approximate, spatially-varying motion of the blurred object with the help of the sharp snapshot, as this motion information provides im-portant constraints on the aforementioned issues. In addition, we take advantage of other relationships that exist between a pair of consecutive short-exposure and long-exposure frames, such as common background areas and consistencies in foreground appearance. With this technique, we demonstrate successful alpha matting results on a variety of moving objects including non-rigid human motion

Ambivalent animal

The Ambivalent Animal project explores the interactions of animals, culture and technology. The project employs both artistic practice and critical theory, each in ways that inspire the other. My creative practice centers around two projects that focus on domestic pets. These projects highlight the animal's uncertain status as they explore the overlapping ontologies of animal, human and machine. They provide concrete artifacts that engage with theoretical issues of anthropocentrism, animality and alterity. My theoretical work navigates between the fields of animal studies, art and design, media and culture studies, and philosophy. My dissertation explores animality through four real and imagined animal roles: cyborg, clone, chimera and shapeshifter. Each animal role is considered in relation to three dialectics: irreducibility and procedurality, autonomy and integration, aura and abjection. These dialectics do not seek full synthesis but instead embrace the oscillations of irresolvable debates and desires. The dialectics bring into focus issues of epistemology, ontology, corporeality and subjectivity. When the four animal roles engage the three dialectics, connected yet varied themes emerge. The cyborgian animal is simultaneously liberated and regulated, assisted and restricted, integrated and isolated. The cloned animal is an emblem of renewal and loss; she is both idealized code and material flesh and finds herself caught in the battles of nature and nurture. The chimera is both rebel and conformist; his unusual juxtapositions pioneer radical corporeal transgressions but also conform to the mechanisms of global capital. And the shapeshifter explores the thrill and anxiety of an altered phenomenology; she gains new perceptions though unstable subjectivity. These roles reveal corporeal adjustments and unfamiliar subjectivities that inspire the creative practice. Both my writing and making employ an ambivalent aesthetic--an aesthetic approach that evokes two or more incompatible sensibilities. The animal's uncertain status contributes to this aesthetic: some animals enjoy remarkable care and attention, while others are routinely exploited, abused and discarded. Ambivalence acknowledges the complexity of lived experience, philosophical and political debate, and academic inquiry. My approach recognizes the light and dark of these complex ambivalences--it privileges paradox and embraces the confusion and wonder of creative research. Rather than erase, conceal or resolve ambiguity, an ambivalent aesthetic foregrounds the limits of language and representation and highlights contradiction and irresolution.Ph.D.Committee Chair: Bolter, Jay; Committee Member: DiSalvo, Carl; Committee Member: Do, Ellen; Committee Member: Prophet, Jane; Committee Member: Thacker, Eugen

Passions, Pedagogies, and 21st Century Technologies

Once again, Gail Hawisher and Cynthia Selfe offer a volume that will set the agenda in the field of computers and composition scholarship for a decade. The technology changes that scholars of composition studies face as the next century opens couldn\u27t be more dramatic or deserving of passionate study. While we have always used technologies (e.g., the pencil) to communicate with each other, the electronic technologies we now use have changed the world in ways that we have yet to identify or appreciate fully. Likewise, the study of language and literate exchange, even our understanding of terms like literacy, text, and visual, has changed beyond recognition, challenging even our capacity to articulate them.https://digitalcommons.usu.edu/usupress_pubs/1118/thumbnail.jp

Astronautics and Aeronautics, 1979-1984: A chronology

This volume of the Astronautics and Aeronautics series covers 1979 through 1984. The series provides a chronological presentation of all significant events and developments in space exploration and the administration of the space program during the period covered