Investigations into applications of photometric stereo and single-pixel imaging

Computational image reconstruction is generally an inverse procedure which helps to recover the original information in a scene. Various imaging techniques have been developed to extract certain kinds of information for applications in different fields. The focus of this thesis is to improve two elegant and powerful methods among those approaches, namely, photometric stereo and single-pixel imaging, into a more practical and applicable phase. With the advances in modern imaging technology, 3D information is playing an increasingly significant role in real-world applications, from robotic vision, manufacturing, entertainment, and biology to security. While an immense amount of research has been conducted over the last few decades, the requirement of generating a rapid and accurate estimation of scene depth information with a cost-efficient system remains challenging. In the first work, we developed an inexpensive computational camera system allowing fast 3D reconstruction of objects based on the principle of photometric stereo. By analysing the estimated 3D data of various objects, we noticed good quantitative agreement with the known reference object with a wide viewing angle. With a low-cost accessory, our system provides a simplified reconstruction routine alongside a high efficiency, which extends its portability and capability for practical applications. Single-pixel imaging is an emerging paradigm which utilises spatial correlation of light with a single-pixel detector to form an image. It provides an alternative strategy to conventional imaging techniques which reply on a pixelated sensor for spatial resolution. In the second work, we combined photometric stereo with single-pixel imaging to evolve a new 3D imaging system with an efficient realtime sampling scheme. By utilising a high-speed structured illumination and four single-pixel detectors, multiple images of a scene with different shading profiles were able to be reconstructed with perfect pixel registration for depth estimation, empowering 3D imaging of dynamic scene. A compressive strategy, known as evolutionary compressed sensing, was further employed to improve the frame rate of 3D single-pixel video at an expense of only a modest reduction in image quality. This system represents a step-forward towards real-time 3D single-pixel imaging. By using single-pixel imaging technique, it offers a feasible solution for situations that are costly or constrained with conventional pixelated camera sensor, for instance, near-infrared (NIR) imaging and fluorescence imaging through multimode fibres. However, the signal-to-noise ratio (SNR) scales poorly when increasing the single-pixel imaging resolution. In the last work, we developed a NIR single-pixel imaging system with micro-scanning, an optimisation approach that generates a higher-resolution image while maintaining the SNR of the lower-resolution images where it is derived from. With the use of sunlight and an infrared heat lamp as the illumination sources and a set of NIR bandpass filters, our system indicated a well capability of revealing the water absorption underneath the surfaces of plant leaves and fruits compared to an expensive pixelated InGaAs camera. Additional efforts were devoted to further improve the image quality of a modified single-pixel imaging system that allows visible and NIR dual-band detection simultaneously

Change blindness: eradication of gestalt strategies

Arrays of eight, texture-defined rectangles were used as stimuli in a one-shot change blindness (CB) task where there was a 50% chance that one rectangle would change orientation between two successive presentations separated by an interval. CB was eliminated by cueing the target rectangle in the first stimulus, reduced by cueing in the interval and unaffected by cueing in the second presentation. This supports the idea that a representation was formed that persisted through the interval before being 'overwritten' by the second presentation (Landman et al, 2003 Vision Research 43149–164]. Another possibility is that participants used some kind of grouping or Gestalt strategy. To test this we changed the spatial position of the rectangles in the second presentation by shifting them along imaginary spokes (by ±1 degree) emanating from the central fixation point. There was no significant difference seen in performance between this and the standard task [F(1,4)=2.565, p=0.185]. This may suggest two things: (i) Gestalt grouping is not used as a strategy in these tasks, and (ii) it gives further weight to the argument that objects may be stored and retrieved from a pre-attentional store during this task

Le cinéma omnistéréo ou l'art d'avoir des yeux tout le tour de la tête

Cette thèse s'intéresse à des aspects du tournage, de la projection et de la perception du cinéma stéréo panoramique, appelé aussi cinéma omnistéréo. Elle s'inscrit en grande partie dans le domaine de la vision par ordinateur, mais elle touche aussi aux domaines de l'infographie et de la perception visuelle humaine. Le cinéma omnistéréo projette sur des écrans immersifs des vidéos qui fournissent de l'information sur la profondeur de la scène tout autour des spectateurs. Ce type de cinéma comporte des défis liés notamment au tournage de vidéos omnistéréo de scènes dynamiques, à la projection polarisée sur écrans très réfléchissants rendant difficile l'estimation de leur forme par reconstruction active, aux distorsions introduites par l'omnistéréo pouvant fausser la perception des profondeurs de la scène. Notre thèse a tenté de relever ces défis en apportant trois contributions majeures. Premièrement, nous avons développé la toute première méthode de création de vidéos omnistéréo par assemblage d'images pour des mouvements stochastiques et localisés. Nous avons mis au point une expérience psychophysique qui montre l'efficacité de la méthode pour des scènes sans structure isolée, comme des courants d'eau. Nous proposons aussi une méthode de tournage qui ajoute à ces vidéos des mouvements moins contraints, comme ceux d'acteurs. Deuxièmement, nous avons introduit de nouveaux motifs lumineux qui permettent à une caméra et un projecteur de retrouver la forme d'objets susceptibles de produire des interréflexions. Ces motifs sont assez généraux pour reconstruire non seulement les écrans omnistéréo, mais aussi des objets très complexes qui comportent des discontinuités de profondeur du point de vue de la caméra. Troisièmement, nous avons montré que les distorsions omnistéréo sont négligeables pour un spectateur placé au centre d'un écran cylindrique, puisqu'elles se situent à la périphérie du champ visuel où l'acuité devient moins précise.This thesis deals with aspects of shooting, projection and perception of stereo panoramic cinema, also called omnistereo cinema. It falls largely in the field of computer vision, but it also in the areas of computer graphics and human visual perception. Omnistereo cinema uses immersive screens to project videos that provide depth information of a scene all around the spectators. Many challenges remain in omnistereo cinema, in particular shooting omnistereo videos for dynamic scenes, polarized projection on highly reflective screens making difficult the process to recover their shape by active reconstruction, and perception of depth distortions introduced by omnistereo images. Our thesis addressed these challenges by making three major contributions. First, we developed the first mosaicing method of omnistereo videos for stochastic and localized motions. We developed a psychophysical experiment that shows the effectiveness of the method for scenes without isolated structure, such as water flows. We also propose a shooting method that adds to these videos foreground motions that are not as constrained, like a moving actor. Second, we introduced new light patterns that allow a camera and a projector to recover the shape of objects likely to produce interreflections. These patterns are general enough to not only recover the shape of omnistereo screens, but also very complex objects that have depth discontinuities from the viewpoint of the camera. Third, we showed that omnistereo distortions are negligible for a viewer located at the center of a cylindrical screen, as they are in the periphery of the visual field where the human visual system becomes less accurate

Three-dimensional geometry characterization using structured light fields

Tese de doutoramento. Engenharia Mecânica. Faculdade de Engenharia. Universidade do Porto. 200

On the design of Neutral Scanning Helium Atom Microscopes (SHeM) : Optimal configurations and evaluation of experimental findings

Scanning Helium Microscopes (SHeMs) are novel microscopy tools using neutral helium atoms as the imaging probe. Helium atoms have several advantages compared to other probes such as electrons or helium ions. Helium atoms are neutral and inert and when compared to electrons their higher mass leads to a smaller de-Broglie wavelength for a given energy. Furthermore, helium atoms are strictly surface sensitive, scattering off the electron density distribution off the surface. These combined properties allow for non-destructive mapping of the surface of virtually any vacuum-compatible solid sample. Helium ions have a similar mass but they interact more strongly with the sample because they are not inert and require much higher energies to achieve electrostatic focusing. Charge neutrality makes helium a great imaging corpuscle, but also means that designing SHeMs is very difficult. Neutral helium atoms are very hard to manipulate, as electromagnetic fields cannot be used to focus and redirect the beam - instead, one needs to use diffraction optics and apertures. They are also hard to detect because helium has the highest ionisation potential of all atoms - hindering the task of ionisation based detectors. Therefore, to have a functioning microscope, one needs to form a highly intense atom beam. This thesis presents the work done over the last years to optimise the intensity of SHeMs, and more generally their atom-optics configuration. Amongst the papers included here are the first ones to show that SHeM optics have well-defined intensity maxima that give optimal designs. These papers show that existing designs were suboptimal and that the intensity could be increased several orders of magnitude. This thesis also features the first paper to present a design for a 3D imaging SHeM. A true nano-scale stereo microscope based on Heliometric stereo, a technique adapted from light. Besides these theoretical papers, two papers are included that focus on understanding the helium beam using experimental data. These papers are important as they provide the experimental foundations for the theoretical models used. Amongst other findings, the papers explore the importance of the Knudsen number at the skimmer, the validity of different intensity models, and the top-hat profile of the beam. The research presented here happened in parallel to a two order of magnitude improvement in detector efficiency. I believe that now we are in the position to build high-resolution SHeMs that have the potential to become an important tool for science and industry.. . .Doktorgradsavhandlin

Project Tech Top study of lunar, planetary and solar topography Final report

Data acquisition techniques for information on lunar, planetary, and solar topograph

Acquisition and modeling of 3D irregular objects.

by Sai-bun Wong.Thesis (M.Phil.)--Chinese University of Hong Kong, 1994.Includes bibliographical references (leaves 127-131).Abstract --- p.vAcknowledgment --- p.viiChapter 1 --- Introduction --- p.1-8Chapter 1.1 --- Overview --- p.2Chapter 1.2 --- Survey --- p.4Chapter 1.3 --- Objectives --- p.6Chapter 1.4 --- Thesis Organization --- p.7Chapter 2 --- Range Sensing --- p.9-30Chapter 2.1 --- Alternative Approaches to Range Sensing --- p.9Chapter 2.1.1 --- Size Constancy --- p.9Chapter 2.1.2 --- Defocusing --- p.11Chapter 2.1.3 --- Deconvolution --- p.14Chapter 2.1.4 --- Binolcular Vision --- p.18Chapter 2.1.5 --- Active Triangulation --- p.20Chapter 2.1.6 --- Time-of-Flight --- p.22Chapter 2.2 --- Transmitter and Detector in Active Sensing --- p.26Chapter 2.2.1 --- Acoustics --- p.26Chapter 2.2.2 --- Optics --- p.28Chapter 2.2.3 --- Microwave --- p.29Chapter 2.3 --- Conclusion --- p.29Chapter 3 --- Scanning Mirror --- p.31-47Chapter 3.1 --- Scanning Mechanisms --- p.31Chapter 3.2 --- Advantages of Scanning Mirror --- p.32Chapter 3.3 --- Feedback of Scanning Mirror --- p.33Chapter 3.4 --- Scanning Mirror Controller --- p.35Chapter 3.5 --- Point-to-Point Scanning --- p.39Chapter 3.6 --- Line Scanning --- p.39Chapter 3.7 --- Specifications and Measurements --- p.41Chapter 4 --- The Rangefinder with Reflectance Sensing --- p.48-58Chapter 4.1 --- Ambient Noises --- p.49Chapter 4.2 --- Occlusion/Shadow --- p.49Chapter 4.3 --- Accuracy and Precision --- p.50Chapter 4.4 --- Optics --- p.53Chapter 4.5 --- Range/Reflectance Crosstalk --- p.56Chapter 4.6 --- Summary --- p.58Chapter 5 --- Computer Generation of Range Map --- p.59-75Chapter 5.1 --- Homogenous Transformation --- p.61Chapter 5.2 --- From Global to Viewer Coordinate --- p.63Chapter 5.3 --- Z-buffering --- p.55Chapter 5.4 --- Generation of Range Map --- p.66Chapter 5.5 --- Experimental Results --- p.68Chapter 6 --- Characterization of Range Map --- p.76-90Chapter 6.1 --- Mean and Gaussian Curvature --- p.76Chapter 6.2 --- Methods of Curvature Generation --- p.78Chapter 6.2.1 --- Convolution --- p.78Chapter 6.2.2 --- Local Surface Patching --- p.81Chapter 6.3 --- Feature Extraction --- p.84Chapter 6.4 --- Conclusion --- p.85Chapter 7 --- Merging Multiple Characteristic Views --- p.91-119Chapter 7.1 --- Rigid Body Model --- p.91Chapter 7.2 --- Sub-rigid Body Model --- p.94Chapter 7.3 --- Probabilistic Relaxation Matching --- p.95Chapter 7.4 --- Merging the Sub-rigid Body Model --- p.99Chapter 7.5 --- Illustration --- p.101Chapter 7.6 --- Merging Multiple Characteristic Views --- p.104Chapter 7.7 --- Mislocation of Feature Extraction --- p.105Chapter 7.7.1 --- The Transform Matrix for Perfect Matching --- p.106Chapter 7.7.2 --- Introducing The Errors in Feature Set --- p.108Chapter 7.8 --- Summary --- p.113Chapter 8 --- Conclusion --- p.120-126References --- p.127-131Appendix A - Projection of Object --- p.A1-A2Appendix B - Performance Analysis on Rangefinder System --- p.B1-B16Appendix C - Matching of Two Characteristic views --- p.C1-C

Useful applications of earth-oriented satellites - Sensors and data systems

Usefulness of satellites in earth-oriented application

LSST Science Book, Version 2.0

A survey that can cover the sky in optical bands over wide fields to faint magnitudes with a fast cadence will enable many of the exciting science opportunities of the next decade. The Large Synoptic Survey Telescope (LSST) will have an effective aperture of 6.7 meters and an imaging camera with field of view of 9.6 deg^2, and will be devoted to a ten-year imaging survey over 20,000 deg^2 south of +15 deg. Each pointing will be imaged 2000 times with fifteen second exposures in six broad bands from 0.35 to 1.1 microns, to a total point-source depth of r~27.5. The LSST Science Book describes the basic parameters of the LSST hardware, software, and observing plans. The book discusses educational and outreach opportunities, then goes on to describe a broad range of science that LSST will revolutionize: mapping the inner and outer Solar System, stellar populations in the Milky Way and nearby galaxies, the structure of the Milky Way disk and halo and other objects in the Local Volume, transient and variable objects both at low and high redshift, and the properties of normal and active galaxies at low and high redshift. It then turns to far-field cosmological topics, exploring properties of supernovae to z~1, strong and weak lensing, the large-scale distribution of galaxies and baryon oscillations, and how these different probes may be combined to constrain cosmological models and the physics of dark energy.Comment: 596 pages. Also available at full resolution at http://www.lsst.org/lsst/sciboo

Reconsidering light transport : acquisition and display of real-world reflectance and geometry

In this thesis, we cover three scenarios that violate common simplifying assumptions about the nature of light transport. We begin with the first ingredient to any çD rendering: a geometry model. Most çD scanners require the object-of-interest to show diffuse refectance. The further a material deviates from the Lambertian model, the more likely these setups are to produce corrupted results. By placing a traditional laser scanning setup in a participating (in particular, fuorescent) medium, we have built a light sheet scanner that delivers robust results for a wide range of materials, including glass. Further investigating the phenomenon of fluorescence, we notice that, despite its ubiquity, it has received moderate attention in computer graphics. In particular, to date no datadriven reflectance models of fluorescent materials have been available. To describe the wavelength-shifling reflectance of fluorescent materials, we define the bispectral bidirectional reflectance and reradiation distribution function (BRRDF), for which we introduce an image-based measurement setup as well as an efficient acquisition scheme. Finally, we envision a computer display that showsmaterials instead of colours, and present a prototypical device that can exhibit anisotropic reflectance distributions similar to common models in computer graphics.In der Computergraphik und Computervision ist es unerlässlich, vereinfachende Annahmen über die Ausbreitung von Licht zumachen. In dieser Dissertation stellen wir drei Fälle vor, in denen diese nicht zutreffen. So wird die dreidimensionale Geometrie von Gegenständen oft mit Hilfe von Laserscannern vermessen und dabei davon ausgegangen, dass ihre Oberfläche diffus reflektiert. Dies ist bei den meisten Materialien jedoch nicht gegeben, so dass die Ergebnisse oft fehlerhaft sind. Indem wir das Objekt in einem fluoreszierenden Medium einbetten, kann ein klassischer CD-Scanner-Aufbau so modifiziert werden, dass er verlässliche Geometriedaten für Objekte aus verschiedensten Materialien liefert, einschließlich Glas. Auch die akkurate Nachbildung des Aussehens von Materialien ist wichtig für die photorealistische Bildsynthese. Wieder interessieren wir uns für Fluoreszenz, diesmal allerdings für ihr charakteristisches Erscheinungsbild, das in der Computergraphik bislang kaum Beachtung gefunden hat. Wir stellen einen bildbasierten Aufbau vor, mit dem die winkel- und wellenlängenabhängige Reflektanz fluoreszierender Oberflächen ausgemessen werden kann, und eine Strategie, um solche Messungen effizient abzuwickeln. Schließlich befassen wir uns mit der Idee, nicht nur Farben dynamisch anzuzeigen, sondern auch Materialien und ihr je nach Lichteinfall und Blickwinkel unterschiedliches Aussehen. Einer generellen Beschreibung des Problems folgt die konkrete Umsetzung in Formzweier Prototypen, die verschiedene Reflektanzverteilungen auf einer Oberfläche darstellen können