In this work, we demonstrate a modified photometric stereo system with perfect pixel

registration, capable of reconstructing continuous real-time 3D video at ~8 Hz for 64 x 64 image resolution by employing evolutionary compressed sensing

Edgar, Matthew

Gibson, Graham

Padgett, Miles

Radwell, Neal

Zhang, Yiwei

English

Yiwei Zhang

Matthew P. Edgar

Neal Radwell

Graham M. Gibson

Miles J. Padgett

Crossref

Fast Compressive 3D Single-pixel Imaging

Enlighten

      Zhang, Y., Edgar, M., Radwell, N., Gibson, G. and Padgett, M. (2016) Fast Compressive 3D Single-pixel Imaging. 3D Image Acquisition and Display: Technology, Perception and Applications, Heidelberg, Germany, 25–28 Jul 2016. TW3A.3. ISBN 9781943580156.     There may be differences between this version and the published version. You are advised to consult the publisher’s version if you wish to cite from it.    http://eprints.gla.ac.uk/151057/             Deposited on: 6 November 2017                       Enlighten – Research publications by members of the University of Glasgow http://eprints.gla.ac.uk Fast Compressive 3D Single-pixel Imaging  Yiwei Zhang, Matthew P. Edgar, Neal Radwell, Graham M. Gibson and  Miles J. Padgett SUPA, School of Physics and Astronomy, University of Glasgow, Glasgow, G12 8QQ, UK Corresponding Author e-mail address: ewayzhang@gmail.com (Y.Z.)  Abstract: In this work, we demonstrate a modified photometric stereo system with perfect pixel registration, capable of reconstructing continuous real-time 3D video at ~8 Hz for 64 x 64 image resolution by employing evolutionary compressed sensing.   OCIS codes: 110.0110; 120.5240.  1.  Introduction Photometric stereo is a well-known 3D imaging technique, which defines the surface normals according to measured intensity differences between images taken under different incident lighting directions [1, 2]. This approach demands the scene to remain completely static whilst the lighting condition changes in order to prevent surface reconstruction errors, which limits its scope in real-time applications. Single-pixel imaging, which determines the apparent lighting conditions of a scene by employing single-pixel detectors as imaging devices and a spatial light modulator to provide either structured detection or structured illumination, is an alternative computational 3D imaging technique to provide perfect pixel registration [3]. However, the size of projection patterns required in the single-pixel system may lead to lengthy acquisition process.  Orthogonal and pseudo-random bases have been employed which significantly reduces the acquisition time, however, the finite modulation rates of micro-electomechanical-systems (MEMS) technology, typically ~20kHz, places restrictions on the achievable frame rates even for relatively low resolution images. A number of compressed sensing schemes have been developed in recent years to enable real-time compressed sensing for large image resolutions. In this work, we employ one of these compressive strategies, known as evolutionary compressed sensing [4, 5], to demonstrate a real-time 3D single-pixel imaging system at ~8 Hz for 64 x 64 image resolution, 4 times faster than a conventional raster-scanning sampling strategy.    2.  Imaging System Design In the system, as illustrated in Figure 1, we choose a 3W white LED, a digital micro-mirror device (DMD) with 1024 x 768 pixels, and a camera lens with a 24mm focal length to provide structured illumination at a rate of 22 kHz. A data acquisition board (DAB) is used to covert analog intensity measured by photodetectors (PD) into digital signals at a rate of 250 kHz which subsequently were processed to reconstruct 2D images via computational imaging, following a 3D image based on photometric stereo.   Fig. 1. 3D single-pixel imaging system. The system contains a DMD, a camera lens and four spatially separated photodetectors fixed surrounding it. The DMD chip is illuminated by a white LED and the structured light patterns are projected through the lens onto the object. A plastic polarizer sheet is used in front of the photodetectors horizontally and the camera lens vertically to remove specular reflection on the object.  To maximize sampling efficiency, a Hadamard matrix in the order of 22k, which provides orthogonal structured patterns to avoid overlap information, is preloaded for the DMD. In our system, a 4096 x 4096 Hadamard matrix is used to generate 4096 of 64 x 64 2D Hadamard derived patterns to modulate the light source. And by this analogy, we generate 16384 of 128 x 128 2D Hadamard derived patterns.    We order the Hadamard patterns based on their corresponding mean signal intensities from the photodetectors, and utilize the top-ranking patterns to form 2D images. Based on evolutionary compressive sensing algorithm, we replace a small percentage (we choose 10% patterns in this experiment) of low-ranking patterns among the top-ranking patterns with the ones that are randomly selected from the rest Hadamard patterns each time for fuzzy self-adjustment due to the variations between adjacent frames. By combining those 2D images, we then performed photometric stereo to gain 3D images.  3.  Results In Figure 2, we perform 3D reconstruction of a skull at 128 x 128 pixel resolution by using five different numbers of pattern pairs: 16384 pattern pairs, 12288 pattern pairs, 8192 pattern pairs, 4096 pattern pairs, and 2048 pattern pairs, equivalent to 100% (zero-compression), 75%, 50% 25% and 12.5% compression ratio.    Fig. 2. 3D reconstruction comparison with evolutionary compressed sensing. The object was reconstructed at 128x128 pixel resolution with five different compression ratios: 12.5%, 25%, 50%, 75% and 100%.  We choose the zero-compression data as the ground truth to compare the relative root-mean square (RMS) errors of the height value in those reconstructed 3D images, and report the performance in Table 1. The result shows that the RMS error of the object’s height value increases when using less pattern pairs (higher compression). Table 1. Relative RMS error comparison at 128x128 pixel resolution.   In Figure 3, we present a sample of real-time compressed 64 x 64 pixel resolution 3D video frames in 1 second for a skull physically rotated about the z-axis and the 3D reconstructed model rotated simultaneously about the y-axis on the front controller at an angle range of +/- 30 degrees. Each frame is produced based on 1024 patterns (25% compression ratio), which equals to the same amount of patterns for a zero-compression 32 x 32 pixel resolution 3D image reconstruction. The frame rate of this 3D video is ~8Hz, four times faster than the zero-compression 64 x 64 pixel resolution one.   Fig. 3. Sample of a compressed video-rate 64x64 pixel resolution 3D frames in 1 second. Each 3D frame is produced based on 1024 patterns (25% compression ratio) acquired at a frame rate of ~8Hz for an object physically rotated about the z-axis, whilst the 3D reconstructed model is rotated about the y-axis at an angel range of [-30, 30] controlled by the system.  In Figure 4, we demonstrate a 10-second real-time 3D video frames of a Santa object reconstructed at 128x128 pixel resolution at a frame rate of 0.9Hz, using 4096 Hadamard patterns (25% compression ratio). The object is physically rotated about the z-axis.   Fig. 4. Sample of a 10-second 128x128 pixel resolution 3D video frames of a Santa object. Each 3D frame is produced based on 4096 patterns (25% compression ratio) acquired at approximately a frame rate of ~1Hz while the Santa object is physically rotated about the z-axis.   References   1. R. J. Woodham, “Photometric method for determining surface orientation from multiple images,” Opt. Eng. 19, 139–144 (1980).  2. Y. Zhang, M. G. Graham, R. Hay, R. W. Bowman, M. J. Padgett, M. P. Edgar, “A fast 3D reconstruction system with a low-cost camera accessory,” Sci. Rep. 5, e10909 (2015).   3. B. Sun, M. P. Edgar, R. Bowman, L. E. Vittert, S. Welsh, A. Bowman, M. J. Padgett, “3D computational imaging with single-pixel detectors,” Science 340, 844-847 (2013).   4. Y. Zhang, M. P. Edgar, B. Sun, N. Radwell, G. M. Gibson, M. J. Padgett, “3D single-pixel video,” J. Opt, 18(3), 035203(2016).  5. M. P. Edgar, G. M. Gibson, R. W. Bowman, B. Sun, N. Radwell, K. J. Mitchell, S. S. Welsh, M. J. Padgett, “Simultaneous real-time visible and infrared video with single-pixel detectors,” Sci. Rep. 5, e10669 (2015).   

Enlighten: Publications

http://eprints.gla.ac.uk/151057/7/151057.pdf

Fast Compressive 3D Single-pixel Imaging

Abstract

Similar works

Full text

Available Versions

Crossref

Enlighten

Enlighten: Publications