On-the-Go Reflectance Transformation Imaging with Ordinary Smartphones

Reflectance Transformation Imaging (RTI) is a popular technique that allows the recovery of per-pixel reflectance information by capturing an object under different light conditions. This can be later used to reveal surface details and interactively relight the subject. Such process, however, typically requires dedicated hardware setups to recover the light direction from multiple locations, making the process tedious when performed outside the lab. We propose a novel RTI method that can be carried out by recording videos with two ordinary smartphones. The flash led-light of one device is used to illuminate the subject while the other captures the reflectance. Since the led is mounted close to the camera lenses, we can infer the light direction for thousands of images by freely moving the illuminating device while observing a fiducial marker surrounding the subject. To deal with such amount of data, we propose a neural relighting model that reconstructs object appearance for arbitrary light directions from extremely compact reflectance distribution data compressed via Principal Components Analysis (PCA). Experiments shows that the proposed technique can be easily performed on the field with a resulting RTI model that can outperform state-of-the-art approaches involving dedicated hardware setups

Calibration of a Telecentric Structured-light Device for Micrometric 3D Reconstruction

Structured-light 3D reconstruction techniques are employed in a wide range of applications for industrial inspection. In particular, some tasks require micrometric precision for the identification of microscopic surface irregularities. We propose a novel calibration technique for structured-light systems adopting telecentric lenses for both camera and projector. The device exploits a fixed light pattern (striped-based) to perform accurate microscopic surface reconstruction and measurements. Our method employs a sphere with a known radius as calibration target and takes advantage of the orthographic projection model of the telecentric lenses to recover the bundle of planes originated by the projector. Once the sheaf of parallel planes is properly described in the camera reference frame, the triangulation of the surface’s object hit by the light stripes is immediate. Moreover, we tested our technique in a real-world scenario for industrial surface inspection by implementing a complete pipeline to recover the intersections between the projected planes and the surface. Experimental analysis shows the robustness of the proposed approach against synthetic and real-world test data

Exploring Audio Compression as Image Completion in Time-Frequency Domain

Audio compression is usually achieved with algorithms that exploit spectral properties of the given signal such as frequency or temporal masking. In this paper we propose to tackle such a problem from a different point of view, considering the time-frequency domain of an audio signal as an intensity map to be reconstructed via a data-driven approach. The compression stage removes some selected input values from the time-frequency representation of the original signal. Then, decompression works by reconstructing the missing samples as an image completion task. Our method is divided into two main parts: first, we analyse the feasibility of a data-driven audio reconstruction with missing samples in its time-frequency representation. To do so, we exploit an existing CNN model designed for depth completion, involving a sequence of sparse convolutions to deal with absent values. Second, we propose a method to select the values to be removed at compression stage, maximizing the perceived audio quality of the decompressed signal. In the experimental section we validate the proposed technique on some standard audio datasets and provide an extensive study on the quality of the reconstructed signal under different conditions

Deep Demosaicing for Polarimetric Filter Array Cameras

Polarisation Filter Array (PFA) cameras allow the analysis of light polarisation state in a simple and cost-effective manner. Such filter arrays work as the Bayer pattern for colour cameras, sharing similar advantages and drawbacks. Among the others, the raw image must be demosaiced considering the local variations of the PFA and the characteristics of the imaged scene. Non-linear effects, like the cross-talk among neighbouring pixels, are difficult to explicitly model and suggest the potential advantage of a data-driven learning approach. However, the PFA cannot be removed from the sensor, making it difficult to acquire the ground-truth polarization state for training. In this work we propose a novel CNN-based model which directly demosaics the raw camera image to a per-pixel Stokes vector. Our contribution is twofold. First, we propose a network architecture composed by a sequence of Mosaiced Convolutions operating coherently with the local arrangement of the different filters. Second, we introduce a new method, employing a consumer LCD screen, to effectively acquire real-world data for training. The process is designed to be invariant by monitor gamma and external lighting conditions. We extensively compared our method against algorithmic and learning-based demosaicing techniques, obtaining a consistently lower error especially in terms of polarisation angle

A Light Source Calibration Technique for Multi-camera Inspection Devices

Industrial manufacturing processes often involve a visual control system to detect possible product defects during production. Such inspection devices usually include one or more cameras and several light sources designed to highlight surface imperfections under different illumination conditions (e.g. bumps, scratches, holes). In such scenarios, a preliminary calibration procedure of each component is a mandatory step to recover the system’s geometrical configuration and thus ensure a good process accuracy. In this paper we propose a procedure to estimate the position of each light source with respect to a camera network using an inexpensive Lambertian spherical target. For each light source, the target is acquired at different positions from different cameras, and an initial guess of the corresponding light vector is recovered from the analysis of the collected intensity isocurves. Then, an energy minimization process based on the Lambertian shading model refines the result for a pr ecise 3D localization. We tested our approach in an industrial setup, performing extensive experiments on synthetic and real-world data to demonstrate the accuracy of the proposed approach

Learning Computer Vision through the development of a Camera-trackable Game Controller

The trade-off between the available classroom time and the complexity of the proposed task is central to the design of any Computer Science laboratory lecture. Special care must be taken to build up an experimental setup that allows the students to get the most significant information from the experience without getting lost in the details. This is especially true when teaching Computer Vision concepts to prospective students that own little or no previous background in programming and a strongly diversified knowledge with respect to mathematics. In this chapter, the authors describe a setup for a laboratory lecture that has been administered through several years to prospective students of the Computer Science course at the University of Venice. The goal is to teach basic concepts such as color spaces or image transforms through a rewarding task, which is the development of a vision-based game controller similar in spirit to the recent human-machine interfaces adopted by the current generation of game consoles

A Practical Setup for Projection-based Augmented Maps

Projected Augmented Reality is a human-computer interaction scenario where synthetic data, rather than being rendered on a display, are directly projected on the real world. Differening from screen-based approaches, which only require the pose of the camera with respect to the world, this setup poses the additional hurdle of knowing the relative pose between capturing and projecting devices. In this chapter, the authors propose a thorough solution that addresses both camera and projector calibration using a simple fiducial marker design. Specifically, they introduce a novel Augmented Maps setup where the user can explore geographically located information by moving a physical inspection tool over a printed map. Since the tool presents both a projection surface and a 3D-localizable marker, it can be used to display suitable information about the area that it covers. The proposed setup has been evaluated in terms of accuracy of the calibration and ease of use declared by the users

Observation of extreme sea waves in a space-time ensemble

In this paper, an observational space-time ensemble of sea surface elevations is investigated in search of the highest waves of the sea state. Wave data were gathered by means of a stereo camera system, which was installed on top of a fixed oceanographic platform located in the Adriatic Sea (Italy). Waves were measured during a mature sea state with an average wind speed of 11 m s-1. By examining the space-time ensemble, the 3D wave groups have been isolated while evolving in the 2D space and grabbed "when and where" they have been close to the apex of their development, thus exhibiting large surface displacements. The authors have selected the groups displaying maximal crest height exceeding the threshold adopted to define rogue waves in a time record, that is, 1.25 times the significant wave height (Hs). The records at the spatial positions where such large crests occurred have been analyzed to derive the empirical distributions of crest and wave heights, which have been compared against standard statistical linear and nonlinear models. Here, the maximal observed wave crests have resulted to be outliers of the standard statistics, behaving as isolated members of the sample, apparently uncorrelated with other waves of the record. However, this study has found that these unexpectedly large wave crests are better approximated by a space-time model for extreme crest heights. The space-time model performance has been improved, deriving a second-order approximation of the linear model, which has provided a fair agreement with the empirical maxima. The present investigation suggests that very large waves may be more numerous than generally expected.In this paper, an observational space-time ensemble of sea surface elevations is investigated in search of the highest waves of the sea state. Wave data were gathered by means of a stereo camera system, which was installed on top of a fixed oceanographic platform located in the Adriatic Sea (Italy). Waves were measured during a mature sea state with an average wind speed of 11 m s(-1). By examining the space-time ensemble, the 3D wave groups have been isolated while evolving in the 2D space and grabbed "when and where" they have been close to the apex of their development, thus exhibiting large surface displacements. The authors have selected the groups displaying maximal crest height exceeding the threshold adopted to define rogue waves in a time record, that is, 1.25 times the significant wave height (H-s). The records at the spatial positions where such large crests occurred have been analyzed to derive the empirical distributions of crest and wave heights, which have been compared against standard statistical linear and nonlinear models. Here, the maximal observed wave crests have resulted to be outliers of the standard statistics, behaving as isolated members of the sample, apparently uncorrelated with other waves of the record. However, this study has found that these unexpectedly large wave crests are better approximated by a space-time model for extreme crest heights. The space-time model performance has been improved, deriving a second-order approximation of the linear model, which has provided a fair agreement with the empirical maxima. The present investigation suggests that very large waves may be more numerous than generally expected

Robust cylinder estimation in point clouds from pairwise axes similarities

The ubiquitous presence of cylindrical shapes in both natural and man-made environments makes their automated extraction a pivotal task for a broad range of applications such as robot manipulation, reverse engineering and automated industrial inspection. Albeit conceptually simple, the task of fitting cylinders from 3D data can quickly become challenging if performed "in-the-wild", when no prior is given to the number of primitives to find or when the point cloud is noisy and not oriented. In this paper we introduce a new robust approach to iteratively extract cylindrical primitives from a 3D point cloud by exploiting mutual consensus of different cylinder candidates. First, a set of possible axes is generated by slicing the point cloud with multiple random planes. Then, a game-theoretic inlier selection process is performed to extract a subset of axes maximizing the fitness against a payoff function based on the shortest geodesic path in SE(3) between pairs of corresponding 3D lines. Finally, the probability distribution resulting from the previous selection step is used to weight the input candidates and robustly obtain the final cylinder coefficients. Compared to other methods, our approach does not require point normals, offers superior resilience to noise and does not depend on delicate tuning of multiple parameters