Object-based 2D-to-3D video conversion for effective stereoscopic content generation in 3D-TV applications

Three-dimensional television (3D-TV) has gained increasing popularity in the broadcasting domain, as it enables enhanced viewing experiences in comparison to conventional two-dimensional (2D) TV. However, its application has been constrained due to the lack of essential contents, i.e., stereoscopic videos. To alleviate such content shortage, an economical and practical solution is to reuse the huge media resources that are available in monoscopic 2D and convert them to stereoscopic 3D. Although stereoscopic video can be generated from monoscopic sequences using depth measurements extracted from cues like focus blur, motion and size, the quality of the resulting video may be poor as such measurements are usually arbitrarily defined and appear inconsistent with the real scenes. To help solve this problem, a novel method for object-based stereoscopic video generation is proposed which features i) optical-flow based occlusion reasoning in determining depth ordinal, ii) object segmentation using improved region-growing from masks of determined depth layers, and iii) a hybrid depth estimation scheme using content-based matching (inside a small library of true stereo image pairs) and depth-ordinal based regularization. Comprehensive experiments have validated the effectiveness of our proposed 2D-to-3D conversion method in generating stereoscopic videos of consistent depth measurements for 3D-TV applications

Tunable lenses: Dynamic characterization and fine-tuned control for high-speed applications

Tunable lenses are becoming ubiquitous, in applications including microscopy, optical coherence tomography, computer vision, quality control, and presbyopic corrections. Many applications require an accurate control of the optical power of the lens in response to a time-dependent input waveform. We present a fast focimeter (3.8 KHz) to characterize the dynamic response of tunable lenses, which was demonstrated on different lens models. We found that the temporal response is repetitive and linear, which allowed the development of a robust compensation strategy based on the optimization of the input wave, using a linear time-invariant model. To our knowledge, this work presents the first procedure for a direct characterization of the transient response of tunable lenses and for compensation of their temporal distortions, and broadens the potential of tunable lenses also in high-speed applicationsVA and EL acknowledge financial support from Comunidad de Madrid and Marie Curie Action of the European Union FP7/2007-2013 COFUND 291820; XB from Comunidad de Madrid Doctorado Industrial IND2017/BMD-7670; EL from Spanish Government Ramon y Cajal Program RyC-2016-21125; EG from Spanish Government Torres-Quevedo Program PTQ-15-07432; LS from EU H2020 SME Innovation Associate GA-739882; EG from EIT Health; SM from ERC Grant Agreement ERC-2011-AdC 294099 and Spanish Government Grants FIS2014-56643-R; SM and CD from Spanish Government Grant FIS2017-84753-R; and CD from DTS16-0012

Astigmatism and Pseudoaccommodation in Pseudophakic Eyes

noAdvanced IOLs with circumferential zones of different power provide pseudoaccommodation. We investigated the potential for power variation with meridian, namely astigmatism, to provide pseudo-accommodation. With appropriate power and axis orientations, acceptable pseudo-accommodation can be achieved

High-quality 3D shape measurement with binarized dual phase-shifting method

ABSTRACT 3-D technology is commonplace in today\u27s world. They are used in many dierent aspects of life. Researchers have been keen on 3-D shape measurement and 3-D reconstruction techniques in past decades as a result of inspirations from dierent applications ranging from manufacturing, medicine to entertainment. The techniques can be broadly divided into contact and non-contact techniques. The contact techniques like coordinate measuring machine (CMM) dates way back to 1950s. It has been used extensively in the industries since then. It becomes predominant in industrial inspections owing to its high accuracy in the order of m. As we know that quality control is an important part of modern industries hence the technology is enjoying great popularity. However, the main disadvantage of this method is its slow speeds due to its requirement of point-by-point touch. Also, since this is a contact process, it might deform a soft object while performing measurements. Such limitations led the researchers to explore non-contact measurement technologies (optical metrology techniques). There are a variety of optical techniques developed till now. Some of the well-known technologies include laser scanners, stereo vision, and structured light systems. The main limitation of laser scanners is its limited speed due to its point-by-point or line-by-line scanning process. The stereo vision uses two cameras which take pictures of the object at two dierent angles. Then epipolar geometry is used to determine the 3-D coordinates of points in real-world. Such technology imitates human vision, but it had a few limitations too like the diculty of correspondence detection for uniform or periodic textures. Hence structured light systems were introduced which addresses the aforementioned limitations. There are various techniques developed including 2-D pseudo-random codication, binary codication, N-ary codication and digital fringe projection (DFP). The limitation of 2-D pseudo-random codication technique is its inability to achieve high spatial resolution since any uniquely generated and projected feature requires a span of several projector pixels. The binary codication techniques reduce the requirement of 2-D features to 1-D ones. However, since there are only two intensities, it is dicult to differentiate between the individual pixels within each black or white stripe. The other disadvantage is that n patterns are required to encode 2n pixels, meaning that the measurement speeds will be severely affected if a scene is to be coded with high-resolution. Dierently, DFP uses continuous sinusoidal patterns. The usage of continuous patterns addresses the main disadvantage of binary codication (i.e. the inability of this technique to differentiate between pixels was resolved by using sinusoid patterns). Thus, the spatial resolution is increased up to camera-pixel-level. On the other hand, since the DFP technique used 8-bit sinusoid patterns, the speed of measurement is limited to the maximum refreshing rate of 8-bit images for many video projectors (e.g. 120 Hz). This made it inapplicable for measurements of highly dynamic scenes. In order to overcome this speed limitation, the binary defocussing technique was proposed which uses 1-bit patterns to produce sinusoidal prole by projector defocusing. Although this technique has signicantly boosted the measurement speed up to kHz-level, if the patterns are not properly defocused (nearly focused or overly defocused), increased phase noise or harmonic errors will deteriorate the reconstructed surface quality. In this thesis research, two techniques are proposed to overcome the limitations of both DFP and binary defocusing technique: binarized dual phase shifting (BDPS) technique and Hilbert binarized dual phase shifting technique (HBDPS). Both techniques were able to achieve high-quality 3-D shape measurements even when the projector is not sufficiently defocused. The harmonic error was reduced by 47% by the BDPS method, and 74% by the HBDPS method. Moreover, both methods use binary patterns which preserve the speed advantage of the binary technology, hence it is potentially applicable to simultaneous high-speed and high-accuracy 3D shape measurements

Lightfield Analysis and Its Applications in Adaptive Optics and Surveillance Systems

An image can only be as good as the optics of a camera or any other imaging system allows it to be. An imaging system is merely a transformation that takes a 3D world coordinate to a 2D image plane. This can be done through both linear/non-linear transfer functions. Depending on the application at hand it is easier to use some models of imaging systems over the others in certain situations. The most well-known models are the 1) Pinhole model, 2) Thin Lens Model and 3) Thick lens model for optical systems. Using light-field analysis the connection through these different models is described. A novel figure of merit is presented on using one optical model over the other for certain applications. After analyzing these optical systems, their applications in plenoptic cameras for adaptive optics applications are introduced. A new technique to use a plenoptic camera to extract information about a localized distorted planar wave front is described. CODEV simulations conducted in this thesis show that its performance is comparable to those of a Shack-Hartmann sensor and that they can potentially increase the dynamic range of angles that can be extracted assuming a paraxial imaging system. As a final application, a novel dual PTZ-surveillance system to track a target through space is presented. 22X optic zoom lenses on high resolution pan/tilt platforms recalibrate a master-slave relationship based on encoder readouts rather than complicated image processing algorithms for real-time target tracking. As the target moves out of a region of interest in the master camera, it is moved to force the target back into the region of interest. Once the master camera is moved, a precalibrated lookup table is interpolated to compute the relationship between the master/slave cameras. The homography that relates the pixels of the master camera to the pan/tilt settings of the slave camera then continue to follow the planar trajectories of targets as they move through space at high accuracies

Deep Learning based Virtual Point Tracking for Real-Time Target-less Dynamic Displacement Measurement in Railway Applications

In the application of computer-vision based displacement measurement, an optical target is usually required to prove the reference. In the case that the optical target cannot be attached to the measuring objective, edge detection, feature matching and template matching are the most common approaches in target-less photogrammetry. However, their performance significantly relies on parameter settings. This becomes problematic in dynamic scenes where complicated background texture exists and varies over time. To tackle this issue, we propose virtual point tracking for real-time target-less dynamic displacement measurement, incorporating deep learning techniques and domain knowledge. Our approach consists of three steps: 1) automatic calibration for detection of region of interest; 2) virtual point detection for each video frame using deep convolutional neural network; 3) domain-knowledge based rule engine for point tracking in adjacent frames. The proposed approach can be executed on an edge computer in a real-time manner (i.e. over 30 frames per second). We demonstrate our approach for a railway application, where the lateral displacement of the wheel on the rail is measured during operation. We also implement an algorithm using template matching and line detection as the baseline for comparison. The numerical experiments have been performed to evaluate the performance and the latency of our approach in the harsh railway environment with noisy and varying backgrounds