New Frame Rate Up-conversion Method Based On Foreground/background Segmentation

Tez (Yüksek Lisans) -- İstanbul Teknik Üniversitesi, Fen Bilimleri Enstitüsü, 2011Thesis (M.Sc.) -- İstanbul Technical University, Institute of Science and Technology, 2011Video çerçeve hız artırımı bir saniyeye düşen çerçeve sayısını artırarak bir videonun kalitesini artırmayı sağlar. Hareket dengeleyici ve hareket dengeleyici olmayan çerçeve hız artırımı bu alanı oluşturan başlıca iki tekniktir. Her iki teknikte de yapaylıklar ve tırtıklı kenarlar öznel ve nesnel olarak videonun kalitesinin düşmesine yol açar. Bu iki tip bozunum çerçeve hız artırımının iki ana problemidir. Bu problemleri ön plan ve arka plan segmentasyonunu kullanarak çözen yeni bir hareket dengeleyici çerçeve hız artırımı tekniğini bu çalışmada sunuyoruz. Bu çalışma elimizdeki çerçevelerin hız vektörlerini ölçekleyerek aradeğerlenmiş çerçevenin hareket vektörlerini elde eder. Hiç bir vektör atanmamış olan boş pikseller komşu piksellerin ön yüz/arka yüz etiketlerine bağlı olarak aradeğerlenme yöntemi ile oluşturulurlar. Çerçeveleri ön yüz ve arka yüz diye ayırabilmek için bir arka yüz çıkarıcı algoritması bu çalışmaya entegre edilmiştir. Önerilen yöntem ile videonun kalitesi artırıldığı gibi [9], [13] ve [14] yayınlarında önerilen metotlardan çok daha iyi tepe sinyal gürültü oranı elde edilmiştir.Frame rate up-conversion (FRUC) increases the quality of a video by increasing its temporal frequency. Motion compensated (MC) and non-motion compensated frame rate up conversion techniques make up the two main classes of techniques used in this area. Halo artifacts and jaggy edges cause the quality of video to be reduced both subjectively and objectively in these techniques. These are the main problem of FRUC techniques. In this work, we introduce a new method of motion compensated FRUC that uses Foreground/Background segmentation to address the occlusion problem. It scales motion vectors of existent frames to obtain motion vectors of interpolated frames. The gap pixels for which no motion vectors have been assigned are interpolated from their neighbors based on their foreground or background identity. A background subtraction algorithm was integrated to this work to segment the frames as background and foreground. The current work improves the quality of video and obtains better PSNR results than the proposed methods of [9], [13] and [14].Yüksek LisansM.Sc

Multiscale neighborhood-wise decision fusion for redundancy detection in image pairs

SIAM Journal Multiscale Modeling & SimulationTo develop better image change detection algorithms, new models able to capture spatio-temporal regularities and geometries present in an image pair are needed. In this paper, we propose a multiscale formulation for modeling semi-local inter-image interactions and detecting local or regional changes in an image pair. By introducing dissimilarity measures to compare patches and binary local decisions, we design collaborative decision rules that use the total number of detections obtained from the neighboring pixels, for different patch sizes. We study the statistical properties of the non-parametric detection approach that guarantees small probabilities of false alarms. Experimental results on several applications demonstrate that the detection algorithm (with no optical flow computation) performs well at detecting occlusions and meaningful changes for a variety of illumination conditions and signal-to-noise ratios. The number of control parameters of the algorithm is small and the adjustment is intuitive in most cases

MAP based motion field refinement methods for motion-compensated frame interpolation

학위논문 (박사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2014. 2. 김태정.In this dissertation, maximum a posteriori probability (MAP) based motion refinement methods are proposed for block-based motion-compensated frame interpolation (MCFI). The first method, called single hypothesis Bayesian approach (SHBA), is aiming at estimating the true MVF of a video frame from its observed MVF, which is the result of a block-based motion estimation (BME), by maximizing the posterior probability of the true MVF. For the estimation, the observed MVF is assumed to be a degraded version of the true MVF by locally stationary additive Gaussian noise (AGN), so the variance of the noise represents the unreliability of the observed MV. The noise variance is directly estimated from the observation vector and its select neighbors. The prior distribution of the true MVF is designed to rely on the distances between the MV and its neighbors and to properly smooth false MVs in the observation. The second algorithm, called multiple hypotheses Bayesian approach (MHBA), estimates the true MVF of a video frame from its multiple observations by maximizing the posterior probability of the true. The multiple observations, which are the results of a BME incorporating blocks of different sizes for matching, are assumed to be degraded versions of the true MVF by locally stationary AGN. The noise variances for the observations are first estimated independently and then adaptively adjusted by block-matching errors in order to solve motion boundary problem. Finally, a method, called single hypothesis Bayesian approach in a bidirectional framework (SHBA-BF), that simultaneously estimates the true forward and backward MVFs of two consecutive frames from the observed forward and backward MVFs is proposed. The observed MVFs are assumed to be degraded versions of the corresponding true MVFs by locally stationary AGN. The true forward and backward MVFs are assumed to follow the proposed joint prior distribution, which is designed such that it adaptively relies on not only the resemblance between spatially neighboring MVs but also the resemblance between the MV and its dual MV so the proposed simultaneous estimation can fully exploit duality of MVF. Experimental results show that the proposed algorithms obtain better or comparable performances as compared to the state-of-the-art BME algorithms at much lower computational complexity.Abstract i Contents iii List of Figures v List of Tables x 1 Introudction 1 1.1 Motion-Compensated Frame Interpolation . . . . . . . . . . . . . . . 1 1.2 Previous Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2.1 Exploit spatio-temporal correlation during ME . . . . . . . . 3 1.2.2 Utilize multiple block sizes for matching in ME . . . . . . . . 6 1.2.3 Correct false motion vectors in given MVFs . . . . . . . . . . 7 1.3 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2 Single Hypothesis Bayesian Approach 11 2.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.1.1 Proposed observation likelihood . . . . . . . . . . . . . . . . 12 2.1.2 New prior distribution for true motion vector field . . . . . . . 15 2.2 Estimation of AGN Variance . . . . . . . . . . . . . . . . . . . . . . 23 2.2.1 Proposed covariance matrix estimation method . . . . . . . . 24 iii 2.2.2 Performance of the proposed reliability measure . . . . . . . 31 2.3 Solution to MAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 2.4 Relation to Previous Works . . . . . . . . . . . . . . . . . . . . . . . 34 2.5 Properties of Proposed Prior Distribution . . . . . . . . . . . . . . . . 36 3 Multiple Hypotheses Bayesian Approach 38 3.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . 38 3.1.1 Proposed observation likelihood . . . . . . . . . . . . . . . . 39 3.1.2 Prior distribution of true motion vector field . . . . . . . . . . 43 3.2 MAP Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 3.3 Adaptive Adjustment of Estimated Noise Variances . . . . . . . . . . 45 4 Single Hypothesis Bayesian Approach in a Bidirectional Framework 50 4.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . 50 4.1.1 Observation likelihood . . . . . . . . . . . . . . . . . . . . . 51 4.1.2 Joint prior distribution of true motion vector fields . . . . . . 52 4.2 MAP Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 5 Experimental Results 56 5.1 Experimental Settings . . . . . . . . . . . . . . . . . . . . . . . . . . 56 5.2 Performance Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 60 5.2.1 Performance of SHBA . . . . . . . . . . . . . . . . . . . . . 60 5.2.2 Performance of MHBA . . . . . . . . . . . . . . . . . . . . . 71 5.2.3 Performance of SHBA-BF . . . . . . . . . . . . . . . . . . . 72 6 Conclusion 89 Abstract In Korean 98Docto

Reconfiguration stéréoscopique

Au cours des dernières années, le cinéma tridimensionnel a connu un regain de popularité. La réalisation de plusieurs films d'animation 3D de qualité, de même que le succès fulgurant du film Avatar aura permis au grand public de constater la qualité de cette nouvelle génération de technologies 3D. Cependant, un problème fondamental ralentit toujours l'adoption à la maison de ce mode de divertissement. En effet, tout contenu visuel produit en se basant sur des techniques de stéréoscopie subira des distorsions visuelles lorsqu'observé dans des conditions différentes de celles considérées lors de la création du contenu. Autrement dit, un film 3D tourné pour un cinéma de grande dimension n'aura pas une richesse de profondeur aussi grande lorsqu'il sera visualisé sur un écran domestique. Ce mémoire présente un cadre de travail, un modèle mathématique et un ensemble de techniques permettant de"reconfigurer", en générant de nouvelles images, le contenu stéréoscopique original afin que l'effet de profondeur original soit préservé dans les nouvelles conditions de visualisation

INTERMEDIATE VIEW RECONSTRUCTION FOR MULTISCOPIC 3D DISPLAY

This thesis focuses on Intermediate View Reconstruction (IVR) which generates additional images from the available stereo images. The main application of IVR is to generate the content of multiscopic 3D displays, and it can be applied to generate different viewpoints to Free-viewpoint TV (FTV). Although IVR is considered a good approach to generate additional images, there are some problems with the reconstruction process, such as detecting and handling the occlusion areas, preserving the discontinuity at edges, and reducing image artifices through formation of the texture of the intermediate image. The occlusion area is defined as the visibility of such an area in one image and its disappearance in the other one. Solving IVR problems is considered a significant challenge for researchers. In this thesis, several novel algorithms have been specifically designed to solve IVR challenges by employing them in a highly robust intermediate view reconstruction algorithm. Computer simulation and experimental results confirm the importance of occluded areas in IVR. Therefore, we propose a novel occlusion detection algorithm and another novel algorithm to Inpaint those areas. Then, these proposed algorithms are employed in a novel occlusion-aware intermediate view reconstruction that finds an intermediate image with a given disparity between two input images. This novelty is addressed by adding occlusion awareness to the reconstruction algorithm and proposing three quality improvement techniques to reduce image artifices: filling the re-sampling holes, removing ghost contours, and handling the disocclusion area. We compared the proposed algorithms to the previously well-known algorithms on each field qualitatively and quantitatively. The obtained results show that our algorithms are superior to the previous well-known algorithms. The performance of the proposed reconstruction algorithm is tested under 13 real images and 13 synthetic images. Moreover, analysis of a human-trial experiment conducted with 21 participants confirmed that the reconstructed images from our proposed algorithm have very high quality compared with the reconstructed images from the other existing algorithms