An improved algorithm for deinterlacing video streams

The MPEG-4 standard for computerized video incorporates the concept of a video object pLane While in the simplest case this can be the full rectangular frame, the standard supports a hierarchical set of arbitrary shaped planes, one for each content sensitive video object. Herein is proposed a method for extracting arbitrary planes from video that does not already contain video object plane information; Deinterlacing is the process of taking two video fields, each at half the height of the finalized image frame, and combining them into that finalized frame. As the fields are not captured simultaneously, temporal artifacts may result. Herein is proposed a method to use the above mentioned video object planes to calculate the intra-field motion of objects in the video stream and correct for such motion leading to a higher quality deinterlaced output.*; *This dissertation is a compound document (contains both a paper copy and a CD as part of the dissertation)

Edge-adaptive spatial video de-interlacing algorithms based on fuzzy logic

Since the human visual system is especially sensitive to image edges, edge-dependent spatial interpolators have been proposed in literature as a means of successfully restoring edges while avoiding the staircase effect of linear spatial algorithms. This paper addresses the application of video de-interlacing, which constitutes an indispensable stage in video format conversion. Classic edge-adaptive de-interlacing algorithms introduce annoying artifacts when the edge directions are evaluated incorrectly. This paper presents two ways of exploiting fuzzy reasoning to reinforce edges without an excessive increase in computational complexity. The performance of the proposed algorithms is analyzed by de-interlacing a wide set of test sequences. The study compares the two proposals both with each other and with other edge-adaptive de-interlacing methods reported in the recent literatur

Adaptive deinterlacing of video sequences using motion data

In this work an efficient motion adaptive deinterlacing method with considerable improvement in picture quality is proposed. A temporal deinterlacing method has a high performance in static images while a spatial method has a better performance in dynamic parts. In the proposed deinterlacing method, a motion adaptive interpolator combines the results of a spatial method and a temporal method based on motion activity level of video sequence. A high performance and low complexity algorithm for motion detection is introduced. This algorithm uses five consecutive interlaced video fields for motion detection. It is able to capture a wide range of motions from slow to fast. The algorithm benefits from a hierarchal structure. It starts with detecting motion in large partitions of a given field. Depending on the detected motion activity level for that partition, the motion detection algorithm might recursively be applied to sub-blocks of the original partition. Two different low pass filters are used during the motion detection to increase the algorithm accuracy. The result of motion detection is then used in the proposed motion adaptive interpolator. The performance of the proposed deinterlacing algorithm is compared to previous methods in the literature. Experimenting with several standard video sequences, the method proposed in this work shows excellent results for motion detection and deinterlacing performance

Mobility of Nano-Particles in Rock Based Micro-Models

A confocal micro-particle image velocimetry (C-μPIV) technique along with associated post-processing algorithms is detailed for obtaining three dimensional distributions of nano-particle velocity and concentrations at select locations of the 2.5D (pseudo 3D) Poly(methyl methacrylate) (PMMA) and ceramic micro-model. The designed and fabricated 2.5D micro-model incorporates microchannel networks with 3D wall structures with one at observation wall which resembles fourteen morphological and flow parameters to those of fully 3D actual reservoir rock (Boise Sandstone) at resolutions of 5 and 10 μm in depth and 5 and 25 μm on plane. In addition, an in-situ, non-destructive method for measuring the geometry of low and high resolution PMMA and ceramic micro-models, including its depth, is described and demonstrated. The flow experiments use 860 nm and 300 nm fluorescence-labeled polystyrene particles, and the data is acquired using confocal laser scanning microscopy. Regular fluorescence microscopy is used for the in-situ geometry measurement along with the use of Rhodamine dye and a depth-to-fluorescence-intensity calibration, which is linear. Monochromatic excitation at a wavelength of 544 nm (green) produced by a HeNe continuous wave laser was used to excite the fluorescence-labeled nanoparticles emitting at 612 nm (red). Confocal images were captured by a highly sensitive fluorescence detector photomultiplier tube. Results of detailed three dimensional velocity, particle concentration distributions, and particle deposition rates from experiments conducted at flow rates of 0.5 nL/min, 1 nL/min, 10 nL/min and 100 nL/min are presented and discussed. The three dimensional micro-model geometry reconstructed from fluorescence data is used as the computational domain to conduct numerical simulations of the flow in the as-tested micro-model for comparisons to experimental results using dimensionless Navier-Stokes model. The flow simulation results are also used to qualitatively compare with velocity distributions of the flowing particles at selected locations. The comparison is qualitative because the particle sizes used in these experiments may not accurately follow the flow itself given the geometry of the micro-models. These larger particles were used for proof of concept purposes, and the techniques and algorithms used permit future use of particles as small as 50 nm

Biologically inspired composite image sensor for deep field target tracking

The use of nonuniform image sensors in mobile based computer vision applications can be an effective solution when computational burden is problematic. Nonuniform image sensors are still in their infancy and as such have not been fully investigated for their unique qualities nor have they been extensively applied in practice. In this dissertation a system has been developed that can perform vision tasks in both the far field and the near field. In order to accomplish this, a new and novel image sensor system has been developed. Inspired by the biological aspects of the visual systems found in both falcons and primates, a composite multi-camera sensor was constructed. The sensor provides for expandable visual range, excellent depth of field, and produces a single compact output image based on the log-polar retinal-cortical mapping that occurs in primates. This mapping provides for scale and rotational tolerant processing which, in turn, supports the mitigation of perspective distortion found in strict Cartesian based sensor systems. Furthermore, the scale-tolerant representation of objects moving on trajectories parallel to the sensor\u27s optical axis allows for fast acquisition and tracking of objects moving at high rates of speed. In order to investigate how effective this combination would be for object detection and tracking at both near and far field, the system was tuned for the application of vehicle detection and tracking from a moving platform. Finally, it was shown that the capturing of license plate information in an autonomous fashion could easily be accomplished from the extraction of information contained in the mapped log-polar representation space. The novel composite log-polar deep-field image sensor opens new horizons for computer vision. This current work demonstrates features that can benefit applications beyond the high-speed vehicle tracking for drivers assistance and license plate capture. Some of the future applications envisioned include obstacle detection for high-speed trains, computer assisted aircraft landing, and computer assisted spacecraft docking

Soft computing techniques for video de-interlacing

This paper presents the application of soft computing techniques to video processing. Specially, the research work has been focused on de-interlacing task. It is necessary whenever the transmission standard uses an interlaced format but the receiver requires a progressive scanning, as happens in consumer displays such as LCDs and plasma. A simple hierarchical solution that combines three simple fuzzy logicbased constituents (interpolators) is presented in this paper. Each interpolator specialized in one of three key image features for de-interlacing: motion, edges, and possible repetition of picture areas. The resulting algorithm offers better results than others with less or similar computational cost. A very interesting result is that our algorithm is competitive with motion-compensated algorithm

HDTV transmission format conversion and migration path

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 1997.Includes bibliographical references (leaves 77-79).by Lon E. Sunshine.Ph.D

Conception de processeurs spécialisés pour le traitement vidéo en temps réel par filtre local

RÉSUMÉ Ce mémoire décrit les travaux visant à explorer les possibilités qu'offrent les processeurs à jeu d'instructions spécialisé pour des applications de vidéo numérique. Spécifiquement une classe particulière d'algorithmes de traitement vidéo est considérée: les filtres locaux. Pour cette classe d'algorithmes, une exploration architecturale a permis d'identifier un ensemble de techniques formant une approche cohérente et systématique pour la conception de processeurs spécialisés performants adaptés au traitement vidéo en temps réel. L'approche de conception proposée vise une utilisation efficace de la bande passante vers la mémoire, laquelle bande passante constitue le goulot d'étranglement de l'application du point de vue de la vitesse de traitement. Il est possible d'approcher la performance limite imposée par ce goulot par une stratégie appropriée de réutilisation des données et en exploitant le parallélisme des données inhérent à la classe d'algorithmes visée. L'approche comporte quatre étapes: tout d'abord, une instruction parallèle (SIMD) qui effectue le calcul de plusieurs pixels de sortie à la fois est créée. Puis, des registres à décalage permettant la réutilisation intra-ligne des pixels d'entrée sont ajoutés. Ensuite, un pipeline est créé par le découpage de l'instruction parallèle et l'ajout de registres pour les résultats intermédiaires. Finalement, les instructions spécialisées de chargement et de sauvegarde sont créées. Quelques-unes de ces étapes ouvrent la porte à des simplifications matérielles spécifiques pour certains algorithmes de la classe cible. La structure matérielle obtenue au final, alliée à la parallélisation des instructions par l'utilisation d'une architecture VLIW, se comporte d'une manière semblable à un réseau systolique pipeliné. Afin de démontrer expérimentalement la validité de l'approche de conception proposée, sept processeurs spécialisés pour des algorithmes de la classe visée ont été conçus par extension du jeu d'instructions d'un processeur configurable à jeu d'instructions extensible. Trois de ces processeurs spécialisés mettent en œuvre autant d'algorithmes de désentrelacement intra-trames, et quatre visent plutôt la convolution 2D, différant entre eux par la taille de la fenêtre de convolution. Les résultats de performance obtenus sont prometteurs. Pour les algorithmes de désentrelacement intra-trames, les facteurs d'accélération varient entre 95 et 1330, alors que les facteurs d'amélioration du produit temps-surface varient entre 29 et 243, tout ceci par rapport à un processeur d'usage général de référence roulant une implémentation purement logicielle de l'algorithme.----------ABSTRACT This master thesis explores the possibilities offered by Application-Specific Instruction-Set Processors (ASIP) for digital video applications, more specifically for a particular algorithm class used for video processing: local neighbourhood functions. For this algorithm class, an architectural exploration lead to the identification of a set of design techniques which, together, form a coherent and systematic approach for the design of high performance ASIPs usable for real-time video processing. The proposed design approach aims at an efficient utilization of available bandwidth to memory, which constitutes the main performance bottleneck of the application. It is possible to approach the processing speed limit imposed by this bottleneck through an appropriate data reuse strategy and by exploiting the data parallelism inherent to the target algorithm class. The design approach comprises four steps: first, a Single Instruction Multiple Data (SIMD) instruction which calculates more than one pixel in parallel is created. Then, shift registers, which are used for intra-line input pixel reuse, are added. Next, a processing pipeline is created by the addition of application-specific registers. Finally, the custom load/store instructions are created. Some of these steps lead to possible hardware simplifications for some algorithms of the target class. The hardware structure thus obtained, together with the instruction-level parallelism made possible through the use of a Very Long Instruction Word (VLIW) architecture, mimics a pipelined systolic array. In order to demonstrate the validity of the proposed design approach experimentally, seven ASIPs have been designed by extending the instruction-set of a configurable and extensible processor. Three of the ASIPs implement intra-field deinterlacing algorithms, and four implement the 2D convolution with different kernel sizes. The results show a significant improvement in performance. For the intra-field deinterlacing algorithms, speedup factors are between 95 and 1330, while the factors of improvement of the Area-Time (AT) product are between 29 and 243, all this compared to a pure software implementation running on a general-purpose processor. In the case of the two-dimensional convolution, speedup factors are between 36 and 80, while factors of improvement of the AT product are between 12 and 22. In all cases, real-time processing of high definition video in the 1080i (deinterlacing) or 1080p (convolution) format is possible given a 130 nm manufacturing process