Digital Signal Processing

Contains an introduction and reports on fifteen research projects.National Science Foundation FellowshipU.S. Navy - Office of Naval Research (Contract N00014-81-K-0742)National Science Foundation (Grant ECS 84-07285)Sanders Associates, Inc.U.S. Air Force - Office of Scientific Research (Contract F19628-85-K-0028)AT&T Bell Laboratories Doctoral Support ProgramCanada, Bell Northern Research ScholarshipCanada, Fonds pour la Formation de Chercheurs et /'Aide a la Recherche Postgraduate FellowshipCanada, Natural Science and Engineering Research Council Postgraduate FellowshipAmoco Foundation FellowshipFannie and John Hertz Foundation Fellowshi

Advanced editing methods for image and video sequences

In the context of image and video editing, this thesis proposes methods for modifying the semantic content of a recorded scene. Two different editing problems are approached: First, the removal of ghosting artifacts from high dynamic range (HDR) images recovered from exposure sequences, and second, the removal of objects from video sequences recorded with and without camera motion. These editings need to be performed in a way that the result looks plausible to humans, but without having to recover detailed models about the content of the scene, e.g. its geometry, reflectance, or illumination. The proposed editing methods add new key ingredients, such as camera noise models and global optimization frameworks, that help achieving results that surpass the capabilities of state-of-the-art methods. Using these ingredients, each proposed method defines local visual properties that approximate well the specific editing requirements of each task. These properties are then encoded into a energy function that, when globally minimized, produces the required editing results. The optimization of such energy functions corresponds to Bayesian inference problems that are solved efficiently using graph cuts. The proposed methods are demonstrated to outperform other state-ofthe-art methods. Furthermore, they are demonstrated to work well on complex real-world scenarios that have not been previously addressed in the literature, i.e., highly cluttered scenes for HDR deghosting, and highly dynamic scenes and unconstraint camera motion for object removal from videos.Diese Arbeit schlägt Methoden zur Änderung des semantischen Inhalts einer aufgenommenen Szene im Kontext der Bild-und Videobearbeitung vor. Zwei unterschiedliche Bearbeitungsmethoden werden angesprochen: Erstens, das Entfernen von Ghosting Artifacts (Geist-ähnliche Artefakte) aus High Dynamic Range (HDR) Bildern welche von Belichtungsreihen erstellt wurden und zweitens, das Entfernen von Objekten aus Videosequenzen mit und ohne Kamerabewegung. Das Bearbeiten muss in einer Weise durchgeführt werden, dass das Ergebnis für den Menschen plausibel aussieht, aber ohne das detaillierte Modelle des Szeneninhalts rekonstruiert werden müssen, z.B. die Geometrie, das Reflexionsverhalten, oder Beleuchtungseigenschaften. Die vorgeschlagenen Bearbeitungsmethoden beinhalten neuartige Elemente, etwa Kameralärm-Modelle und globale Optimierungs-Systeme, mit deren Hilfe es möglich ist die Eigenschaften der modernsten existierenden Methoden zu übertreffen. Mit Hilfe dieser Elemente definieren die vorgeschlagenen Methoden lokale visuelle Eigenschaften welche die beschriebenen Bearbeitungsmethoden gut annähern. Diese Eigenschaften werden dann als Energiefunktion codiert, welche, nach globalem minimieren, die gewünschten Bearbeitung liefert. Die Optimierung solcher Energiefunktionen entspricht dem Bayes’schen Inferenz Modell welches effizient mittels Graph-Cut Algorithmen gelöst werden kann. Es wird gezeigt, dass die vorgeschlagenen Methoden den heutigen Stand der Technik übertreffen. Darüber hinaus sind sie nachweislich gut auf komplexe natürliche Szenarien anwendbar, welche in der existierenden Literatur bisher noch nicht angegangen wurden, d.h. sehr unübersichtliche Szenen für HDR Deghosting und sehr dynamische Szenen und unbeschränkte Kamerabewegungen für das Entfernen von Objekten aus Videosequenzen

Live HDR video streaming on commodity hardware

High Dynamic Range (HDR) video provides a step change in viewing experience, for example the ability to clearly see the soccer ball when it is kicked from the shadow of the stadium into sunshine. To achieve the full potential of HDR video, so-called true HDR, it is crucial that all the dynamic range that was captured is delivered to the display device and tone mapping is confined only to the display. Furthermore, to ensure widespread uptake of HDR imaging, it should be low cost and available on commodity hardware. This paper describes an end-to-end HDR pipeline for capturing, encoding and streaming high-definition HDR video in real-time using off-the-shelf components. All the lighting that is captured by HDR-enabled consumer cameras is delivered via the pipeline to any display, including HDR displays and even mobile devices with minimum latency. The system thus provides an integrated HDR video pipeline that includes everything from capture to post-production, archival and storage, compression, transmission, and display

HDR video past, present and future : a perspective

High Dynamic Range (HDR) video has emerged from research labs around the world and entered the realm of consumer electronics. The dynamic range that a human can see in a scene with minimal eye adaption (approximately 1,000,000: 1) is vastly greater than traditional imaging technology which can only capture about 8 f-stops (256: 1). HDR technology, on the other hand, has the potential to capture the full range of light in a scene; even more than a human eye can see. This paper examines the field of HDR video from capture to display; past, present and future. In particular the paper looks beyond the current marketing hype around HDR, to show how HDR video in the future can and, indeed, should bring about a step change in imaging, analogous to the change from black and white to colour

Subjective and Objective Evaluation of Tone-Mapping and De-Ghosting Algorithms

With the increasing importance of high dynamic range (HDR) imaging and low availability of HDR displays, HDR cameras the need for efficient tone mapping, De-ghosting techniques is very crucial. However the tone mapping operators, De-Ghosting tend to introduce distortions in the HDR images, thus making it visually unpleasant in normal displays. Subjective evaluation of images is important for rating these algorithms as the users should be able to visualize the complete details present in both the brightly and poorly illuminated regions of the scene. To facilitate a systematic subjective study we have created a database of HDR images tone mapped, De-Ghosted using popular algorithms. We conducted a subjective study of the tone mapped images, computed objective scores by using some of the state-of-the-art no-reference low dynamic range image quality assessment algorithms and evaluated their performance. We show that a moderate and low correlation between objective and subjective scores indicates the need for the consideration of human perception in rating tone mapping operators and De-Ghosting algorithms

Digital Signal Processing

Contains an introduction and reports on twenty research projects.National Science Foundation (Grant ECS 84-07285)U.S. Navy - Office of Naval Research (Contract N00014-81-K-0742)National Science Foundation FellowshipSanders Associates, Inc.U.S. Air Force - Office of Scientific Research (Contract F19628-85-K-0028)Canada, Bell Northern Research ScholarshipCanada, Fonds pour la Formation de Chercheurs et l'Aide a la Recherche Postgraduate FellowshipCanada, Natural Science and Engineering Research Council Postgraduate FellowshipU.S. Navy - Office of Naval Research (Contract N00014-81-K-0472)Fanny and John Hertz Foundation FellowshipCenter for Advanced Television StudiesAmoco Foundation FellowshipU.S. Air Force - Office of Scientific Research (Contract F19628-85-K-0028

New structures and algorithms for adaptive system identification and channel equalization

The main drawback of the ADF is that it takes lot of iteration and fails to identify nonlinear systems. BAF converges fast while maintaining the same performance as ADF but its performance degrades at nonlinear conditions.In this thesis we propose an ANN, which provides better and faster converges when employed for identifying nonlinear systems. This network employs chebyschev based nonlinear inputs updated with the RLS algorithm. Through extensive computer simulation it is demonstrated that CFLANN updated with RLS is a better candidate compared to FLANN and MLP in terms of less complex structure, less number of input simple needed and does accurate identification