4 research outputs found
Confidence-based cost modulation for stereo matching
We present a novel operator to be applied at raw
matching costs in the context of low level vision tasks
such as stereo matching or optical \ufb02ow. It aims at im-
proving matching reliability by ef\ufb01ciently modulating
pixel-wise pairing costs, injecting a con\ufb01dence backed
bias before the aggregation step. It works analyzing a
noisy estimate of the correspondances in order to fa-
vor or prune potential matches. We test the operator by
developing a local, realtime stereo matching algorithm
and showing that our solution can drastically clean the
resulting depth map while also reducing border bleed-
ing. Its good performance is also evaluated quanti-
tavely by testing the algorithm against the popular Mid-
dlebury benchmark where our local greedy implemen-
tation is able to obtain results comparable to those of
n\ua8 aive global approaches
MRF Stereo Matching with Statistical Estimation of Parameters
For about the last ten years, stereo matching in computer vision has been treated as a combinatorial optimization problem. Assuming that the points in stereo images form a Markov Random Field (MRF), a variety of combinatorial optimization algorithms has been developed to optimize their underlying cost functions. In many of these algorithms, the MRF parameters of the cost functions have often been manually tuned or heuristically determined for achieving good performance results. Recently, several algorithms for statistical, hence, automatic estimation of the parameters have been published. Overall, these algorithms perform well in labeling, but they lack in performance for handling discontinuity in labeling along the surface borders.
In this dissertation, we develop an algorithm for optimization of the cost function with automatic estimation of the MRF parameters – the data and smoothness parameters. Both the parameters are estimated statistically and applied in the cost function with support of adaptive neighborhood defined based on color similarity. With the proposed algorithm, discontinuity handling with higher consistency than of the existing algorithms is achieved along surface borders. The data parameters are pre-estimated from one of the stereo images by applying a hypothesis, called noise equivalence hypothesis, to eliminate interdependency between the estimations of the data and smoothness parameters. The smoothness parameters are estimated applying a combination of maximum likelihood and disparity gradient constraint, to eliminate nested inference for the estimation. The parameters for handling discontinuities in data and smoothness are defined statistically as well. We model cost functions to match the images symmetrically for improved matching performance and also to detect occlusions. Finally, we fill the occlusions in the disparity map by applying several existing and proposed algorithms and show that our best proposed segmentation based least squares algorithm performs better than the existing algorithms.
We conduct experiments with the proposed algorithm on publicly available ground truth test datasets provided by the Middlebury College. Experiments show that results better than the existing algorithms’ are delivered by the proposed algorithm having the MRF parameters estimated automatically. In addition, applying the parameter estimation technique in existing stereo matching algorithm, we observe significant improvement in computational time
Ein Beitrag zur Entwicklung von Methoden zur Stereoanalyse und Bildsynthese im Anwendungskontext der Videokommunikation
This thesis contributes to the research area of stereo vision and view
synthesis in the field of private video communication. During private video
communication eye contact between the participants is typically lost due to
the different placement of the camera and the video window. The goal of
this thesis is to re-establish the eye contact by synthesizing of the view
of a virtual camera such that the virtual camera faces towards the
participant.
The thesis firstly sketches the positive effect of eye contact in video
communication. An in-depth review of mathematical foundations in the fields
of stereo vision and view synthesis follows. On this foundation the thesis
comprehensively covers the state of the art of image based rendering and
particularly of eye-gaze correction via 3D-analysis and synthesis.In the
first step of the method development the thesis establishes a model of
quality factors which determines decisions about camera placement and
recording system. Measurements with respect to synchronization and data
storage are presented. Local and global algorithms for stereo vision are
analyzed and adapted. The thesis contributes to the field of stereo vision
algorithms by means of development and combination of different cost
functions, consistency based inpainting, spatial and temporal smoothing and
segmentation with respect to the use case of private video communication.
Using the extracted disparity map, two approaches for view synthesis -
trifocal transfer and 3D warping - are employed and extended. One important
contribution of the thesis is a contour-based inpainting algorithm as well
as point base image smoothing techniques.
Two comprehensive subjective studies prove the assumption that eye contact
can be re-established by the proposed system. They demonstrate the well
perceived eye-contact as well as the significantly improved acceptance of
quality due to the developed methods compared to the initial situation. The
thesis finally discusses the results, followed by a qualitative comparison
to the state of the art.Die vorliegende Arbeit leistet einen Beitrag zum Forschungsbereich der
Stereoanalyse und Bildsynthese im speziellen Kontext der privaten
Videokommunikation. Bei der privaten Videokommunikation geht durch die
unterschiedliche Positionierung der Kamera und des Videofensters
typischerweise der Blickkontakt zwischen den Kommunikationsteilnehmern
verloren. Ziel dieser Arbeit ist die Wiederherstellung des Blickkontaktes
mittels der Synthese einer virtuellen Kameraansicht, die in Blickrichtung
der Kommunizierenden ausgerichtet ist.
Die Arbeit umreißt zunächst den positiven Einfluss des Blickkontaktes in
der Videokommunikation. Anschließend wird eine tiefgehende Betrachtung der
notwendigen technischen Grundlagen im Bereich Stereoanalyse und
Bildsynthese durchgeführt. Aufbauend auf diesen Grundlagen wird der der
Stand der Technik im Bereich des bildbasierten Renderings im Allgemeinen
sowie der Blickkorrektur mittels 3D-Analyse und -synthese im Speziellen
umfassend behandelt.
Zunächst wird ein Modell von Qualitätsparametern entwickelt, welches die
Entscheidungen hinsichtlich Kameraanordnung und Aufnahmesystem
determiniert. Notwendige Messungen hinsichtlich Synchronizität und
Datenspeicherung werden präsentiert. Im Bereich der Algorithmen der
Stereoanalyse werden etablierte lokale und globale Algorithmen analysiert
und adaptiert. Verschiedene Kostenmaße, konsistenzbasiertes Füllen,
zeitliche und örtliche Glättung sowie eine abschließende Segmentierung
werden hinsichtlich des konkreten Anwendungsfalls der Blickkorrektur in der
privaten Videokommunikation entwickelt. Darauf aufbauend werden die beiden
Syntheseverfahren des trifokalen Transfers sowie des 3D-Warpings weiter
entwickelt. Ein wichtiger Beitrag der Arbeit ist ein konturbasiertes
Füllverfahren sowie Maßnahmen im Bereich der Punktglättung.
Zwei umfangreiche Experimente mit zahlreichen Probanden bestätigen die
Korrektheit der Annahme, dass Blickkontakt durch das vorgestellte Verfahren
hergestellt werden kann. Sie demonstrieren sowohl die sehr gute Wahrnehmung
des Augenkontaktes als auch die signifikante Verbesserung der Akzeptanz und
subjektiven Qualitätswahrnehmung durch die entwickelten Algorithmen im
Vergleich zum Ausgangspunkt der Arbeit. Eine qualitativer Vergleich mit dem
Stand der Technik und eine Diskussion der Ergebnisse, gepaart mit einem
Ausblick in die Zukunft des behandelten Forschungsgebietes, schließen die
Arbeit ab
Stereo Matching with the Distinctive Similarity Measure
International audienceThe point ambiguity owing to the ambiguous local appearances of image points is the one of the main causes making the stereo problem difficult. Under the point ambiguity, local similarity measures are easy to be ambiguous and this results in false matches in ambiguous regions. In this paper, we present the new similarity measure to resolve the point ambiguity problem based on the idea that the distinctiveness, not the interest, is the appropriate criterion for the feature selection under the point ambiguity. The proposed similarity measure named the Distinctive Similarity Measure (DSM) is essentially based on the distinctiveness of image points and the dissimilarity between them, which are both closely related to the local appearances of image points; the distinctiveness of an image point is related to the probability of a mismatch while the dissimilarity is related to the probability of a good match. We verify the efficiency of the proposed DSM by using testbed image sets. Experimental results show that the proposed DSM is very effective and can be easily used for improving the performance of existing stereo methods under the point ambiguity