On Unusual Pixel Shapes and Image Motion

We introduce the integral-pixel camera model, where measurements integrate over large and potentially overlapping parts of the visual field. This models a wide variety of novel camera designs, including omnidirectional cameras, compressive sensing cameras, and novel programmable-pixel imaging chips. We explore the relationship of integral-pixel measurements with image motion and find (a) that direct motion estimation using integral-pixels is possible and in some cases quite good, (b) standard compressive-sensing reconstructions are not good for estimating motion, and (c) when we design image reconstruction algorithms that explicitly reason about image motion, they outperform standard compressive-sensing video reconstruction. We show experimental results for a variety of simulated cases, and have preliminary results showing a prototype camera with integral-pixels whose design makes direct motion estimation possible

Distortion Estimation Through Explicit Modeling of the Refractive Surface

Precise calibration is a must for high reliance 3D computer vision algorithms. A challenging case is when the camera is behind a protective glass or transparent object: due to refraction, the image is heavily distorted; the pinhole camera model alone can not be used and a distortion correction step is required. By directly modeling the geometry of the refractive media, we build the image generation process by tracing individual light rays from the camera to a target. Comparing the generated images to their distorted - observed - counterparts, we estimate the geometry parameters of the refractive surface via model inversion by employing an RBF neural network. We present an image collection methodology that produces data suited for finding the distortion parameters and test our algorithm on synthetic and real-world data. We analyze the results of the algorithm.Comment: Accepted to ICANN 201

Compact handheld fringe projection based underwater 3D-scanner

A new, fringe projection based compact handheld 3D scanner for the surface reconstruction of measurement objects under water is introduced. The weight of the scanner is about 10 kg and can be used in a water depth of maximal 40 metres. A measurement field of about 250 mm x 200 mm is covered under water, and the lateral resolution of the measured object points is about 150 μm. Larger measurement objects can be digitized in a unique geometric model by merging subsequently recorded datasets. The recording time for one 3D scan is a third of a second. The projection unit for the structured illumination of the scene as well as the computer for device control and measurement data analysis are included into the scanners housing. A display on the backside of the device realizes the graphical presentation of the current measurement data. It allows the user to evaluate the quality of the measurement result in real-time already during the recording of the measurement under water. For the calibration of the underwater scanner a combined method of air- and water-calibration was developed which needs only a few recorded underwater images of a plane surface and an object with known lengths. First measurement results obtained with the new scanner are presented

Underwater 3D measurements with advanced camera modelling

A novel concept of camera modelling for underwater 3D measurements based on stereo camera utilisation is introduced. The geometrical description of the ray course subject to refraction in underwater cameras is presented under assumption of conditions, which are typically satisfied or can be achieved approximately. Possibilities of simplification are shown, which allow an approximation of the ray course by classical pinhole modelling. It is shown how the expected measurement errors can be estimated, as well as its influence on the expected 3D measurement result. Final processing of the 3D measurement data according to the requirements regarding accuracy is performed using several kinds of refinement. For example, calibration parameters can be refined, or systematic errors can be decreased by subsequent compensation by suitable error correction functions. Experimental data of simulations and real measurements obtained by two different underwater 3D scanners are presented and discussed. If inverse image magnification is larger than about one hundred, remaining errors caused by refraction effects can be usually neglected and the classical pinhole model can be used for stereo camera-based underwater 3D measurement systems

Calibration by correlation using metric embedding from non-metric similarities

This paper presents a new intrinsic calibration method that allows us to calibrate a generic single-view point camera just by waving it around. From the video sequence obtained while the camera undergoes random motion, we compute the pairwise time correlation of the luminance signal for a subset of the pixels. We show that, if the camera undergoes a random uniform motion, then the pairwise correlation of any pixels pair is a function of the distance between the pixel directions on the visual sphere. This leads to formalizing calibration as a problem of metric embedding from non-metric measurements: we want to find the disposition of pixels on the visual sphere from similarities that are an unknown function of the distances. This problem is a generalization of multidimensional scaling (MDS) that has so far resisted a comprehensive observability analysis (can we reconstruct a metrically accurate embedding?) and a solid generic solution (how to do so?). We show that the observability depends both on the local geometric properties (curvature) as well as on the global topological properties (connectedness) of the target manifold. We show that, in contrast to the Euclidean case, on the sphere we can recover the scale of the points distribution, therefore obtaining a metrically accurate solution from non-metric measurements. We describe an algorithm that is robust across manifolds and can recover a metrically accurate solution when the metric information is observable. We demonstrate the performance of the algorithm for several cameras (pin-hole, fish-eye, omnidirectional), and we obtain results comparable to calibration using classical methods. Additional synthetic benchmarks show that the algorithm performs as theoretically predicted for all corner cases of the observability analysis

Das "Surface Model" – Eine unsichere kontinuierliche Repräsentation des generischen Kameramodells und dessen Kalibrierung

Using digital cameras for measurement purposes requires the knowledge of the mapping between 3D world points and 2D positions on the image plane. There are many different mathematical models that provide this mapping for a specific imaging system. Grossberg and Nayar proposed a discrete generic camera model, which does not make any assumptions about the structure of this system. The model describes a digital camera by assigning an arbitrary viewing ray to each pixel of the camera image. This makes the model applicable to any kind of camera, especially also to non-central ones like onmidirectional catadioptrics. However, this model is difficult to use in practice, as there is no direct method for mapping a 3D point to the image or determining rays for subpixel image positions. In this work, the Surface Model, an uncertain continuous representation of the generic camera model, will be introduced. It uses a spline surface in 6D Plücker space to describe the camera. The interpolation abilities of the spline surface allow the viewing ray and its uncertainty for any (subpixel) position to be easily determined. Furthermore, the description facilitates the mapping from 3D world points to the image. The calibration of the generic model has to be performed pixel-wise and is technically involved and time-consuming. In this work, hand-held sparse planar chessboard patterns are used for calibration. The uncertainties of the corresponding image point measurements are taken into account and propagated during the complete calibration procedure to obtain an uncertain model. Simulations prove the validity of each step and the practical applicability of the procedure is shown by calibrating several real cameras of different types.Um digitale Kameras zu Vermessungszwecken einzusetzen muss der mathematische Zusammenhang zwischen 3D Weltpunkten und 2D Bildpunkten bekannt sein. Es existiert eine Vielzahl an mathematischen Modellen, welche diese Abbildung für spezifische Kamerasysteme beschreiben. Für deren Gültigkeit ist die Einhaltung der zugehörigen Randbedingungen, beispielsweise die hochgenaue Ausrichtung von Bildsensor, Linsen und Spiegeln, zwingend erforderlich. Andernfalls können fehlerhafte Messergebnisse die Folge sein. Um diese Problematik zu meiden, haben Grossberg und Nayar ein diskretes generisches Kameramodell vorgeschlagen, welches jedem einzelnen Pixel einen separaten Sehstrahl zuordnet. Somit kann jede erdenkliche Kamera beschrieben werden. Dies gilt auch für omnidirektionale catadioptrische Systeme, welche oftmals kein punktförmiges optisches Zentrum besitzen. Jedoch kann weder für jede beliebige Subpixel-Position ein Sehstrahl ermittelt werden, noch ist die Projektion eines beliebigen 3D-Punktes ins Kamerabild ohne weiteres möglich. In dieser Arbeit wird das "Surface Model" vorgestellt. Es dient als eine kontinuierliche Repräsentation des generischen Kameramodells, welche Modellunsicherheiten explizit berücksichtigt. Zur mathematischen Beschreibung wird eine Splineoberfläche im 6D Plücker-Raum genutzt. Deren Interpolationsfähigkeiten erlauben es, für jedwede Subpixel-Position direkt einen Sehstrahl zu ermitteln, sowie einen beliebigen 3D-Punkt ins Kamerabild zu projizieren. Die Kalibrierung des diskreten generischen Modells erfordert mehrere Messungen für jeden einzelnen Pixel. Um diesen aufwändigen Prozess zu vereinfachen, werden in dieser Arbeit von Hand platzierte planare Schachbrettmuster eingesetzt. Während der Kalibrierung treten unweigerlich Messungenauigkeiten auf. Beim hier vorgestellten Verfahren zur Parameterermittlung des Surface Models werden diese Unsicherheiten explizit zur Stabilisierung und Verbesserung der Genauigkeit genutzt. Dies resultiert in einem unsicheren Kameramodell, welches für die Anwendungen der Sehstrahlermittlung und der Punktprojektion Ergebnisunsicherheiten in Form von Kovarianzmatrizen zur Verfügung stellt. Mittels Simulationen wird die Anwendbarkeit sämtlicher vorgestellter Verfahren validiert. Durch die Kalibrierung verschiedener realer Kameras wird darüber hinaus deren praktische Nutzbarkeit aufgezeigt

Design and Development of an Ultraviolet All-Sky Imaging System

T.M. and J.M.-T. were supported by the UK Space Agency projects ST/W00190X/1 and ST/V00610X/1.Peer reviewedPublisher PD