MLPnP - A Real-Time Maximum Likelihood Solution to the Perspective-n-Point Problem

In this paper, a statistically optimal solution to the Perspective-n-Point (PnP) problem is presented. Many solutions to the PnP problem are geometrically optimal, but do not consider the uncertainties of the observations. In addition, it would be desirable to have an internal estimation of the accuracy of the estimated rotation and translation parameters of the camera pose. Thus, we propose a novel maximum likelihood solution to the PnP problem, that incorporates image observation uncertainties and remains real-time capable at the same time. Further, the presented method is general, as is works with 3D direction vectors instead of 2D image points and is thus able to cope with arbitrary central camera models. This is achieved by projecting (and thus reducing) the covariance matrices of the observations to the corresponding vector tangent space.Comment: Submitted to the ISPRS congress (2016) in Prague. Oral Presentation. Published in ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., III-3, 131-13

Accurate and linear time pose estimation from points and lines

The final publication is available at link.springer.comThe Perspective-n-Point (PnP) problem seeks to estimate the pose of a calibrated camera from n 3Dto-2D point correspondences. There are situations, though, where PnP solutions are prone to fail because feature point correspondences cannot be reliably estimated (e.g. scenes with repetitive patterns or with low texture). In such scenarios, one can still exploit alternative geometric entities, such as lines, yielding the so-called Perspective-n-Line (PnL) algorithms. Unfortunately, existing PnL solutions are not as accurate and efficient as their point-based counterparts. In this paper we propose a novel approach to introduce 3D-to-2D line correspondences into a PnP formulation, allowing to simultaneously process points and lines. For this purpose we introduce an algebraic line error that can be formulated as linear constraints on the line endpoints, even when these are not directly observable. These constraints can then be naturally integrated within the linear formulations of two state-of-the-art point-based algorithms, the OPnP and the EPnP, allowing them to indistinctly handle points, lines, or a combination of them. Exhaustive experiments show that the proposed formulation brings remarkable boost in performance compared to only point or only line based solutions, with a negligible computational overhead compared to the original OPnP and EPnP.Peer ReviewedPostprint (author's final draft

Single Frame Resection of Compact Digital Cameras for UAV Imagery

Recently, UAVs (Unmanned Aerial Vehicles) gaina wider acceptance from many disciplines. One major applicationis for monitoring and mapping. Flying beyond eye sightautonomously and collecting data over large areas are theirobvious advantages. To support a large scale urban city mapping,we have developed a UAV system which can carry a compactdigital camera as well as a navigational grade of a GlobalPositioning System (GPS) board mounted on the vehicle.Unfortunately, such a navigational system fails to providesufficient accuracy required to process images become a largescale map. Ubiquitous digital compact cameras, despite their lowcost benefits, are widely known to suffer instabilities in theirinternal lenses and electronics imaging system. Hence thesecameras are less suitable for mapping related purposes. However,this paper presents a photogrammetric technique to preciselydetermine intrinsic and extrinsic camera parameters ofphotographed images provided that sufficient numbers ofsurveyed control points are available. A rigorous Mathematicalmodel is derived to compute each image position with respect tothe imaging coordinate system as well as a location of theprincipal point of an image sensor and the focal length of thecamera. An iterative Gaussian-Newton least squares adjustmentmethod is utilized to compute those parameters. Finally, surveyeddata are processed and elaborated to justify the mathematicalmodels

Camera pose estimation in unknown environments using a sequence of wide-baseline monocular images

In this paper, a feature-based technique for the camera pose estimation in a sequence of wide-baseline images has been proposed. Camera pose estimation is an important issue in many computer vision and robotics applications, such as, augmented reality and visual SLAM. The proposed method can track captured images taken by hand-held camera in room-sized workspaces with maximum scene depth of 3-4 meters. The system can be used in unknown environments with no additional information available from the outside world except in the first two images that are used for initialization. Pose estimation is performed using only natural feature points extracted and matched in successive images. In wide-baseline images unlike consecutive frames of a video stream, displacement of the feature points in consecutive images is notable and hence cannot be traced easily using patch-based methods. To handle this problem, a hybrid strategy is employed to obtain accurate feature correspondences. In this strategy, first initial feature correspondences are found using similarity of their descriptors and then outlier matchings are removed by applying RANSAC algorithm. Further, to provide a set of required feature matchings a mechanism based on sidelong result of robust estimator was employed. The proposed method is applied on indoor real data with images in VGA quality (640×480 pixels) and on average the translation error of camera pose is less than 2 cm which indicates the effectiveness and accuracy of the proposed approach

Single Frame Resection of Compact Digital Cameras for UAV Imagery

Recently, UAVs (Unmanned Aerial Vehicles) gain a wider acceptance from many disciplines. One major application is for monitoring and mapping. Flying beyond eye sight autonomously and collecting data over large areas are their obvious advantages. To support a large scale urban city mapping, we have developed a UAV system which can carry a compact digital camera as well as a navigational grade of a Global Positioning System (GPS) board mounted on the vehicle. Unfortunately, such a navigational system fails to provide sufficient accuracy required to process images become a large scale map. Ubiquitous digital compact cameras, despite their low cost benefits, are widely known to suffer instabilities in their internal lenses and electronics imaging system. Hence these cameras are less suitable for mapping related purposes. However, this paper presents a photogrammetric technique to precisely determine intrinsic and extrinsic camera parameters of photographed images provided that sufficient numbers of surveyed control points are available. A rigorous Mathematical model is derived to compute each image position with respect to the imaging coordinate system as well as a location of the principal point of an image sensor and the focal length of the camera. An iterative Gaussian-Newton least squares adjustment method is utilized to compute those parameters. Finally, surveyed data are processed and elaborated to justify the mathematical models

Dimensional Measurement of Objects in Single Images Independent from Restrictive Camera Parameters

Recent advances in microelectronics have produced new generations of digital cameras with variable focal lengths and pixel sizes which facilitate automatic and high-quality imaging. However, without knowing the values of these critical camera parameters, it is difficult to measure objects in images using existing algorithms. This work investigates this important problem aiming at dimensional measurements (e.g., diameter, length, width and height) of regularly shaped physical objects in a single 2-D image free from restrictive camera parameters. Traditionally, such measurements usually require determinations of the poses of a certain reference feature, i.e., the location and orientation of the feature relative to the camera, in order to establish a geometric model for the dimensional calculation. Points or lines associated with certain shapes (including triangles and rectangles) are often used as reference features for the pose estimation. However, with only a single image as the input, these methods assume the availability of 3-D spatial relationships of the points or lines, which limits the applications of these methods to practical problems where this knowledge is unavailable or difficult to estimate, such as in the problem of image-based food portion size estimation in dietary assessment. In addition to points and lines, the circle has also been used as a reference feature because it has a single elliptic perspective projection in images. However, almost all the existing approaches treat the parameters of focal length and pixel size as the necessary prior information. Here, we propose a new approach to dimensional estimation based on single image input using the circular reference feature and a pin-hole model without considering camera distortion. Without knowing the focal length and pixel size, our approach provides a closed-form solution for the orientation estimation of the circular feature. With additional information provided, such as the size of the circular reference feature, analytical solutions are provided for physical length estimation between an arbitrary pair of points on the reference plane. Studies using both synthetic and actual objects have been conducted to evaluate this new method, which exhibited satisfactory results. This method has also been applied to the measurement of food dimensions based on digital pictures of foods in circular dining plates

A nonlinear observer for 6 DOF pose estimation from inertial and bearing measurements

Abstract — This paper considers the problem of estimating pose from inertial and bearing-only vision measurements. We present a non-linear observer that evolves directly on the special Euclidean group SE(3) from inertial measurements and bearing measurements, such as provided by a visual system tracking known landmarks. Local asymptotic convergence of the observer is proved. The observer is computationally simple and its gains are easy to tune. Simulation results demonstrate robustness to measurement noise and initial conditions

3D Monitor Based on Head Pose Detection

S vývojem možností zpracování obrazu, stereoskopického zobrazení, cen webových kamer a výkonu počítačů vyvstala možnost jak znásobit zážitek uživatele během práce s 3D programy. Z obrazu z webové kamery lze odhadnout polohu uživatelovy hlavy a podle této polohy natočit trojrozměrnou scénu zobrazovanou na monitoru počítače. Uživateli se potom při pohybu hlavy bude zdát, jako by byl monitor okno, skrze které může nahlížet na scénu za ním. Pomocí systému, který je výsledkem této práce, bude možné jednoduše a levně dodat uvedené chování libovolnému 3D programu.With the development of posibilities of image processing, stereoscopy, prices of web cameras and power of computers an opportunity to multiply an experience with working with 3D programs showed. From the picture from webcamera an estimation of a pose of user's head can be made. According to this pose a view on 3D scene can be changed. Then, when user moves his head, he will have a feeling as if monitor was a window through which one can see the scene behind. With the system which is the result of this project it will be possible to easily and cheaply add this kind of behaviour to any 3D application.