Incremental Model-Based Estimation Using Geometric Consistency Constraints

W e present a physics-based deformable model framework for the incremental object shape estimation and tracking in image sequences. The model is estimated by an optimization process that relates image-based cost functions to model motion via the Lagrangian dynamics equations. Although previous approaches have investigated various combinations of cues in the context of deformable model shape and motion estimation, they generally assume a fixed, known, model parameterization, along with a single model discretization in terms of points. Our technique for object shape estimation and tracking is based on the incremental fusing of point information and new line information. Assuming that a deformable model has been initialized to fit part of a complex object (e.g., a bicycle) new line features belonging to the object but excluded from the initial model parameterization are identified during tracking. The identification is based on a set of novel model-based geometric consistency checks relating separate, independent tracking processes at the image feature and model levels, respectively. Identified features are reconstructed and integrated into the model parameterization which results in more accurate object shape estimation and subsequently they are used to increase model coverage in the image, thereby increasing tracking robustness. We derive the forward transfer under the action of the Euclidean group and Jacobian matrices for the underlying line feature mapping. New correspondi- ng image alignment and generalized forces are introduced as soft constraints and combined with the forces derived from model contours to incrementally improve its motion estimation. We demonstrate our approach on image sequences with complex object shape and motion

Visual localisation of electricity pylons for power line inspection

Inspection of power infrastructure is a regular maintenance event. To date the inspection process has mostly been done manually, but there is growing interest in automating the process. The automation of the inspection process will require an accurate means for the localisation of the power infrastructure components. In this research, we studied the visual localisation of a pylon. The pylon is the most prominent component of the power infrastructure and can provide a context for the inspection of the other components. Point-based descriptors tend to perform poorly on texture less objects such as pylons, therefore we explored the localisation using convolutional neural networks and geometric constraints. The crossings of the pylon, or vertices, are salient points on the pylon. These vertices aid with recognition and pose estimation of the pylon. We were successfully able to use a convolutional neural network for the detection of the vertices. A model-based technique, geometric hashing, was used to establish the correspondence between the stored pylon model and the scene object. We showed the effectiveness of the method as a voting technique to determine the pose estimation from a single image. In a localisation framework, the method serves as the initialization of the tracking process. We were able to incorporate an extended Kalman filter for subsequent incremental tracking of the camera relative to the pylon. Also, we demonstrated an alternative tracking using heatmap details from the vertex detection. We successfully demonstrated the proposed algorithms and evaluated their effectiveness using a model pylon we built in the laboratory. Furthermore, we revalidated the results on a real-world outdoor electricity pylon. Our experiments illustrate that model-based techniques can be deployed as part of the navigation aspect of a robot

Lifting GIS Maps into Strong Geometric Context for Scene Understanding

Contextual information can have a substantial impact on the performance of visual tasks such as semantic segmentation, object detection, and geometric estimation. Data stored in Geographic Information Systems (GIS) offers a rich source of contextual information that has been largely untapped by computer vision. We propose to leverage such information for scene understanding by combining GIS resources with large sets of unorganized photographs using Structure from Motion (SfM) techniques. We present a pipeline to quickly generate strong 3D geometric priors from 2D GIS data using SfM models aligned with minimal user input. Given an image resectioned against this model, we generate robust predictions of depth, surface normals, and semantic labels. We show that the precision of the predicted geometry is substantially more accurate other single-image depth estimation methods. We then demonstrate the utility of these contextual constraints for re-scoring pedestrian detections, and use these GIS contextual features alongside object detection score maps to improve a CRF-based semantic segmentation framework, boosting accuracy over baseline models

Reliability in Constrained Gauss-Markov Models: An Analytical and Differential Approach with Applications in Photogrammetry

This report was prepared by Jackson Cothren, a graduate research associate in the Department of Civil and Environmental Engineering and Geodetic Science at the Ohio State University, under the supervision of Professor Burkhard Schaffrin.This report was also submitted to the Graduate School of the Ohio State University as a dissertation in partial fulfillment of the requirements for the Ph.D. degree.Reliability analysis explains the contribution of each observation in an estimation model to the overall redundancy of the model, taking into account the geometry of the network as well as the precision of the observations themselves. It is principally used to design networks resistant to outliers in the observations by making the outliers more detectible using standard statistical tests.It has been studied extensively, and principally, in Gauss- Markov models. We show how the same analysis may be extended to various constrained Gauss-Markov models and present preliminary work for its use in unconstrained Gauss-Helmert models. In particular, we analyze the prominent reliability matrix of the constrained model to separate the contribution of the constraints to the redundancy of the observations from the observations themselves. In addition, we make extensive use of matrix differential calculus to find the Jacobian of the reliability matrix with respect to the parameters that define the network through both the original design and constraint matrices. The resulting Jacobian matrix reveals the sensitivity of reliability matrix elements highlighting weak areas in the network where changes in observations may result in unreliable observations. We apply the analytical framework to photogrammetric networks in which exterior orientation parameters are directly observed by GPS/INS systems. Tie-point observations provide some redundancy and even a few collinear tie-point and tie-point distance constraints improve the reliability of these direct observations by as much as 33%. Using the same theory we compare networks in which tie-points are observed on multiple images (n-fold points) and tie-points are observed in photo pairs only (two-fold points). Apparently, the use of two-fold tiepoints does not significantly degrade the reliability of the direct exterior observation observations. Coplanarity constraints added to the common two-fold points do not add significantly to the reliability of the direct exterior orientation observations. The differential calculus results may also be used to provide a new measure of redundancy number stability in networks. We show that a typical photogrammetric network with n-fold tie-points was less stable with respect to at least some tie-point movement than an equivalent network with n-fold tie-points decomposed into many two-fold tie-points

Reliability-based design optimization of shells with uncertain geometry using adaptive Kriging metamodels

Optimal design under uncertainty has gained much attention in the past ten years due to the ever increasing need for manufacturers to build robust systems at the lowest cost. Reliability-based design optimization (RBDO) allows the analyst to minimize some cost function while ensuring some minimal performances cast as admissible failure probabilities for a set of performance functions. In order to address real-world engineering problems in which the performance is assessed through computational models (e.g., finite element models in structural mechanics) metamodeling techniques have been developed in the past decade. This paper introduces adaptive Kriging surrogate models to solve the RBDO problem. The latter is cast in an augmented space that "sums up" the range of the design space and the aleatory uncertainty in the design parameters and the environmental conditions. The surrogate model is used (i) for evaluating robust estimates of the failure probabilities (and for enhancing the computational experimental design by adaptive sampling) in order to achieve the requested accuracy and (ii) for applying a gradient-based optimization algorithm to get optimal values of the design parameters. The approach is applied to the optimal design of ring-stiffened cylindrical shells used in submarine engineering under uncertain geometric imperfections. For this application the performance of the structure is related to buckling which is addressed here by means of a finite element solution based on the asymptotic numerical method

RGBDTAM: A Cost-Effective and Accurate RGB-D Tracking and Mapping System

Simultaneous Localization and Mapping using RGB-D cameras has been a fertile research topic in the latest decade, due to the suitability of such sensors for indoor robotics. In this paper we propose a direct RGB-D SLAM algorithm with state-of-the-art accuracy and robustness at a los cost. Our experiments in the RGB-D TUM dataset [34] effectively show a better accuracy and robustness in CPU real time than direct RGB-D SLAM systems that make use of the GPU. The key ingredients of our approach are mainly two. Firstly, the combination of a semi-dense photometric and dense geometric error for the pose tracking (see Figure 1), which we demonstrate to be the most accurate alternative. And secondly, a model of the multi-view constraints and their errors in the mapping and tracking threads, which adds extra information over other approaches. We release the open-source implementation of our approach 1 . The reader is referred to a video with our results 2 for a more illustrative visualization of its performance

Efficient Constellation-Based Map-Merging for Semantic SLAM

Data association in SLAM is fundamentally challenging, and handling ambiguity well is crucial to achieve robust operation in real-world environments. When ambiguous measurements arise, conservatism often mandates that the measurement is discarded or a new landmark is initialized rather than risking an incorrect association. To address the inevitable `duplicate' landmarks that arise, we present an efficient map-merging framework to detect duplicate constellations of landmarks, providing a high-confidence loop-closure mechanism well-suited for object-level SLAM. This approach uses an incrementally-computable approximation of landmark uncertainty that only depends on local information in the SLAM graph, avoiding expensive recovery of the full system covariance matrix. This enables a search based on geometric consistency (GC) (rather than full joint compatibility (JC)) that inexpensively reduces the search space to a handful of `best' hypotheses. Furthermore, we reformulate the commonly-used interpretation tree to allow for more efficient integration of clique-based pairwise compatibility, accelerating the branch-and-bound max-cardinality search. Our method is demonstrated to match the performance of full JC methods at significantly-reduced computational cost, facilitating robust object-based loop-closure over large SLAM problems.Comment: Accepted to IEEE International Conference on Robotics and Automation (ICRA) 201

Private Incremental Regression

Data is continuously generated by modern data sources, and a recent challenge in machine learning has been to develop techniques that perform well in an incremental (streaming) setting. In this paper, we investigate the problem of private machine learning, where as common in practice, the data is not given at once, but rather arrives incrementally over time. We introduce the problems of private incremental ERM and private incremental regression where the general goal is to always maintain a good empirical risk minimizer for the history observed under differential privacy. Our first contribution is a generic transformation of private batch ERM mechanisms into private incremental ERM mechanisms, based on a simple idea of invoking the private batch ERM procedure at some regular time intervals. We take this construction as a baseline for comparison. We then provide two mechanisms for the private incremental regression problem. Our first mechanism is based on privately constructing a noisy incremental gradient function, which is then used in a modified projected gradient procedure at every timestep. This mechanism has an excess empirical risk of $\approx\sqrt{d}$, where $d$ is the dimensionality of the data. While from the results of [Bassily et al. 2014] this bound is tight in the worst-case, we show that certain geometric properties of the input and constraint set can be used to derive significantly better results for certain interesting regression problems.Comment: To appear in PODS 201