Vision-Based Navigation III: Pose and Motion from Omnidirectional Optical Flow and a Digital Terrain Map

An algorithm for pose and motion estimation using corresponding features in omnidirectional images and a digital terrain map is proposed. In previous paper, such algorithm for regular camera was considered. Using a Digital Terrain (or Digital Elevation) Map (DTM/DEM) as a global reference enables recovering the absolute position and orientation of the camera. In order to do this, the DTM is used to formulate a constraint between corresponding features in two consecutive frames. In this paper, these constraints are extended to handle non-central projection, as is the case with many omnidirectional systems. The utilization of omnidirectional data is shown to improve the robustness and accuracy of the navigation algorithm. The feasibility of this algorithm is established through lab experimentation with two kinds of omnidirectional acquisition systems. The first one is polydioptric cameras while the second is catadioptric camera.Comment: 6 pages, 9 figure

Inpainting methods for optical flow

Current methods for computing optical flow are based on a four-step pipeline. The goal of the first step is finding point correspondences between two consecutive images. The aim of the second step is filtering problematic or even false correspondences. The purpose of the third step-inpainting, is filling in the missing information from the neighborhood. The final step refines the obtained dense flow field using a variational approach. Up to now, there was little research that deals with the inpainting step and no work if a variational approach could improve the inpainting step. A common technique for the final step of the optical flow pipeline is minimizing an energy functional. In contrast, this thesis uses the minimization of an energy function for the inpainting step, which is also, the focus of this thesis. The inpainting energy functional consists of a similarity term and a smoothness term. For the smoothness term several possible extensions are proposed, that incorporate image information and enable an anisotropic smoothing behavior. Finally, all extensions are compared with each other and with the results from EpicFlow (Revaud et al., 2015)

Computer Vision-Based Hand Tracking and 3D Reconstruction as a Human-Computer Input Modality with Clinical Application

The recent pandemic has impeded patients with hand injuries from connecting in person with their therapists. To address this challenge and improve hand telerehabilitation, we propose two computer vision-based technologies, photogrammetry and augmented reality as alternative and affordable solutions for visualization and remote monitoring of hand trauma without costly equipment. In this thesis, we extend the application of 3D rendering and virtual reality-based user interface to hand therapy. We compare the performance of four popular photogrammetry software in reconstructing a 3D model of a synthetic human hand from videos captured through a smartphone. The visual quality, reconstruction time and geometric accuracy of output model meshes are compared. Reality Capture produces the best result, with output mesh having the least error of 1mm and a total reconstruction time of 15 minutes. We developed an augmented reality app using MediaPipe algorithms that extract hand key points, finger joint coordinates and angles in real-time from hand images or live stream media. We conducted a study to investigate its input variability and validity as a reliable tool for remote assessment of finger range of motion. The intraclass correlation coefficient between DIGITS and in-person measurement obtained is 0.767- 0.81 for finger extension and 0.958–0.857 for finger flexion. Finally, we develop and surveyed the usability of a mobile application that collects patient data medical history, self-reported pain levels and hand 3D models and transfer them to therapists. These technologies can improve hand telerehabilitation, aid clinicians in monitoring hand conditions remotely and make decisions on appropriate therapy, medication, and hand orthoses

Recursive Estimation of Structure and Motion from Monocular Images

The determination of the 3D motion of a camera and the 3D structure of the scene in which the camera is moving, known as the Structure from Motion (SFM) problem, is a central problem in computer vision. Specifically, the recursive (online) estimation is of major interest for robotics applications such as navigation and mapping. Many problems still hinder the deployment of SFM in real-life applications namely, (1) the robustness to noise, outliers and ambiguous motions, (2) the numerical tractability with a large number of features and (3) the cases of rapidly varying camera velocities. Towards solving those problems, this research presents the following four contributions that can be used individually, together, or combined with other approaches. A motion-only filter is devised by capitalizing on algebraic threading constraints. This filter efficiently integrates information over multiple frames achieving a performance comparable to the best state of the art filters. However, unlike other filter based approaches, it is not affected by large baselines (displacement between camera centers). An approach is introduced to incorporate, with only a small computational overhead, a large number of frame-to-frame features (i.e., features that are matched only in pairs of consecutive frames) in any analytic filter. The computational overhead grows linearly with the number of added frame-to-frame features and the experimental results show an increased accuracy and consistency. A novel filtering approach scalable to accommodate a large number of features is proposed. This approach achieves both the scalability of the state of the art filter in scalability and the accuracy of the state of the art filter in accuracy. A solution to the problem of prediction over large baselines in monocular Bayesian filters is presented. This problem is due to the fact that a simple prediction, using constant velocity models for example, is not suitable for large baselines, and the projections of the 3D points that are in the state vector can not be used in the prediction due to the need of preserving the statistical independence of the prediction and update steps

Recursive Estimation of Time-Varying Motion and Structure Parameters

We present a computational framework for recovering both 1 st -order motion parameters (observer direction of translation and observer rotation), 2 nd -order motion parameters (observer rotational acceleration) and relative depth maps from time-varying optical flow. We recover translation speed and acceleration in units which are scaled relative to the distance to the object. Our assumption is that the observer rotational motion is no more than "second order"; in other words, observer motion is either constant or has at most constant acceleration. We examine the effect of noise on the solution of the motion and structure parameters. This ensemble of unknowns comprises a solution to the classical `structure-and-motion from optic flow' problem. Our complete framework utilizes a method for interpreting the bilinear image velocity equation by solving simple systems of linear equations. Since our noise analysis yields uncertainty measures for each parameter, a Kalman filter is employed..