WarpNet: Weakly Supervised Matching for Single-view Reconstruction

We present an approach to matching images of objects in fine-grained datasets without using part annotations, with an application to the challenging problem of weakly supervised single-view reconstruction. This is in contrast to prior works that require part annotations, since matching objects across class and pose variations is challenging with appearance features alone. We overcome this challenge through a novel deep learning architecture, WarpNet, that aligns an object in one image with a different object in another. We exploit the structure of the fine-grained dataset to create artificial data for training this network in an unsupervised-discriminative learning approach. The output of the network acts as a spatial prior that allows generalization at test time to match real images across variations in appearance, viewpoint and articulation. On the CUB-200-2011 dataset of bird categories, we improve the AP over an appearance-only network by 13.6%. We further demonstrate that our WarpNet matches, together with the structure of fine-grained datasets, allow single-view reconstructions with quality comparable to using annotated point correspondences.Comment: to appear in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 201

Visual Object Tracking in First Person Vision

The understanding of human-object interactions is fundamental in First Person Vision (FPV). Visual tracking algorithms which follow the objects manipulated by the camera wearer can provide useful information to effectively model such interactions. In the last years, the computer vision community has significantly improved the performance of tracking algorithms for a large variety of target objects and scenarios. Despite a few previous attempts to exploit trackers in the FPV domain, a methodical analysis of the performance of state-of-the-art trackers is still missing. This research gap raises the question of whether current solutions can be used “off-the-shelf” or more domain-specific investigations should be carried out. This paper aims to provide answers to such questions. We present the first systematic investigation of single object tracking in FPV. Our study extensively analyses the performance of 42 algorithms including generic object trackers and baseline FPV-specific trackers. The analysis is carried out by focusing on different aspects of the FPV setting, introducing new performance measures, and in relation to FPV-specific tasks. The study is made possible through the introduction of TREK-150, a novel benchmark dataset composed of 150 densely annotated video sequences. Our results show that object tracking in FPV poses new challenges to current visual trackers. We highlight the factors causing such behavior and point out possible research directions. Despite their difficulties, we prove that trackers bring benefits to FPV downstream tasks requiring short-term object tracking. We expect that generic object tracking will gain popularity in FPV as new and FPV-specific methodologies are investigated

Construction of boundary element models in bioelectromagnetism

Multisensor electro- and magnetoencephalographic (EEG and MEG) as well as electro- and magnetocardiographic (ECG and MCG) recordings have been proved useful in noninvasively extracting information on bioelectric excitation. The anatomy of the patient needs to be taken into account, when excitation sites are localized by solving the inverse problem. In this work, a methodology has been developed to construct patient specific boundary element models for bioelectromagnetic inverse problems from magnetic resonance (MR) data volumes as well as from two orthogonal X-ray projections. The process consists of three main steps: reconstruction of 3-D geometry, triangulation of reconstructed geometry, and registration of the model with a bioelectromagnetic measurement system. The 3-D geometry is reconstructed from MR data by matching a 3-D deformable boundary element template to images. The deformation is accomplished as an energy minimization process consisting of image and model based terms. The robustness of the matching is improved by multi-resolution and global-to-local approaches as well as using oriented distance maps. A boundary element template is also used when 3-D geometry is reconstructed from X-ray projections. The deformation is first accomplished in 2-D for the contours of simulated, built from the template, and real X-ray projections. The produced 2-D vector field is back-projected and interpolated on the 3-D template surface. A marching cube triangulation is computed for the reconstructed 3-D geometry. Thereafter, a non-iterative mesh-simplification method is applied. The method is based on the Voronoi-Delaunay duality on a 3-D surface with discrete distance measures. Finally, the triangulated surfaces are registered with a bioelectromagnetic measurement utilizing markers. More than fifty boundary element models have been successfully constructed from MR images using the methods developed in this work. A simulation demonstrated the feasibility of X-ray reconstruction; some practical problems of X-ray imaging need to be solved to begin tests with real data.reviewe