The Unbalanced Gromov Wasserstein Distance: Conic Formulation and Relaxation

Comparing metric measure spaces (i.e. a metric space endowed with aprobability distribution) is at the heart of many machine learning problems. The most popular distance between such metric measure spaces is theGromov-Wasserstein (GW) distance, which is the solution of a quadratic assignment problem. The GW distance is however limited to the comparison of metric measure spaces endowed with a probability distribution.To alleviate this issue, we introduce two Unbalanced Gromov-Wasserstein formulations: a distance and a more tractable upper-bounding relaxation.They both allow the comparison of metric spaces equipped with arbitrary positive measures up to isometries. The first formulation is a positive and definite divergence based on a relaxation of the mass conservation constraint using a novel type of quadratically-homogeneous divergence. This divergence works hand in hand with the entropic regularization approach which is popular to solve large scale optimal transport problems. We show that the underlying non-convex optimization problem can be efficiently tackled using a highly parallelizable and GPU-friendly iterative scheme. The second formulation is a distance between mm-spaces up to isometries based on a conic lifting. Lastly, we provide numerical experiments onsynthetic examples and domain adaptation data with a Positive-Unlabeled learning task to highlight the salient features of the unbalanced divergence and its potential applications in ML

Multiscale neighborhood-wise decision fusion for redundancy detection in image pairs

SIAM Journal Multiscale Modeling & SimulationTo develop better image change detection algorithms, new models able to capture spatio-temporal regularities and geometries present in an image pair are needed. In this paper, we propose a multiscale formulation for modeling semi-local inter-image interactions and detecting local or regional changes in an image pair. By introducing dissimilarity measures to compare patches and binary local decisions, we design collaborative decision rules that use the total number of detections obtained from the neighboring pixels, for different patch sizes. We study the statistical properties of the non-parametric detection approach that guarantees small probabilities of false alarms. Experimental results on several applications demonstrate that the detection algorithm (with no optical flow computation) performs well at detecting occlusions and meaningful changes for a variety of illumination conditions and signal-to-noise ratios. The number of control parameters of the algorithm is small and the adjustment is intuitive in most cases

Municipal Road Infrastructure Assessment Using Street View Images

Road quality assessment is a crucial part in Municipalities' work to maintain their infrastructure, plan upgrades, and manage their budgets. Properly maintaining this infrastructure relies heavily on consistently monitoring its condition and deterioration over time. This can be a challenge, especially in larger towns and cities where there is a lot of city property to keep an eye on. Municipalities rely on surveyors to keep them up to date on the condition of their infrastructure to prevent this failure before it happens. This is both to prevent injuries and further damage from occurring as a result of infrastructure failure, and since it is can be more cost effective to maintain property rather than have to replace it. Surveying can either be done manually or automatically, but it is not done frequently as it is expensive and also time consuming. Manual surveying can be inaccurate, while a large portion of automatic surveying techniques rely on expensive equipment. To solve this problem, we propose an automated infrastructure assessment method that relies on Street View images for its input and uses various computer vision and pattern recognition methods to generate its assessments. First, we segment the image into 'road' and 'background' regions. We propose a road segmentation algorithm specifically aimed at segmenting roads from street view images. We use Fisher vectors calculated on SIFT descriptors to encode small windows extracted from the main image at multiple scales. Then we classify these patches using an SVM and utilize a Gaussian voting scheme to obtain a segmentation. We additionally utilize a spatial prior to improve this segmentation. Optionally, we improve the segmentation further by making use of a weighted contour map calculated on a shadow-free intrinsic image, and a find an optimal segmentation by utilizing a purity tree. Our algorithm performs well and outputs a good segmentation for further use in road evaluation. We test our method on the KITTI road dataset, and compare it to the state-of-the-art on this dataset, along with a manually annotated subset of Google Street View. After segmenting the road, we describe an algorithm aimed at identifying distressed road regions and pinpointing cracks within them. We predict distressed regions by re-using the computed Fisher vectors and classifying them with a different SVM trained to distinguish between road qualities. We follow this step with a comparison to the weighed contour map within these distressed regions to identify exact crack and defect locations, and use the contour weights to predict the crack severity. Promising results are obtained on our manually annotated dataset, which indicate the viability of using this cost-effective system to perform road quality assessment at a municipal level