Mode-Seeking on Hypergraphs for Robust Geometric Model Fitting

In this paper, we propose a novel geometric model fitting method, called Mode-Seeking on Hypergraphs (MSH),to deal with multi-structure data even in the presence of severe outliers. The proposed method formulates geometric model fitting as a mode seeking problem on a hypergraph in which vertices represent model hypotheses and hyperedges denote data points. MSH intuitively detects model instances by a simple and effective mode seeking algorithm. In addition to the mode seeking algorithm, MSH includes a similarity measure between vertices on the hypergraph and a weight-aware sampling technique. The proposed method not only alleviates sensitivity to the data distribution, but also is scalable to large scale problems. Experimental results further demonstrate that the proposed method has significant superiority over the state-of-the-art fitting methods on both synthetic data and real images.Comment: Proceedings of the IEEE International Conference on Computer Vision, pp. 2902-2910, 201

Quantum Multi-Model Fitting

Geometric model fitting is a challenging but fundamental computer vision problem. Recently, quantum optimization has been shown to enhance robust fitting for the case of a single model, while leaving the question of multi-model fitting open. In response to this challenge, this paper shows that the latter case can significantly benefit from quantum hardware and proposes the first quantum approach to multi-model fitting (MMF). We formulate MMF as a problem that can be efficiently sampled by modern adiabatic quantum computers without the relaxation of the objective function. We also propose an iterative and decomposed version of our method, which supports real-world-sized problems. The experimental evaluation demonstrates promising results on a variety of datasets. The source code is available at: https://github.com/FarinaMatteo/qmmf.Comment: In Computer Vision and Pattern Recognition (CVPR) 2023; Highligh

Improved image analysis by maximised statistical use of geometry-shape constraints

Identifying the underlying models in a set of data points contaminated by noise and outliers, leads to a highly complex multi-model fitting problem. This problem can be posed as a clustering problem by the construction of higher order affinities between data points into a hypergraph, which can then be partitioned using spectral clustering. Calculating the weights of all hyperedges is computationally expensive. Hence an approximation is required. In this thesis, the aim is to find an efficient and effective approximation that produces an excellent segmentation outcome. Firstly, the effect of hyperedge sizes on the speed and accuracy of the clustering is investigated. Almost all previous work on hypergraph clustering in computer vision, has considered the smallest possible hyperedge size, due to the lack of research into the potential benefits of large hyperedges and effective algorithms to generate them. In this thesis, it is shown that large hyperedges are better from both theoretical and empirical standpoints. The efficiency of this technique on various higher-order grouping problems is investigated. In particular, we show that our approach improves the accuracy and efficiency of motion segmentation from dense, long-term, trajectories. A shortcoming of the above approach is that the probability of a generated sample being impure increases as the size of the sample increases. To address this issue, a novel guided sampling strategy for large hyperedges, based on the concept of minimizing the largest residual, is also included. It is proposed to guide each sample by optimizing over a $k$\textsuperscript{th} order statistics based cost function. Samples are generated using a greedy algorithm coupled with a data sub-sampling strategy. The experimental analysis shows that this proposed step is both accurate and computationally efficient compared to state-of-the-art robust multi-model fitting techniques. However, the optimization method for guiding samples involves hard-to-tune parameters. Thus a sampling method is eventually developed that significantly facilitates solving the segmentation problem using a new form of the Markov-Chain-Monte-Carlo (MCMC) method to efficiently sample from hyperedge distribution. To sample from the above distribution effectively, the proposed Markov Chain includes new types of long and short jumps to perform exploration and exploitation of all structures. Unlike common sampling methods, this method does not require any specific prior knowledge about the distribution of models. The output set of samples leads to a clustering solution by which the final model parameters for each segment are obtained. The overall method competes favorably with the state-of-the-art both in terms of computation power and segmentation accuracy

Deep Adaptive Feature Embedding with Local Sample Distributions for Person Re-identification

Person re-identification (re-id) aims to match pedestrians observed by disjoint camera views. It attracts increasing attention in computer vision due to its importance to surveillance system. To combat the major challenge of cross-view visual variations, deep embedding approaches are proposed by learning a compact feature space from images such that the Euclidean distances correspond to their cross-view similarity metric. However, the global Euclidean distance cannot faithfully characterize the ideal similarity in a complex visual feature space because features of pedestrian images exhibit unknown distributions due to large variations in poses, illumination and occlusion. Moreover, intra-personal training samples within a local range are robust to guide deep embedding against uncontrolled variations, which however, cannot be captured by a global Euclidean distance. In this paper, we study the problem of person re-id by proposing a novel sampling to mine suitable \textit{positives} (i.e. intra-class) within a local range to improve the deep embedding in the context of large intra-class variations. Our method is capable of learning a deep similarity metric adaptive to local sample structure by minimizing each sample's local distances while propagating through the relationship between samples to attain the whole intra-class minimization. To this end, a novel objective function is proposed to jointly optimize similarity metric learning, local positive mining and robust deep embedding. This yields local discriminations by selecting local-ranged positive samples, and the learned features are robust to dramatic intra-class variations. Experiments on benchmarks show state-of-the-art results achieved by our method.Comment: Published on Pattern Recognitio

Progressive-X: Efficient, Anytime, Multi-Model Fitting Algorithm

The Progressive-X algorithm, Prog-X in short, is proposed for geometric multi-model fitting. The method interleaves sampling and consolidation of the current data interpretation via repetitive hypothesis proposal, fast rejection, and integration of the new hypothesis into the kept instance set by labeling energy minimization. Due to exploring the data progressively, the method has several beneficial properties compared with the state-of-the-art. First, a clear criterion, adopted from RANSAC, controls the termination and stops the algorithm when the probability of finding a new model with a reasonable number of inliers falls below a threshold. Second, Prog-X is an any-time algorithm. Thus, whenever is interrupted, e.g. due to a time limit, the returned instances cover real and, likely, the most dominant ones. The method is superior to the state-of-the-art in terms of accuracy in both synthetic experiments and on publicly available real-world datasets for homography, two-view motion, and motion segmentation

Multiple structure recovery with T-linkage

reserved2noThis work addresses the problem of robust fitting of geometric structures to noisy data corrupted by outliers. An extension of J-linkage (called T-linkage) is presented and elaborated. T-linkage improves the preference analysis implemented by J-linkage in term of performances and robustness, considering both the representation and the segmentation steps. A strategy to reject outliers and to estimate the inlier threshold is proposed, resulting in a versatile tool, suitable for multi-model fitting “in the wild”. Experiments demonstrate that our methods perform better than J-linkage on simulated data, and compare favorably with state-of-the-art methods on public domain real datasets.mixedMagri L.; Fusiello A.Magri, L.; Fusiello, A

Multi-class Model Fitting by Energy Minimization and Mode-Seeking

We propose a general formulation, called Multi-X, for multi-class multi-instance model fitting - the problem of interpreting the input data as a mixture of noisy observations originating from multiple instances of multiple classes. We extend the commonly used alpha-expansion-based technique with a new move in the label space. The move replaces a set of labels with the corresponding density mode in the model parameter domain, thus achieving fast and robust optimization. Key optimization parameters like the bandwidth of the mode seeking are set automatically within the algorithm. Considering that a group of outliers may form spatially coherent structures in the data, we propose a cross-validation-based technique removing statistically insignificant instances. Multi-X outperforms significantly the state-of-the-art on publicly available datasets for diverse problems: multiple plane and rigid motion detection; motion segmentation; simultaneous plane and cylinder fitting; circle and line fitting

Higher-Order Multicuts for Geometric Model Fitting and Motion Segmentation

Minimum cost lifted multicut problem is a generalization of the multicut problem and is a means to optimizing a decomposition of a graph w.r.t. both positive and negative edge costs. Its main advantage is that multicut-based formulations do not require the number of components given a priori; instead, it is deduced from the solution. However, the standard multicut cost function is limited to pairwise relationships between nodes, while several important applications either require or can benefit from a higher-order cost function, i.e. hyper-edges. In this paper, we propose a pseudo-boolean formulation for a multiple model fitting problem. It is based on a formulation of any-order minimum cost lifted multicuts, which allows to partition an undirected graph with pairwise connectivity such as to minimize costs defined over any set of hyper-edges. As the proposed formulation is NP-hard and the branch-and-bound algorithm is too slow in practice, we propose an efficient local search algorithm for inference into resulting problems. We demonstrate versatility and effectiveness of our approach in several applications: geometric multiple model fitting, homography and motion estimation, motion segmentation