Modeling the Background for Incremental and Weakly-Supervised Semantic Segmentation

Deep neural networks have enabled major progresses in semantic segmentation. However, even the most advanced neural architectures suffer from important limitations. First, they are vulnerable to catastrophic forgetting, i.e. they perform poorly when they are required to incrementally update their model as new classes are available. Second, they rely on large amount of pixel-level annotations to produce accurate segmentation maps. To tackle these issues, we introduce a novel incremental class learning approach for semantic segmentation taking into account a peculiar aspect of this task: since each training step provides annotation only for a subset of all possible classes, pixels of the background class exhibit a semantic shift. Therefore, we revisit the traditional distillation paradigm by designing novel loss terms which explicitly account for the background shift. Additionally, we introduce a novel strategy to initialize classifiers parameters at each step in order to prevent biased predictions toward the background class. Finally, we demonstrate that our approach can be extended to point- and scribble-based weakly supervised segmentation, modeling the partial annotations to create priors for unlabeled pixels. We demonstrate the effectiveness of our approach with an extensive evaluation on the Pascal-VOC, ADE20K, and Cityscapes datasets, significantly outperforming state-of-the-art methods

Fast 3D surface reconstruction by unambiguous compound phase coding

Phase shift methods have proven to be very robust and accurate for photometric 3D reconstruction. One problem of these approaches is the existence of ambiguities arising from the periodicity of the fringe patterns. While several techniques for disambiguation exist, all of them require the projection of a significant number of additional patterns. For instance, a global Gray coding sequence or several supplemental sinusoidal patterns of different periods are commonly used to complement the basic phase shift technique. In this paper we propose a new pattern strategy to reduce the total number of patterns projected by encoding multiple phases into a single sequence. This is obtained by mixing multiple equal-amplitude sinusoidal signals, which can be efficiently computed using inverse Fourier transformation. The initial phase for each fringe is then recovered independently through Fourier analysis and the unique projected coordinate is computed from the phase vectors using the disambiguation approach based on multiple periods fringes proposed by Lilienblum and Michaelis[6]. With respect to competing approaches, our method is simpler and requires fewer structured light patterns, thus reducing the measurement time, while retaining high level of accuracy. ©2009 IEEE

A game-theoretic approach to hypergraph clustering

Abstract—Hypergraph clustering refers to the process of extracting maximally coherent groups from a set of objects using high-order (rather than pairwise) similarities. Traditional approaches to this problem are based on the idea of partitioning the input data into a predetermined number of classes, thereby obtaining the clusters as a by-product of the partitioning process. In this paper, we offer a radically different view of the problem. In contrast to the classical approach, we attempt to provide a meaningful formalization of the very notion of a cluster and we show that game theory offers an attractive and unexplored perspective that serves our purpose well. To this end, we formulate the hypergraph clustering problem in terms of a noncooperative multiplayer “clustering game, ” and show that a natural notion of a cluster turns out to be equivalent to a classical (evolutionary) game-theoretic equilibrium concept. We prove that the problem of finding the equilibria of our clustering game is equivalent to locally optimizing a polynomial function over the standard simplex, and we provide a discrete-time high-order replicator dynamics to perform this optimization, based on the Baum-Eagon inequality. Experiments over synthetic as well as real-world data are presented which show the superiority of our approach over the state of the art. Index Terms—Hypergraph clustering, evolutionary game theory, polynomial optimization, Baum-Eagon inequality, high-order replicator dynamics Ç

Grouping with Asymmetric Affinities: A Game-Theoretic Perspective

Pairwise grouping and clustering approaches have tra-ditionally worked under the assumption that the similari-ties or compatibilities between the elements to be grouped are symmetric. However, asymmetric compatibilities arise naturally in many areas of computer vision and pattern recognition. Hence, there is a need for a new generic ap-proach to clustering and grouping that can deal with asym-metries in the compatibilities. In this paper, we present a generic framework for grouping and clustering derived from a game-theoretic formalization of the competition be-tween the hypotheses of group membership, and apply it to perceptual grouping. In the proposed approach groups correspond to evolutionary stable strategies, a classic no-tion in evolutionary game theory. We also provide a com-binatorial characterization of the stable strategies, and, hence, of the elements that belong to a group. Experiments show that our approach outperforms both state-of-the-art clustering-based perceptual grouping approacheswith sym-metric compatibilities, and other approaches explicitly de-signed to make use of asymmetric compatibilities. 1