11 research outputs found
Domain Generalization by Solving Jigsaw Puzzles
Human adaptability relies crucially on the ability to learn and merge
knowledge both from supervised and unsupervised learning: the parents point out
few important concepts, but then the children fill in the gaps on their own.
This is particularly effective, because supervised learning can never be
exhaustive and thus learning autonomously allows to discover invariances and
regularities that help to generalize. In this paper we propose to apply a
similar approach to the task of object recognition across domains: our model
learns the semantic labels in a supervised fashion, and broadens its
understanding of the data by learning from self-supervised signals how to solve
a jigsaw puzzle on the same images. This secondary task helps the network to
learn the concepts of spatial correlation while acting as a regularizer for the
classification task. Multiple experiments on the PACS, VLCS, Office-Home and
digits datasets confirm our intuition and show that this simple method
outperforms previous domain generalization and adaptation solutions. An
ablation study further illustrates the inner workings of our approach.Comment: Accepted at CVPR 2019 (oral
A Novel Hybrid Scheme Using Genetic Algorithms and Deep Learning for the Reconstruction of Portuguese Tile Panels
This paper presents a novel scheme, based on a unique combination of genetic
algorithms (GAs) and deep learning (DL), for the automatic reconstruction of
Portuguese tile panels, a challenging real-world variant of the jigsaw puzzle
problem (JPP) with important national heritage implications. Specifically, we
introduce an enhanced GA-based puzzle solver, whose integration with a novel
DL-based compatibility measure (DLCM) yields state-of-the-art performance,
regarding the above application. Current compatibility measures consider
typically (the chromatic information of) edge pixels (between adjacent tiles),
and help achieve high accuracy for the synthetic JPP variant. However, such
measures exhibit rather poor performance when applied to the Portuguese tile
panels, which are susceptible to various real-world effects, e.g.,
monochromatic panels, non-squared tiles, edge degradation, etc. To overcome
such difficulties, we have developed a novel DLCM to extract high-level
texture/color statistics from the entire tile information.
Integrating this measure with our enhanced GA-based puzzle solver, we have
demonstrated, for the first time, how to deal most effectively with large-scale
real-world problems, such as the Portuguese tile problem. Specifically, we have
achieved 82% accuracy for the reconstruction of Portuguese tile panels with
unknown piece rotation and puzzle dimension (compared to merely 3.5% average
accuracy achieved by the best method known for solving this problem variant).
The proposed method outperforms even human experts in several cases, correcting
their mistakes in the manual tile assembly
Solving Jigsaw Puzzles By the Graph Connection Laplacian
We propose a novel mathematical framework to address the problem of
automatically solving large jigsaw puzzles. This problem assumes a large image,
which is cut into equal square pieces that are arbitrarily rotated and
shuffled, and asks to recover the original image given the transformed pieces.
The main contribution of this work is a method for recovering the rotations of
the pieces when both shuffles and rotations are unknown. A major challenge of
this procedure is estimating the graph connection Laplacian without the
knowledge of shuffles. We guarantee some robustness of the latter estimate to
measurement errors. A careful combination of our proposed method for estimating
rotations with any existing method for estimating shuffles results in a
practical solution for the jigsaw puzzle problem. Numerical experiments
demonstrate the competitive accuracy of this solution, its robustness to
corruption and its computational advantage for large puzzles
A Generalized Genetic Algorithm-Based Solver for Very Large Jigsaw Puzzles of Complex Types
In this paper we introduce new types of square-piece jigsaw puzzles, where in addition to the unknown location and orientation of each piece, a piece might also need to be flipped. These puzzles, which are associated with a number of real world problems, are considerably harder, from a computational standpoint. Specifically, we present a novel generalized genetic algorithm (GA)-based solver that can handle puzzle pieces of unknown location and orientation (Type 2 puzzles) and (two-sided) puzzle pieces of unknown location, orientation, and face (Type 4 puzzles). To the best of our knowledge, our solver provides a new state-of-the-art, solving previously attempted puzzles faster and far more accurately, handling puzzle sizes that have never been attempted before, and assembling the newly introduced two-sided puzzles automatically and effectively. This paper also presents, among other results, the most extensive set of experimental results, compiled as of yet, on Type 2 puzzles