10 research outputs found
Two-Level Adversarial Visual-Semantic Coupling for Generalized Zero-shot Learning
The performance of generative zero-shot methods mainly depends on the quality
of generated features and how well the model facilitates knowledge transfer
between visual and semantic domains. The quality of generated features is a
direct consequence of the ability of the model to capture the several modes of
the underlying data distribution. To address these issues, we propose a new
two-level joint maximization idea to augment the generative network with an
inference network during training which helps our model capture the several
modes of the data and generate features that better represent the underlying
data distribution. This provides strong cross-modal interaction for effective
transfer of knowledge between visual and semantic domains. Furthermore,
existing methods train the zero-shot classifier either on generate synthetic
image features or latent embeddings produced by leveraging representation
learning. In this work, we unify these paradigms into a single model which in
addition to synthesizing image features, also utilizes the representation
learning capabilities of the inference network to provide discriminative
features for the final zero-shot recognition task. We evaluate our approach on
four benchmark datasets i.e. CUB, FLO, AWA1 and AWA2 against several
state-of-the-art methods, and show its performance. We also perform ablation
studies to analyze and understand our method more carefully for the Generalized
Zero-shot Learning task.Comment: Under Submissio
Joint one-sided synthetic unpaired image translation and segmentation for colorectal cancer prevention
Deep learning has shown excellent performance in analysing medical images.
However, datasets are difficult to obtain due privacy issues, standardization
problems, and lack of annotations. We address these problems by producing
realistic synthetic images using a combination of 3D technologies and
generative adversarial networks. We propose CUT-seg, a joint training where a
segmentation model and a generative model are jointly trained to produce
realistic images while learning to segment polyps. We take advantage of recent
one-sided translation models because they use significantly less memory,
allowing us to add a segmentation model in the training loop. CUT-seg performs
better, is computationally less expensive, and requires less real images than
other memory-intensive image translation approaches that require two stage
training. Promising results are achieved on five real polyp segmentation
datasets using only one real image and zero real annotations. As a part of this
study we release Synth-Colon, an entirely synthetic dataset that includes 20000
realistic colon images and additional details about depth and 3D geometry:
https://enric1994.github.io/synth-colonComment: arXiv admin note: substantial text overlap with arXiv:2202.0868