655 research outputs found
Deep Learning for Free-Hand Sketch: A Survey
Free-hand sketches are highly illustrative, and have been widely used by
humans to depict objects or stories from ancient times to the present. The
recent prevalence of touchscreen devices has made sketch creation a much easier
task than ever and consequently made sketch-oriented applications increasingly
popular. The progress of deep learning has immensely benefited free-hand sketch
research and applications. This paper presents a comprehensive survey of the
deep learning techniques oriented at free-hand sketch data, and the
applications that they enable. The main contents of this survey include: (i) A
discussion of the intrinsic traits and unique challenges of free-hand sketch,
to highlight the essential differences between sketch data and other data
modalities, e.g., natural photos. (ii) A review of the developments of
free-hand sketch research in the deep learning era, by surveying existing
datasets, research topics, and the state-of-the-art methods through a detailed
taxonomy and experimental evaluation. (iii) Promotion of future work via a
discussion of bottlenecks, open problems, and potential research directions for
the community.Comment: This paper is accepted by IEEE TPAM
INSIDE: Steering Spatial Attention with Non-Imaging Information in CNNs
We consider the problem of integrating non-imaging information into
segmentation networks to improve performance. Conditioning layers such as FiLM
provide the means to selectively amplify or suppress the contribution of
different feature maps in a linear fashion. However, spatial dependency is
difficult to learn within a convolutional paradigm. In this paper, we propose a
mechanism to allow for spatial localisation conditioned on non-imaging
information, using a feature-wise attention mechanism comprising a
differentiable parametrised function (e.g. Gaussian), prior to applying the
feature-wise modulation. We name our method INstance modulation with SpatIal
DEpendency (INSIDE). The conditioning information might comprise any factors
that relate to spatial or spatio-temporal information such as lesion location,
size, and cardiac cycle phase. Our method can be trained end-to-end and does
not require additional supervision. We evaluate the method on two datasets: a
new CLEVR-Seg dataset where we segment objects based on location, and the ACDC
dataset conditioned on cardiac phase and slice location within the volume. Code
and the CLEVR-Seg dataset are available at https://github.com/jacenkow/inside.Comment: Accepted at International Conference on Medical Image Computing and
Computer Assisted Intervention (MICCAI) 202
- …