784 research outputs found
YODA – Your Only Design Assistant
Converting user interface designs created by graphic designers into computer code is a typical job of a front end engineer in order to develop functional web and mobile applications. This conversion process can often be extremely tedious, slow and prone to human error. In this project, deep learning based object detection along with optical character recognition is used to generate platform ready prototypes directly from design sketches. Also, a new design language is introduced to facilitate expressive prototyping and allowing the creation of more expressive and functional designs. It is observed that the AI powered application along with modern web technology can significantly help streamline and automate the overall product development routine and eliminate hurdles from the product development process
Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwriting
Self-supervised learning has gained prominence due to its efficacy at
learning powerful representations from unlabelled data that achieve excellent
performance on many challenging downstream tasks. However supervision-free
pre-text tasks are challenging to design and usually modality specific.
Although there is a rich literature of self-supervised methods for either
spatial (such as images) or temporal data (sound or text) modalities, a common
pre-text task that benefits both modalities is largely missing. In this paper,
we are interested in defining a self-supervised pre-text task for sketches and
handwriting data. This data is uniquely characterised by its existence in dual
modalities of rasterized images and vector coordinate sequences. We address and
exploit this dual representation by proposing two novel cross-modal translation
pre-text tasks for self-supervised feature learning: Vectorization and
Rasterization. Vectorization learns to map image space to vector coordinates
and rasterization maps vector coordinates to image space. We show that the our
learned encoder modules benefit both raster-based and vector-based downstream
approaches to analysing hand-drawn data. Empirical evidence shows that our
novel pre-text tasks surpass existing single and multi-modal self-supervision
methods.Comment: IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2021
Code : https://github.com/AyanKumarBhunia/Self-Supervised-Learning-for-Sketc
BĂ©zierSketch: A Generative Model for Scalable Vector Sketches
The study of neural generative models of human sketches is a fascinating
contemporary modeling problem due to the links between sketch image generation
and the human drawing process. The landmark SketchRNN provided breakthrough by
sequentially generating sketches as a sequence of waypoints. However this leads
to low-resolution image generation, and failure to model long sketches. In this
paper we present B\'ezierSketch, a novel generative model for fully vector
sketches that are automatically scalable and high-resolution. To this end, we
first introduce a novel inverse graphics approach to stroke embedding that
trains an encoder to embed each stroke to its best fit B\'ezier curve. This
enables us to treat sketches as short sequences of paramaterized strokes and
thus train a recurrent sketch generator with greater capacity for longer
sketches, while producing scalable high-resolution results. We report
qualitative and quantitative results on the Quick, Draw! benchmark.Comment: Accepted as poster at ECCV 202
Sketch-based Video Object Segmentation: Benchmark and Analysis
Reference-based video object segmentation is an emerging topic which aims to
segment the corresponding target object in each video frame referred by a given
reference, such as a language expression or a photo mask. However, language
expressions can sometimes be vague in conveying an intended concept and
ambiguous when similar objects in one frame are hard to distinguish by
language. Meanwhile, photo masks are costly to annotate and less practical to
provide in a real application. This paper introduces a new task of sketch-based
video object segmentation, an associated benchmark, and a strong baseline. Our
benchmark includes three datasets, Sketch-DAVIS16, Sketch-DAVIS17 and
Sketch-YouTube-VOS, which exploit human-drawn sketches as an informative yet
low-cost reference for video object segmentation. We take advantage of STCN, a
popular baseline of semi-supervised VOS task, and evaluate what the most
effective design for incorporating a sketch reference is. Experimental results
show sketch is more effective yet annotation-efficient than other references,
such as photo masks, language and scribble.Comment: BMVC 202
- …