457 research outputs found
CLIPVG: Text-Guided Image Manipulation Using Differentiable Vector Graphics
Considerable progress has recently been made in leveraging CLIP (Contrastive
Language-Image Pre-Training) models for text-guided image manipulation.
However, all existing works rely on additional generative models to ensure the
quality of results, because CLIP alone cannot provide enough guidance
information for fine-scale pixel-level changes. In this paper, we introduce
CLIPVG, a text-guided image manipulation framework using differentiable vector
graphics, which is also the first CLIP-based general image manipulation
framework that does not require any additional generative models. We
demonstrate that CLIPVG can not only achieve state-of-art performance in both
semantic correctness and synthesis quality, but also is flexible enough to
support various applications far beyond the capability of all existing methods.Comment: 8 pages, 10 figures, AAAI202
Adaptive image vectorisation and brushing using mesh colours
We propose the use of curved triangles and mesh colours as a vector primitive for image vectorisation. We show that our representation has clear benefits for rendering performance, texture detail, as well as further editing of the resulting vector images. The proposed method focuses on efficiency, but it still leads to results that compare favourably with those from previous work. We show results over a variety of input images ranging from photos, drawings, paintings, all the way to designs and cartoons. We implemented several editing workflows facilitated by our representation: interactive user-guided vectorisation, and novel raster-style feature-aware brushing capabilities
Im2Vec: Synthesizing Vector Graphics without Vector Supervision
Vector graphics are widely used to represent fonts, logos, digital artworks, and graphic designs. But, while a vast body of work has focused on generative algorithms for raster images, only a handful of options exists for vector graphics. One can always rasterize the input graphic and resort to image-based generative approaches, but this negates the advantages of the vector representation. The current alternative is to use specialized models that require explicit supervision on the vector graphics representation at training time. This is not ideal because large-scale high quality vector-graphics datasets are difficult to obtain. Furthermore, the vector representation for a given design is not unique, so models that supervise on the vector representation are unnecessarily constrained. Instead, we propose a new neural network that can generate complex vector graphics with varying topologies, and only requires indirect supervision from readily-available raster training images (i.e., with no vector counterparts). To enable this, we use a differentiable rasterization pipeline that renders the generated vector shapes and composites them together onto a raster canvas. We demonstrate our method on a range of datasets, and provide comparison with state-of-the-art SVG-VAE and DeepSVG, both of which require explicit vector graphics supervision. Finally, we also demonstrate our approach on the MNIST dataset, for which no groundtruth vector representation is available. Source code, datasets, and more results are available at geometry.cs.ucl.ac.uk/projects/2021/Im2Vec
The Scalable Brain Atlas: instant web-based access to public brain atlases and related content
The Scalable Brain Atlas (SBA) is a collection of web services that provide
unified access to a large collection of brain atlas templates for different
species. Its main component is an atlas viewer that displays brain atlas data
as a stack of slices in which stereotaxic coordinates and brain regions can be
selected. These are subsequently used to launch web queries to resources that
require coordinates or region names as input. It supports plugins which run
inside the viewer and respond when a new slice, coordinate or region is
selected. It contains 20 atlas templates in six species, and plugins to compute
coordinate transformations, display anatomical connectivity and fiducial
points, and retrieve properties, descriptions, definitions and 3d
reconstructions of brain regions. The ambition of SBA is to provide a unified
representation of all publicly available brain atlases directly in the web
browser, while remaining a responsive and light weight resource that
specializes in atlas comparisons, searches, coordinate transformations and
interactive displays.Comment: Rolf K\"otter sadly passed away on June 9th, 2010. He co-initiated
this project and played a crucial role in the design and quality assurance of
the Scalable Brain Atla
Vectorization of Large Amounts of Raster Satellite Images in a Distributed Architecture Using HIPI
Vectorization process focus on grouping pixels of a raster image into raw
line segments, and forming lines, polylines or poligons. To vectorize massive
raster images regarding resource and performane problems, weuse a distributed
HIPI image processing interface based on MapReduce approach. Apache Hadoop is
placed at the core of the framework. To realize such a system, we first define
mapper function, and then its input and output formats. In this paper, mappers
convert raster mosaics into vector counterparts. Reduc functions are not needed
for vectorization. Vector representations of raster images is expected to give
better performance in distributed computations by reducing the negative effects
of bandwidth problem and horizontal scalability analysis is done.Comment: In Turkish, Proceedings of International Artificial Intelligence and
Data Processing Symposium (IDAP) 201
Character Generation through Self-Supervised Vectorization
The prevalent approach in self-supervised image generation is to operate on
pixel level representations. While this approach can produce high quality
images, it cannot benefit from the simplicity and innate quality of
vectorization. Here we present a drawing agent that operates on stroke-level
representation of images. At each time step, the agent first assesses the
current canvas and decides whether to stop or keep drawing. When a 'draw'
decision is made, the agent outputs a program indicating the stroke to be
drawn. As a result, it produces a final raster image by drawing the strokes on
a canvas, using a minimal number of strokes and dynamically deciding when to
stop. We train our agent through reinforcement learning on MNIST and Omniglot
datasets for unconditional generation and parsing (reconstruction) tasks. We
utilize our parsing agent for exemplar generation and type conditioned concept
generation in Omniglot challenge without any further training. We present
successful results on all three generation tasks and the parsing task.
Crucially, we do not need any stroke-level or vector supervision; we only use
raster images for training
Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwriting
Self-supervised learning has gained prominence due to its efficacy at
learning powerful representations from unlabelled data that achieve excellent
performance on many challenging downstream tasks. However supervision-free
pre-text tasks are challenging to design and usually modality specific.
Although there is a rich literature of self-supervised methods for either
spatial (such as images) or temporal data (sound or text) modalities, a common
pre-text task that benefits both modalities is largely missing. In this paper,
we are interested in defining a self-supervised pre-text task for sketches and
handwriting data. This data is uniquely characterised by its existence in dual
modalities of rasterized images and vector coordinate sequences. We address and
exploit this dual representation by proposing two novel cross-modal translation
pre-text tasks for self-supervised feature learning: Vectorization and
Rasterization. Vectorization learns to map image space to vector coordinates
and rasterization maps vector coordinates to image space. We show that the our
learned encoder modules benefit both raster-based and vector-based downstream
approaches to analysing hand-drawn data. Empirical evidence shows that our
novel pre-text tasks surpass existing single and multi-modal self-supervision
methods.Comment: IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2021
Code : https://github.com/AyanKumarBhunia/Self-Supervised-Learning-for-Sketc
- …