358 research outputs found
Deep Learning for Free-Hand Sketch: A Survey
Free-hand sketches are highly illustrative, and have been widely used by
humans to depict objects or stories from ancient times to the present. The
recent prevalence of touchscreen devices has made sketch creation a much easier
task than ever and consequently made sketch-oriented applications increasingly
popular. The progress of deep learning has immensely benefited free-hand sketch
research and applications. This paper presents a comprehensive survey of the
deep learning techniques oriented at free-hand sketch data, and the
applications that they enable. The main contents of this survey include: (i) A
discussion of the intrinsic traits and unique challenges of free-hand sketch,
to highlight the essential differences between sketch data and other data
modalities, e.g., natural photos. (ii) A review of the developments of
free-hand sketch research in the deep learning era, by surveying existing
datasets, research topics, and the state-of-the-art methods through a detailed
taxonomy and experimental evaluation. (iii) Promotion of future work via a
discussion of bottlenecks, open problems, and potential research directions for
the community.Comment: This paper is accepted by IEEE TPAM
Component Segmentation of Engineering Drawings Using Graph Convolutional Networks
We present a data-driven framework to automate the vectorization and machine
interpretation of 2D engineering part drawings. In industrial settings, most
manufacturing engineers still rely on manual reads to identify the topological
and manufacturing requirements from drawings submitted by designers. The
interpretation process is laborious and time-consuming, which severely inhibits
the efficiency of part quotation and manufacturing tasks. While recent advances
in image-based computer vision methods have demonstrated great potential in
interpreting natural images through semantic segmentation approaches, the
application of such methods in parsing engineering technical drawings into
semantically accurate components remains a significant challenge. The severe
pixel sparsity in engineering drawings also restricts the effective
featurization of image-based data-driven methods. To overcome these challenges,
we propose a deep learning based framework that predicts the semantic type of
each vectorized component. Taking a raster image as input, we vectorize all
components through thinning, stroke tracing, and cubic bezier fitting. Then a
graph of such components is generated based on the connectivity between the
components. Finally, a graph convolutional neural network is trained on this
graph data to identify the semantic type of each component. We test our
framework in the context of semantic segmentation of text, dimension and,
contour components in engineering drawings. Results show that our method yields
the best performance compared to recent image, and graph-based segmentation
methods.Comment: Preprint accepted to Computers in Industr
Can i teach a robot to replicate a line art
Line art is arguably one of the fundamental and versatile modes of
expression. We propose a pipeline for a robot to look at a grayscale line art
and redraw it. The key novel elements of our pipeline are: a) we propose a
novel task of mimicking line drawings, b) to solve the pipeline we modify the
Quick-draw dataset to obtain supervised training for converting a line drawing
into a series of strokes c) we propose a multi-stage segmentation and graph
interpretation pipeline for solving the problem. The resultant method has also
been deployed on a CNC plotter as well as a robotic arm. We have trained
several variations of the proposed methods and evaluate these on a dataset
obtained from Quick-draw. Through the best methods we observe an accuracy of
around 98% for this task, which is a significant improvement over the baseline
architecture we adapted from. This therefore allows for deployment of the
method on robots for replicating line art in a reliable manner. We also show
that while the rule-based vectorization methods do suffice for simple drawings,
it fails for more complicated sketches, unlike our method which generalizes
well to more complicated distributions.Comment: 9 pages, Accepted for the 2020 Winter Conference on Applications of
Computer Vision (WACV '20); Supplementary Video: https://youtu.be/nMt5Dw04Xh
Deep Vectorization of Technical Drawings
We present a new method for vectorization of technical line drawings, such as
floor plans, architectural drawings, and 2D CAD images. Our method includes (1)
a deep learning-based cleaning stage to eliminate the background and
imperfections in the image and fill in missing parts, (2) a transformer-based
network to estimate vector primitives, and (3) optimization procedure to obtain
the final primitive configurations. We train the networks on synthetic data,
renderings of vector line drawings, and manually vectorized scans of line
drawings. Our method quantitatively and qualitatively outperforms a number of
existing techniques on a collection of representative technical drawings
A survey of comics research in computer science
Graphical novels such as comics and mangas are well known all over the world.
The digital transition started to change the way people are reading comics,
more and more on smartphones and tablets and less and less on paper. In the
recent years, a wide variety of research about comics has been proposed and
might change the way comics are created, distributed and read in future years.
Early work focuses on low level document image analysis: indeed comic books are
complex, they contains text, drawings, balloon, panels, onomatopoeia, etc.
Different fields of computer science covered research about user interaction
and content generation such as multimedia, artificial intelligence,
human-computer interaction, etc. with different sets of values. We propose in
this paper to review the previous research about comics in computer science, to
state what have been done and to give some insights about the main outlooks
SuperpixelGraph: Semi-automatic generation of building footprint through semantic-sensitive superpixel and neural graph networks
Most urban applications necessitate building footprints in the form of
concise vector graphics with sharp boundaries rather than pixel-wise raster
images. This need contrasts with the majority of existing methods, which
typically generate over-smoothed footprint polygons. Editing these
automatically produced polygons can be inefficient, if not more time-consuming
than manual digitization. This paper introduces a semi-automatic approach for
building footprint extraction through semantically-sensitive superpixels and
neural graph networks. Drawing inspiration from object-based classification
techniques, we first learn to generate superpixels that are not only
boundary-preserving but also semantically-sensitive. The superpixels respond
exclusively to building boundaries rather than other natural objects, while
simultaneously producing semantic segmentation of the buildings. These
intermediate superpixel representations can be naturally considered as nodes
within a graph. Consequently, graph neural networks are employed to model the
global interactions among all superpixels and enhance the representativeness of
node features for building segmentation. Classical approaches are utilized to
extract and regularize boundaries for the vectorized building footprints.
Utilizing minimal clicks and straightforward strokes, we efficiently accomplish
accurate segmentation outcomes, eliminating the necessity for editing polygon
vertices. Our proposed approach demonstrates superior precision and efficacy,
as validated by experimental assessments on various public benchmark datasets.
A significant improvement of 8% in AP50 was observed in vector graphics
evaluation, surpassing established techniques. Additionally, we have devised an
optimized and sophisticated pipeline for interactive editing, poised to further
augment the overall quality of the results
- …