Towards sketch-based exploration of terrain : a feasibility study

CISRG discussion paper ; 1

User-directed sketch interpretation

Thesis (M. Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2004.Includes bibliographical references (p. 91-92).I present a novel approach to creating structured diagrams (such as flow charts and object diagrams) by combining an off-line sketch recognition system with the user interface of a traditional structured graphics editor. The system, called UDSI (user-directed sketch interpretation), aims to provide drawing freedom by allowing the user to sketch entirely off-line using a pure pen-and-paper interface. The results of the drawing can then be presented to UDSI, which recognizes shapes and lines and text areas that the user can then polish as desired. The system can infer multiple interpretations for a given sketch, to aid during the user's polishing stage. The UDSI program offers three novel features. First, it implements a greedy algorithm for determing alternative interpretations of the user's original pen drawing. Second, it introduces a user interface for selecting from these multiple candidate interpretations. Third, it implements a circle recognizer using a novel circle-detection algorithm and combines it with other hand-coded recognizers to provide a robust sketch recognition system.by Matthew J. Notowidigdo.M.Eng

DifferSketching: How Differently Do People Sketch 3D Objects?

Multiple sketch datasets have been proposed to understand how people draw 3D objects. However, such datasets are often of small scale and cover a small set of objects or categories. In addition, these datasets contain freehand sketches mostly from expert users, making it difficult to compare the drawings by expert and novice users, while such comparisons are critical in informing more effective sketch-based interfaces for either user groups. These observations motivate us to analyze how differently people with and without adequate drawing skills sketch 3D objects. We invited 70 novice users and 38 expert users to sketch 136 3D objects, which were presented as 362 images rendered from multiple views. This leads to a new dataset of 3,620 freehand multi-view sketches, which are registered with their corresponding 3D objects under certain views. Our dataset is an order of magnitude larger than the existing datasets. We analyze the collected data at three levels, i.e., sketch-level, stroke-level, and pixel-level, under both spatial and temporal characteristics, and within and across groups of creators. We found that the drawings by professionals and novices show significant differences at stroke-level, both intrinsically and extrinsically. We demonstrate the usefulness of our dataset in two applications: (i) freehand-style sketch synthesis, and (ii) posing it as a potential benchmark for sketch-based 3D reconstruction. Our dataset and code are available at https://chufengxiao.github.io/DifferSketching/.Comment: SIGGRAPH Asia 2022 (Journal Track

Deep Learning for Free-Hand Sketch: A Survey

Free-hand sketches are highly illustrative, and have been widely used by humans to depict objects or stories from ancient times to the present. The recent prevalence of touchscreen devices has made sketch creation a much easier task than ever and consequently made sketch-oriented applications increasingly popular. The progress of deep learning has immensely benefited free-hand sketch research and applications. This paper presents a comprehensive survey of the deep learning techniques oriented at free-hand sketch data, and the applications that they enable. The main contents of this survey include: (i) A discussion of the intrinsic traits and unique challenges of free-hand sketch, to highlight the essential differences between sketch data and other data modalities, e.g., natural photos. (ii) A review of the developments of free-hand sketch research in the deep learning era, by surveying existing datasets, research topics, and the state-of-the-art methods through a detailed taxonomy and experimental evaluation. (iii) Promotion of future work via a discussion of bottlenecks, open problems, and potential research directions for the community.Comment: This paper is accepted by IEEE TPAM

Comics reading: An automatic script generation

With the advent of portable devices, reading comic ebooks is a popular activity. However, a simple scan of a comic page is not well adapted for portable device screens and a panel to panel reading without animations and adapted transitions is quite uncomfortable and not suitable. Moreover, applying manually transitions between each panel to script a complete comic book is a tricky task and seems impossible for a complete collection of comics. We present a model able to automatically script comics reading by using panel lines of force. Our results demonstrate that this model proposes a coherent solution for 87.2% of panels in an interactive time

Super-resolução de imagens refinada com informação de bordas utilizando redes neurais residuais

Orientador: Hélio PedriniDissertação (mestrado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: Assim como em outros domínios do conhecimento, as técnicas de aprendizado profundo revolucionaram o desenvolvimento de abordagens para a super-resolução de imagens. Algoritmos recentes para solucionar este problema têm empregado redes neurais convolucionais em arquiteturas residuais com várias camadas e funções gerais de perda. Essas estruturas (arquiteturas e funções de perda) são genéricas e não abordam as principais características de uma imagem para a percepção visual humana (luminância, contraste e estrutura), resultando em melhores imagens, no entanto, com ruído principalmente em suas bordas. Neste trabalho, apresentamos e avaliamos um método, denominado super-resolução de imagens refinada com informação de bordas (Edge Enhanced Super-Resolution - EESR) usando uma nova rede neural residual com foco nas bordas da imagem e uma combinação de funções de perda: Peak Signal-to-Noise Ratio (PSNR), L1, Multiple-Scale Structural Similarity (MS-SSIM) e uma nova função baseada na técnica Pencil Sketch. Como principal contribuição do trabalho, o modelo proposto visa alavancar os limites da super-resolução de imagens, apresentando uma melhoria dos resultados em termos da métrica SSIM e alcançando resultados promissores para a métrica PSNR. Os resultados experimentais obtidos mostram que o modelo desenvolvido é competitivo quando comparado com o estado da arte para os quatro conjuntos de dados (Set05, Set14, B100, Urban100) avaliados para super-resolução de imagensAbstract: As in other knowledge domains, deep learning techniques have revolutionized the development of approaches to image super-resolution. Recent algorithms for addressing this problem have employed convolutional neural networks in multi-layered residual architectures and general loss functions. These structures (architectures and loss functions) are generic and do not address the main features of an image for human visual perception (luminance, contrast and structure), resulting in better images, however, with noise mainly at its edges. In this work, we present and evaluate a method, called Edge Enhanced Super Resolution (EESR), using a new residual neural network focusing on the edges of the image and a combination of loss functions: Peak Signal-to-Noise Ratio (PSNR), L1, Multiple-Scale Structural Similarity (MS-SSIM) and a new function based on the Pencil Sketch technique. As main contribution of this work, the proposed model aims to leverage the limits of image super-resolution, presenting an improvement of the results in terms of the SSIM metric and achieving promising results for the PSNR metric. The obtained experimental results show that the developed model is competitive when compared to the state of the art for the four data sets (Set05, Set14, B100, Urban100) evaluated for image super-resolutionMestradoCiência da ComputaçãoMestre em Ciência da Computaçã