A new sketch based interface using the gray-level co-occurrence matrix for perceptual simplification of paper based scribbles

The sketching activity has an important role in conceptual design and a variety of tools exist which help designers to facilitate the generation of 3D models form sketched drawings. This paper describes a new sketch-to-3D tool, which uses annotations to aid the interpretation of the drawing. Over-traced lines present in the designer’s scrib- bles provide an interpretation challenge, which must be resolved in order to obtain 3D models from these sketches. Perceptual grouping techniques used to interpret such images require that the drawing is represented as vectors. These are generally obtained through thinning or edge detection. However, we show that processing scribbles using these techniques result in a large number of vectors which do not provide a faithful representation of the drawing. This paper investigates the use of the co-occurrence matrix to perceptually simplify these drawings, thus obtaining a smaller number of vectors which describe the drawing more faithfully.peer-reviewe

ImageSpirit: Verbal Guided Image Parsing

Humans describe images in terms of nouns and adjectives while algorithms operate on images represented as sets of pixels. Bridging this gap between how humans would like to access images versus their typical representation is the goal of image parsing, which involves assigning object and attribute labels to pixel. In this paper we propose treating nouns as object labels and adjectives as visual attribute labels. This allows us to formulate the image parsing problem as one of jointly estimating per-pixel object and attribute labels from a set of training images. We propose an efficient (interactive time) solution. Using the extracted labels as handles, our system empowers a user to verbally refine the results. This enables hands-free parsing of an image into pixel-wise object/attribute labels that correspond to human semantics. Verbally selecting objects of interests enables a novel and natural interaction modality that can possibly be used to interact with new generation devices (e.g. smart phones, Google Glass, living room devices). We demonstrate our system on a large number of real-world images with varying complexity. To help understand the tradeoffs compared to traditional mouse based interactions, results are reported for both a large scale quantitative evaluation and a user study.Comment: http://mmcheng.net/imagespirit

Ink-and-Ray: Bas-Relief Meshes for Adding Global Illumination Effects to Hand-Drawn Characters

We present a new approach for generating global illumination renderings of hand-drawn characters using only a small set of simple annotations. Our system exploits the concept of bas-relief sculptures, making it possible to generate 3D proxies suitable for rendering without requiring side-views or extensive user input. We formulate an optimization process that automatically constructs approximate geometry sufficient to evoke the impression of a consistent 3D shape. The resulting renders provide the richer stylization capabilities of 3D global illumination while still retaining the 2D handdrawn look-and-feel. We demonstrate our approach on a varied set of handdrawn images and animations, showing that even in comparison to ground truth renderings of full 3D objects, our bas-relief approximation is able to produce convincing global illumination effects, including self-shadowing, glossy reflections, and diffuse color bleeding

Drawing, Handwriting Processing Analysis: New Advances and Challenges

International audienceDrawing and handwriting are communicational skills that are fundamental in geopolitical, ideological and technological evolutions of all time. drawingand handwriting are still useful in defining innovative applications in numerous fields. In this regard, researchers have to solve new problems like those related to the manner in which drawing and handwriting become an efficient way to command various connected objects; or to validate graphomotor skills as evident and objective sources of data useful in the study of human beings, their capabilities and their limits from birth to decline

Bitmap or Vector? A study on sketch representations for deep stroke segmentation

National audienceDeep learning achieves impressive performances on image segmentation, which has motivated the recent developmentof deep neural networks for the related task of sketch segmentation, where the goal is to assign labels to thedifferent strokes that compose a line drawing. However, while natural images are well represented as bitmaps, linedrawings can also be represented as vector graphics, such as point sequences and point clouds. In addition to offeringdifferent trade-offs on resolution and storage, vector representations often come with additional information,such as stroke ordering and speed. In this paper, we evaluate three crucial design choices for sketch segmentationusing deep-learning : which sketch representation to use, which information to encode in this representation,and which loss function to optimize. Our findings suggest that point clouds represent a competitive alternative tobitmaps for sketch segmentation, and that providing extra-geometric information improves performance

The Line of Action: an Intuitive Interface for Expressive Character Posing

International audienceThe line of action is a conceptual tool often used by cartoonists and illustrators to help make their figures more consistent and more dramatic. We often see the expression of characters--may it be the dynamism of a super hero, or the elegance of a fashion model--well captured and amplified by a single aesthetic line. Usually this line is laid down in early stages of the drawing and used to describe the body's principal shape. By focusing on this simple abstraction, the person drawing can quickly adjust and refine the overall pose of his or her character from a given viewpoint. In this paper, we propose a mathematical definition of the line of action (LOA), which allows us to automatically align a 3D virtual character to a user specified LOA by solving an optimization problem. We generalize this framework to other types of lines found in the drawing literature, such as secondary lines used to place arms. Finally, we show a wide range of poses and animations that were rapidly created using our system

Latency management in scribble-based interactive segmentation of medical images

Objective: During an interactive image segmentation task, the outcome is strongly influenced by human factors. In particular, a reduction in computation time does not guarantee an improvement in the overall segmentation time. This paper characterizes user efficiency during scribble-based interactive segmentation as a function of computation time. Methods: We report a controlled experiment with users who experienced eight different levels of simulated latency (ranging from 100 to 2000 ms) with two techniques for refreshing visual feedback (either automatic, where the segmentation was recomputed and displayed continuously during label drawing, or user initiated, which was only computed and displayed each time the user hits a defined button). Results: For short latencies, the user's attention is focused on the automatic visual feedback, slowing down his/her labeling performance. This effect is attenuated as the latency grows larger, and the two refresh techniques yield similar user performance at the largest latencies. Moreover, during the segmentation task, participants spent in average 72.67% ± 2.42% for automatic refresh and 96.23% ± 0.06% for user-initiated refresh of the overall segmentation time interpreting the results. Conclusion: The latency is perceived differently according to the refresh method used during the segmentation task. Therefore, it is possible to reduce its impact on the user performance. Significance: This is the first time a study investigates the effects of latency in an interactive segmentation task. The analysis and recommendations provided in this paper help understanding the cognitive mechanisms in interactive image segmentation