A structural representation for understanding line-drawing images

International audienceIn this paper, we are concerned with the problem of finding a good and homogeneous representation to encode line-drawing documents (which may be handwritten). We propose a method in which the problems induced by a first-step skeletonization have been avoided. First, we vectorize the image, to get a fine description of the drawing, using only vectors and quadrilateral primitives. A structural graph is built with the primitives extracted from the initial line-drawing image. The objective is to manage attributes relative to elementary objects so as to provide a description of the spatial relationships (inclusion, junction, intersection, etc.) that exist between the graphics in the images. This is done with a representation that provides a global vision of the drawings. The capacity of the representation to evolve and to carry highly semantic information is also highlighted. Finally, we show how an architecture using this structural representation and a mechanism of perceptive cycles can lead to a high-quality interpretation of line drawings

Reconstruction of machine-made shapes from bitmap sketches

We propose a method of reconstructing 3D machine-made shapes from bitmap sketches by separating an input image into individual patches and jointly optimizing their geometry. We rely on two main observations: (1) human observers interpret sketches of man-made shapes as a collection of simple geometric primitives, and (2) sketch strokes often indicate occlusion contours or sharp ridges between those primitives. Using these main observations we design a system that takes a single bitmap image of a shape, estimates image depth and segmentation into primitives with neural networks, then fits primitives to the predicted depth while determining occlusion contours and aligning intersections with the input drawing via optimization. Unlike previous work, our approach does not require additional input, annotation, or templates, and does not require retraining for a new category of man-made shapes. Our method produces triangular meshes that display sharp geometric features and are suitable for downstream applications, such as editing, rendering, and shading

Analysis of Children's Sketches to Improve Recognition Accuracy in Sketch-Based Applications

The current education systems in elementary schools are usually using traditional teaching methods such as paper and pencil or drawing on the board. The benefit of paper and pencil is their ease of use. Researchers have tried to bring this ease of use to computer-based educational systems through the use of sketch-recognition. Sketch-recognition allows students to draw naturally while at the same time receiving automated assistance and feedback from the computer. There are many sketch-based educational systems for children. However, current sketch-based educational systems use the same sketch recognizer for both adults and children. The problem of this approach is that the recognizers are trained by using sample data drawn by adults, even though the drawing patterns of children and adults are markedly different. We propose that if we make a separate recognizer for children, we can increase the recognition accuracy of shapes drawn by children. By creating a separate recognizer for children, we improved the recognition accuracy of children’s drawings from 81.25% (using the adults’ threshold) to 83.75% (using adjusted threshold for children). Additionally, we were able to automatically distinguish children’s drawings from adults’ drawings. We correctly identified the drawer’s age (age 3, 4, 7, or adult) with 78.3%. When distinguishing toddlers (age 3 and 4) from matures (age 7 and adult), we got a precision of 95.2% using 10-fold cross validation. When we removed adults and distinguished between toddlers and 7 year olds, we got a precision of 90.2%. Distinguishing between 3, 4, and 7 year olds, we got a precision of 86.8%. Furthermore, we revealed that there is a potential gender difference since our recognizer was more accurately able to recognize the drawings of female children (91.4%) than the male children (85.4%). Finally, this paper introduces a sketch-based teaching assistant tool for children, EasySketch, which teaches children how to draw digits and characters. Children can learn how to draw digits and characters by instructions and feedback

Real-time spatial modeling to detect and track resources on construction sites

For more than 10 years the U.S. construction industry has experienced over 1,000 fatalities annually. Many fatalities may have been prevented had the individuals and equipment involved been more aware of and alert to the physical state of the environment around them. Awareness may be improved by automatic 3D (three-dimensional) sensing and modeling of the job site environment in real-time. Existing 3D modeling approaches based on range scanning techniques are capable of modeling static objects only, and thus cannot model in real-time dynamic objects in an environment comprised of moving humans, equipment, and materials. Emerging prototype 3D video range cameras offer another alternative by facilitating affordable, wide field of view, automated static and dynamic object detection and tracking at frame rates better than 1Hz (real-time). This dissertation presents an imperical work and methodology to rapidly create a spatial model of construction sites and in particular to detect, model, and track the position, dimension, direction, and velocity of static and moving project resources in real-time, based on range data obtained from a three-dimensional video range camera in a static or moving position. Existing construction site 3D modeling approaches based on optical range sensing technologies (laser scanners, rangefinders, etc.) and 3D modeling approaches (dense, sparse, etc.) that offered potential solutions for this research are reviewed. The choice of an emerging sensing tool and preliminary experiments with this prototype sensing technology are discussed. These findings led to the development of a range data processing algorithm based on three-dimensional occupancy grids which is demonstrated in detail. Testing and validation of the proposed algorithms have been conducted to quantify the performance of sensor and algorithm through extensive experimentation involving static and moving objects. Experiments in indoor laboratory and outdoor construction environments have been conducted with construction resources such as humans, equipment, materials, or structures to verify the accuracy of the occupancy grid modeling approach. Results show that modeling objects and measuring their position, dimension, direction, and speed had an accuracy level compatible to the requirements of active safety features for construction. Results demonstrate that video rate 3D data acquisition and analysis of construction environments can support effective detection, tracking, and convex hull modeling of objects. Exploiting rapidly generated three-dimensional models for improved visualization, communications, and process control has inherent value, broad application, and potential impact, e.g. as-built vs. as-planned comparison, condition assessment, maintenance, operations, and construction activities control. In combination with effective management practices, this sensing approach has the potential to assist equipment operators to avoid incidents that result in reduce human injury, death, or collateral damage on construction sites.Civil, Architectural, and Environmental Engineerin