Multi-domain sketch understanding

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2004.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Includes bibliographical references (p. 121-128).by Christine J. Alvarado.Ph.D

Towards a Programmer’s Apprentice (Again)

Programmers are loathe to interrupt their workflow to document their design rationale, leading to frequent errors when software is modified—often much later and by different programmers. A Pro- grammer’s Assistant could interact with the programmer to capture and preserve design rationale, in a natural way that would make rationale capture “cost less than it’s worth”, and could also detect common flaws in program design. Such a programmer’s assistant was not practical when it was first proposed decades ago, but advances over the years make now the time to revisit the concept, as our prototype shows.This work was supported by the Center for Brains, Minds and Machines (CBMM), funded by NSF STC award CCF-1231216

On-line hand-drawn electric circuit diagram recognition using 2D dynamic programming

9 pagesInternational audienceIn order to facilitate sketch recognition, most online existing works assume that people will not start to draw a new symbol before the current one has been finished. We propose in this paper a method that relaxes this constraint. The proposed methodology relies on a two-dimensional dynamic programming (2D-DP) technique allowing symbol hypothesis generation, which can correctly segment and recognize interspersed symbols. In addition, as discriminative classifiers usually have limited capability to reject outliers, some domain specific knowledge is included to circumvent those errors due to untrained patterns corresponding to erroneous segmentation hypotheses. With a point-level measurement, the experiment shows that the proposed novel approach is able to achieve an accuracy of more than 90 percent

Tolerance Zone-Based Grouping Method for Online Multiple Overtracing Freehand Sketches

Multiple overtracing strokes are common drawing behaviors in freehand sketching; that is, additional strokes are often drawn repeatedly over the existing ones to add more details. This paper proposes a method based on stroke-tolerance zones to group multiple overtraced strokes which are drawn to express a 2D primitive, aiming to convert online freehand sketches into 2D line drawings, which is a base for further 3D reconstruction. Firstly, after the user inputs a new stroke, a tolerance zone around the stroke is constructed by reference to its polygonal approximation points obtained from the stroke preprocessing. Then, the input strokes are divided into stroke groups, each representing a primitive through the stroke grouping process based on the overtraced ratio of two strokes. At last, each stroke group is fitted into one or more 2D geometric primitives including line segments, polylines, ellipses, and arcs. The proposed method groups two strokes together based on their screen-space proximity directly instead of classifying and fitting them firstly, so that it can group strokes of arbitrary shapes. A sketch-recognition prototype system has been implemented to test the effectiveness of the proposed method. The results showed that the proposed method could support online multiple overtracing freehand sketching with no limitation on drawing sequence, but it only deals with strokes with relatively high overtraced ratio

Interactive interpretation of structured documents: Application to the recognition of handwritten architectural plans

International audienceThis paper addresses a whole architecture, including the IMISketch method. IMISketch method incorporates two aspects: document analysis and interactivity. This paper describes a global vision of all the parts of the project. IMISketch is a generic method for an interactive interpretation of handwritten sketches. The analysis of complex documents requires the management of uncertainty. While, in practice the similar methods often induce a large combinatorics, IMISketch method presents several optimization strategies to reduce the combinatorics. The goal of these optimizations is to have a time analysis compatible with user expectations. The decision process is able to solicit the user in the case of strong ambiguity: when it is not sure to make the right decision, the user explicitly validates the right decision to avoid a fastidious a posteriori verification phase due to propagation of errors.This interaction requires solving two major problems: how interpretation results will be presented to the user, and how the user will interact with analysis process. We propose to study the effects of those two aspects. The experiments demonstrate that (i) a progressive presentation of the analysis results, (ii) user interventions during it and (iii) the user solicitation by the analysis process are an efficient strategy for the recognition of complex off-line documents.To validate this interactive analysis method, several experiments are reported on off-line handwritten 2D architectural floor plans

Picturephone: A game for sketch data capture

ABSTRACT This article introduces Picturephone, a sketch-based game for collecting data on how people make and describe sketches. The approach is based on the technique of human computation, where players provide information about drawings in exchange for entertainment. The system collects raw sketch input and associates it with human-provided text descriptions. Researchers may browse and download this data for their own purposes. The proposed sketching game has users describe drawings textually in one phase followed by another phase of drawing based on a text description. To score points, users must reconstruct a drawing based on a text description or vice-versa. A third phase asks users to judge the work of other players, which awards points appropriately. The Picturephone game system architecture is briefly described, followed by a discussion of future work on sketching games as a platform for researching sketch recognition and interaction

Semantic representation of digital ink in the classroom learning partner

Thesis (M. Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2006.Includes bibliographical references (p. 59-63).The research presented in this thesis addresses a critical issue in the introduction of new tablet-PC-based educational technology in the classroom: interpretation and semantic representation of digital ink. The teaching paradigm being investigated is that of in-class exercises, which have proven beneficial to student learning. Recent advances in educational technology support such exercises by means of distributed wireless presentation systems. Such systems have proven successful, but are limited in their scope because they rely on digital ink as the communication medium between an instructor and his or her students. The work done in this thesis extends the use of such systems and makes the following contributions towards the creation of a learning partner in the classroom: * an ink interpreter capable of text and arrow interpretation which can rival PRS on multiple choice and true-false questions * a framework for sketch and text interpretation inspired from state of the art research in handwriting and sketch interpretation * an infrastructure necessary for creating in-class exercises as part of Classroom Presenter, interpreting student answers, and aggregating them.by Michel A. Rbeiz.M.Eng

Using context to resolve ambiguity in sketch understanding

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2006.Includes bibliographical references (p. 48-49).This thesis presents methods for improving sketch understanding, without knowledge of a domain or the particular symbols being used in the sketch, by recognizing common sketch primitives. We address two issues that complicate recognition in its early stages. The first is imprecision and inconsistencies within a single sketch or between sketches by the same person. This problem is addressed with a graphical model approach that incorporates limited knowledge of the surrounding area in the sketch to better decide the intended meaning of a small piece of the sketch. The second problem, that of variation among sketches with different authors, is addressed by forming groups from the authors in training set. We apply these methods to the problem of finding corners, a common sketch primitive, and describe how this can translate into better recognition of entire sketches. We also describe the collection of a data set of sketches.by Sonya J. Cates.S.M