Integrating Multiple Sketch Recognition Methods to Improve Accuracy and Speed

Sketch recognition is the computer understanding of hand drawn diagrams. Recognizing sketches instantaneously is necessary to build beautiful interfaces with real time feedback. There are various techniques to quickly recognize sketches into ten or twenty classes. However for much larger datasets of sketches from a large number of classes, these existing techniques can take an extended period of time to accurately classify an incoming sketch and require significant computational overhead. Thus, to make classification of large datasets feasible, we propose using multiple stages of recognition. In the initial stage, gesture-based feature values are calculated and the trained model is used to classify the incoming sketch. Sketches with an accuracy less than a threshold value, go through a second stage of geometric recognition techniques. In the second geometric stage, the sketch is segmented, and sent to shape-specific recognizers. The sketches are matched against predefined shape descriptions, and confidence values are calculated. The system outputs a list of classes that the sketch could be classified as, along with the accuracy, and precision for each sketch. This process both significantly reduces the time taken to classify such huge datasets of sketches, and increases both the accuracy and precision of the recognition

Integrating Multiple Sketch Recognition Methods to Improve Accuracy and Speed

Sketch recognition is the computer understanding of hand drawn diagrams. Recognizing sketches instantaneously is necessary to build beautiful interfaces with real time feedback. There are various techniques to quickly recognize sketches into ten or twenty classes. However for much larger datasets of sketches from a large number of classes, these existing techniques can take an extended period of time to accurately classify an incoming sketch and require significant computational overhead. Thus, to make classification of large datasets feasible, we propose using multiple stages of recognition. In the initial stage, gesture-based feature values are calculated and the trained model is used to classify the incoming sketch. Sketches with an accuracy less than a threshold value, go through a second stage of geometric recognition techniques. In the second geometric stage, the sketch is segmented, and sent to shape-specific recognizers. The sketches are matched against predefined shape descriptions, and confidence values are calculated. The system outputs a list of classes that the sketch could be classified as, along with the accuracy, and precision for each sketch. This process both significantly reduces the time taken to classify such huge datasets of sketches, and increases both the accuracy and precision of the recognition

Natural error correction techniques for sketch recognition

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2013.Cataloged from PDF version of thesis.Includes bibliographical references (p. 55-56).Over the past few years, a plethora of tablet devices has made it very easy for users to input information by sketching as if on paper. In addition, sketch recognition systems help users convert these sketches into information that the computer understands. While lots of work has been done in developing better sketch recognizers, very little work has previously been done on how to edit the sketch once it's been drawn, whether the error is the user's or the sketch recognizer's. In response, we developed and studied intuitive methods of interacting with a sketch recognition system to correct errors made by both the recognizer and the user. The editor allows users to click and lasso to select parts of the sketch, label the selected strokes, erase by scribbling over strokes, and even overwrite errors. Letting users provide feedback to the sketch recognizer helps improve the accuracy of the sketch as well as allows the sketch recognizer's performance to improve over time.by Danica H. Chang.S.M

Evaluation of Conceptual Sketches on Stylus-Based Devices

Design Sketching is an important tool for designers and creative professionals to express their ideas and thoughts onto visual medium. Being a very critical and versatile skill for engineering students, this course is often taught in universities on pen and paper. However, this traditional pedagogy is limited by the availability of human instructors for their feedback. Also, students having low self-efficacy do not learn efficiently in traditional learning environment. Using intelligent interfaces this problem can be solved where we try to mimic the feedback given by an instructor and assess the student drawn sketches to give them insight of the areas they need to improve on. PerSketchTivity is an intelligent tutoring system which allows students to practice their drawing fundamentals and gives them real-time assessment and feedback. This research deals with finding the evaluation metrics that will enable us to grade students from their sketch data. There are seven metrics that we will work with to analyse how each of them contribute in deciding the quality of the sketches. The main contribution of this research is to identify the features of the sketch that can distinguish a good quality sketch from a poor one and design a grading metric for the sketches that can give a final score between 0 and 1 to the user sketches. Using these obtained features and our grading metric method, we grade all the sketches of students and experts

A Fine Motor Skill Classifying Framework to Support Children's Self-Regulation Skills and School Readiness

Children’s self-regulation skills predict their school-readiness and social behaviors, and assessing these skills enables parents and teachers to target areas for improvement or prepare children to enter school ready to learn and achieve. Assessing these skills enables parents and teachers to target areas for improvement or prepare children to enter school ready to learn and achieve. To assess children’s fine motor skills, current educators are assessing those skills by either determining their shape drawing correctness or measuring their drawing time durations through paper-based assessments. However, the methods involve human experts manually assessing children’s fine motor skills, which are time consuming and prone to human error and bias. As there are many children that use sketch-based applications on mobile and tablet devices, computer-based fine motor skill assessment has high potential to solve the limitations of the paper-based assessments. Furthermore, sketch recognition technology is able to offer more detailed, accurate, and immediate drawing skill information than the paper-based assessments such as drawing time or curvature difference. While a number of educational sketch applications exist for teaching children how to sketch, they are lacking the ability to assess children’s fine motor skills and have not proved the validity of the traditional methods onto tablet-environments. We introduce our fine motor skill classifying framework based on children’s digital drawings on tablet-computers. The framework contains two fine motor skill classifiers and a sketch-based educational interface (EasySketch). The fine motor skill classifiers contain: (1) KimCHI: the classifier that determines children’s fine motor skills based on their overall drawing skills and (2) KimCHI2: the classifier that determines children’s fine motor skills based on their curvature- and corner-drawing skills. Our fine motor skill classifiers determine children’s fine motor skills by generating 131 sketch features, which can analyze their drawing ability (e.g. DCR sketch feature can determine their curvature-drawing skills). We first implemented the KimCHI classifier, which can determine children’s fine motor skills based on their overall drawing skills. From our evaluation with 10- fold cross-validation, we found that the classifier can determine children’s fine motor skills with an f-measure of 0.904. After that, we implemented the KimCHI2 classifier, which can determine children’s fine motor skills based on their curvature- and corner-drawing skills. From our evaluation with 10-fold cross-validation, we found that the classifier can determine children’s curvature-drawing skills with an f-measure of 0.82 and corner-drawing skills with an f-measure of 0.78. The KimCHI2 classifier outperformed the KimCHI classifier during the fine motor skill evaluation. EasySketch is a sketch-based educational interface that (1) determines children’s fine motor skills based on their drawing skills and (2) assists children how to draw basic shapes such as alphabet letters or numbers based on their learning progress. When we evaluated our interface with children, our interface determined children’s fine motor skills more accurately than the conventional methodology by f-measures of 0.907 and 0.744, accordingly. Furthermore, children improved their drawing skills from our pedagogical feedback. Finally, we introduce our findings that sketch features (DCR and Polyline Test) can explain children’s fine motor skill developmental stages. From the sketch feature distributions per each age group, we found that from age 5 years, they show notable fine motor skill development

Rethinking Pen Input Interaction: Enabling Freehand Sketching Through Improved Primitive Recognition

Online sketch recognition uses machine learning and artificial intelligence techniques to interpret markings made by users via an electronic stylus or pen. The goal of sketch recognition is to understand the intention and meaning of a particular user's drawing. Diagramming applications have been the primary beneficiaries of sketch recognition technology, as it is commonplace for the users of these tools to rst create a rough sketch of a diagram on paper before translating it into a machine understandable model, using computer-aided design tools, which can then be used to perform simulations or other meaningful tasks. Traditional methods for performing sketch recognition can be broken down into three distinct categories: appearance-based, gesture-based, and geometric-based. Although each approach has its advantages and disadvantages, geometric-based methods have proven to be the most generalizable for multi-domain recognition. Tools, such as the LADDER symbol description language, have shown to be capable of recognizing sketches from over 30 different domains using generalizable, geometric techniques. The LADDER system is limited, however, in the fact that it uses a low-level recognizer that supports only a few primitive shapes, the building blocks for describing higher-level symbols. Systems which support a larger number of primitive shapes have been shown to have questionable accuracies as the number of primitives increase, or they place constraints on how users must input shapes (e.g. circles can only be drawn in a clockwise motion; rectangles must be drawn starting at the top-left corner). This dissertation allows for a significant growth in the possibility of free-sketch recognition systems, those which place little to no drawing constraints on users. In this dissertation, we describe multiple techniques to recognize upwards of 18 primitive shapes while maintaining high accuracy. We also provide methods for producing confidence values and generating multiple interpretations, and explore the difficulties of recognizing multi-stroke primitives. In addition, we show the need for a standardized data repository for sketch recognition algorithm testing and propose SOUSA (sketch-based online user study application), our online system for performing and sharing user study sketch data. Finally, we will show how the principles we have learned through our work extend to other domains, including activity recognition using trained hand posture cues

Eye Tracking Methods for Analysis of Visuo-Cognitive Behavior in Medical Imaging

Predictive modeling of human visual search behavior and the underlying metacognitive processes is now possible thanks to significant advances in bio-sensing device technology and machine intelligence. Eye tracking bio-sensors, for example, can measure psycho-physiological response through change events in configuration of the human eye. These events include positional changes such as visual fixation, saccadic movements, and scanpath, and non-positional changes such as blinks and pupil dilation and constriction. Using data from eye-tracking sensors, we can model human perception, cognitive processes, and responses to external stimuli. In this study, we investigated the visuo-cognitive behavior of clinicians during the diagnostic decision process for breast cancer screening under clinically equivalent experimental conditions involving multiple monitors and breast projection views. Using a head-mounted eye tracking device and a customized user interface, we recorded eye change events and diagnostic decisions from 10 clinicians (three breast-imaging radiologists and seven Radiology residents) for a corpus of 100 screening mammograms (comprising cases of varied pathology and breast parenchyma density). We proposed novel features and gaze analysis techniques, which help to encode discriminative pattern changes in positional and non-positional measures of eye events. These changes were shown to correlate with individual image readers' identity and experience level, mammographic case pathology and breast parenchyma density, and diagnostic decision. Furthermore, our results suggest that a combination of machine intelligence and bio-sensing modalities can provide adequate predictive capability for the characterization of a mammographic case and image readers diagnostic performance. Lastly, features characterizing eye movements can be utilized for biometric identification purposes. These findings are impactful in real-time performance monitoring and personalized intelligent training and evaluation systems in screening mammography. Further, the developed algorithms are applicable in other application domains involving high-risk visual tasks

Analytical Variations – Eight Critical Essays on Applied Music Theory

This book gives a critical account of various methods used in music analysis. In the first chapter, a number of current approaches such as semiotics, musical implications, Schenkerian analysis, and generative theory are demonstrated on Mozart’s K. 331 theme. Five essays deal with important concepts in music analysis: ambiguity, formal proportions, and similarity within and between works. A further chapter provides a discussion of probability, kinship, and influence – decisive criteria when judging musical plagiarism. The last essay, studying a piece by Schubert, sifts the prospects of deciphering a composer’s sexual leanings from his music

Black and white in ink : discourses of resistance in South African cartooning, 1985-1994.

Thesis (M.A.)-University of KwaZulu-Natal, Durban, 2004.In the last decade of apartheid (1985-1994), South African cartoonists demonstrated a range of responses to the political imperatives of the day. While some worked in support of the status quo, the cartoonists who are the subject of this study opposed it. Like practitioners in other areas of cultural activity during this period, oppositional cartoonists were passionately engaged with the political process and participated in the articulation and dissemination of discourses of resistance. This study situates South African cartooning both in the context of South African resistance discourse, and in the historical and discursive context of cartooning as a form of international popular culture. It presents an argument as to how cartooning should be defined and studied - as a cluster of signifying practices that produce a range of forms in a variety of media. In terms of this definition, anti-apartheid cartooning in South Africa is identified as a specific historical category, within which distinct streams of cartooning are identified. The study locates the various activities of South African cartooning within these streams, and examines the ideological and educational functions they performed during the 1985-1994 period. The study positions cartooning within the broad theoretical field of cultural and media studies, and examines some theoretical problems that are specific to the analysis of visual culture. A language of exposition appropriate to the study of cartooning is developed, borrowing terms from the sometimes widely variant traditions of art history, literary criticism and cultural studies. A methodology for the interpretation of symbolic forms is derived from the work of British cultural theorist, John B. Thompson (1990), whereby selected cartooning texts are subjected to a combination of textual interpretation, socio-historical analysis and discursive analysis, reinforced by insights derived from conversations with 15 selected South African cartoonists. Textual analysis of selected cartooning texts from the 1985-1994 period clearly demonstrates that oppositional cartoonists gave visual expression to discourses of resistance that existed in the anti-apartheid movement, and amongst the broader public, at that time. In so doing, they contributed to the disruption of the hegemony of the apartheid state, to the legitimation of the anti-apartheid struggle and to the provision of symbols and icons that ordinary South Africans were able to utilise in 'rethinking' their own lives in relation to the demands of a rapidly transforming society

The Austronesian languages

This is a revised edition of the 2009 The Austronesian languages, which was published as a paperback in the then Pacific Linguistics series (ISBN 9780858836020). This revision includes typographical corrections, an improved index, and various minor content changes. The release of the open access edition serves to meet the strong ongoing demand for this important handbook, of which only 200 copies of the first edition were printed. This is the first single-authored book that attempts to describe the Austronesian language family in its entirety. Topics covered include: the physical and cultural background, official and national languages, largest and smallest languages in all major geographical regions, language contact, sound systems, linguistic palaeontology, morphology, syntax, the history of scholarship on Austronesian languages, and a critical assessment of the reconstruction of Proto Austronesian phonology.Australian National University, College of Asia and the Pacifi