A Sketch-Based Educational System for Learning Chinese Handwriting

Learning Chinese as a Second Language (CSL) is a difficult task for students in English-speaking countries due to the large symbol set and complicated writing techniques. Traditional classroom methods of teaching Chinese handwriting have major drawbacks due to human experts’ bias and the lack of assessment on writing techniques. In this work, we propose a sketch-based educational system to help CSL students learn Chinese handwriting faster and better in a novel way. Our system allows students to draw freehand symbols to answer questions, and uses sketch recognition and AI techniques to recognize, assess, and provide feedback in real time. Results have shown that the system reaches a recognition accuracy of 86% on novice learners’ inputs, higher than 95% detection rate for mistakes in writing techniques, and 80.3% F-measure on the classification between expert and novice handwriting inputs

Sketchography - Automatic Grading of Map Sketches for Geography Education

Geography is a vital classroom subject that teaches students about the physical features of the planet we live on. Despite the importance of geographic knowledge, almost 75% of 8th graders scored below proficient in geography on the 2014 National Assessment of Educational Progress. Sketchography is a pen-based intelligent tutoring system that provides real-time feedback to students learning the locations, directions, and topography of rivers around the world. Sketchography uses sketch recognition and artificial intelligence to understand the user’s sketched intentions. As sketches are inherently messy, and even the most expert geographer will draw only a close approximation of the river’s flow, data has been gathered from both novice and expert sketchers. This data, in combination with professors’ grading rubrics and statistically driving AI-algorithms, provide real-time automatic grading that is similar to a human grader’s score. Results show the system to be 94.64% accurate compared to human grading

Chinese Calligraphist: A Sketch Based Learning Tool for Learning Written Chinese

Learning Chinese as a foreign language is becoming more and more popular in western countries, however it is also very hard to be proficient, especially in writing. The involvement of the teachers in the process of learning Chinese writing is extremely necessary because they can give timely critiques and feedbacks as well as correct the students’ bad writing habits. However, it is inadequate and inefficient of the large class capacity therefore it is urgent and necessary to design a computer-based system to help students in practice Chinese writing, correct their bad writing habits early, and give feedback personally. The current written Chinese learning tools such as online tutorials emphasize writing rules including stroke order, but it could not provide practicing sessions and feedback. Hashigo, a novel CALL (Computer Assisted Language Learning) system, introduced the concept of sketch-based learning, but it’s low level recognizer is not proper for Chinese character domain. Therefore in order to help western students learn Chinese with better understanding, we adopted LADDER description language, machine learning techniques, and sketch recognition algorithms to improve handwritten Chinese stroke recognition rate. With our multilayer perceptron recognizer, it improved Chinese stroke recognition accuracy by 15.7% than the average of the four basic recognizer. In feature selection study we found that the most important features were “the aspect of the bounding box”, and the “density metrics”, and “curviness”. We chose 8 most important features after the recursive selecting stabilized. We discovered that in most situations, feature recognition is more important than template recognition. Since the writing technique is emphasized while they are taught, only 2 templates is enough. It worked as well as 20 templates, which improved recognition speed dramatically. In conclusion, in this thesis our contribution is that we (1) proposed a natural way to describe Chinese characters; (2) implemented a hierarchical Chinese character recognizer combining LADDER with the multilayer perceptron low level recognizer; (3) analyzed the performance of different recognition schemes; (4) designed a sketch-based Chinese writing learning tool, Chinese Calligraphist; and (5) find the best feature combination to recognize Chinese strokes while improving the recognition accuracy

Integration of sketch-based ideation and 3D modeling with CAD systems

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.This thesis is concerned with the study of how sketch-based systems can be improved to enhance idea generation process in conceptual design stage. It is also concerned with achieving a kind of integration between sketch-based systems and CAD systems to complete the digitization of the design process as sketching phase is still not integrated with other phases due to the different nature of it and the incomplete digitization of sketching phase itself. Previous studies identified three main related issues: sketching process, sketch-based modeling, and the integration between the digitized design phases. Here, the thesis is motivated from the desire to improve sketch-based modeling to support idea generation process but unlike previous studies that only focused on the technical or drawing part of sketching, this thesis attempts to concentrate more on the mental part of the sketching process which play a key role in developing ideas in design. Another motivation of this thesis is to produce a kind of integration between sketch-based systems and CAD systems to enable 3D models produced by sketching to be edited in detailed design stage. As such, there are two main contributions have been addressed in this thesis. The first contribution is the presenting of a new approach in designing sketch-based systems that enable more support for idea generation by separating thinking and developing ideas from the 3D modeling process. This kind of separation allows designers to think freely and concentrate more on their ideas rather than 3D modeling. the second contribution is achieving a kind of integration between gesture-based systems and CAD systems by using an IGES file in exchanging data between systems and a new method to organize data within the file in an order that make it more understood by feature recognition embedded in commercial CAD systems.This study is funded by the Ministry of Higher Education of Egypt

Geometristen muotojen reaaliaikainen tunnistus

Kynä- ja kosketuskäyttöliittymät vaativat toimiakseen tehokasta ja tarkkaa hahmontunnistusta. Tässä työssä esitellään reaaliaikaisen hahmontunnistuksen käsitteistöä, yleisiä menetelmiä ja aikaisempaa tutkimusta. Lyhyesti käsitellään eri tutkimusryhmien esittämiä hahmontunnistusjärjestelmiä. Lisäksi esitellään geometrisiin piirteisiin perustuva hahmontunnistusjärjestelmä. Työ antaa yksityiskohtaiset kuvaukset piirtoviivan esiprosessointi- ja piirteenirrotusalgoritmeista sekä hahmoluokittelumenetelmästä. Lisäksi kuvaillaan hahmontunnistusheuristiikka kahdelle yksinkertaiselle muodolle (nuoli ja tähti). Joukko koehenkilöitä käytti työssä toteutettua graa_sta käyttöliittymää, minkä tuloksena saatiin realistiset tulokset järjestelmän laskennallisesta suorituskyvystä ja tarkkuudesta: toteutettu järjestelmä on laskennallisesti nopea mutta tunnistustarkkuus monitulkintainen. Lopuksi pohditaan valitun lähestymistavan ongelmia ja rajoitteita.Effective sketch recognition is the basis for pen and touch-based human-computer interfaces. In this thesis the concepts, common methods and earlier work in the research area of online symbol recognition are presented. A set of shape recognition approaches proposed in the past by various research teams are briefly introduced. An online shape recognizer using global geometric features is described. The preprocessing and feature extraction algorithms as well as the shape classification method are described in detail. Recognition heuristics for two simple shapes (arrow and star) are suggested. A graphical user interface was implemented and a group of subjects employed to obtain realistic results of the computational performance and recognition accuracy of the system: the implemented system performs fast but the results on the recognition accuracy were ambiguous. Finally, the problems and restrictions of the approach are discussed

Segmenting Hand-Drawn Strokes

Pen-based interfaces utilize sketch recognition so users can create and interact with complex, graphical systems via drawn input. In order for people to freely draw within these systems, users' drawing styles should not be constrained. The low-level techniques involved with sketch recognition must then be perfected, because poor low-level accuracy can impair a user's interaction experience. Corner finding, also known as stroke segmentation, is one of the first steps to free-form sketch recognition. Corner finding breaks a drawn stroke into a set of primitive symbols such as lines, arcs, and circles, so that the original stoke data can be transformed into a more machine-friendly format. By working with sketched primitives, drawn objects can then be described in a visual language, noting what primitive shapes have been drawn and the shapes? geometric relationships to each other. We present three new corner finding techniques that improve segmentation accuracy. Our first technique, MergeCF, is a multi-primitive segmenter that splits drawn strokes into primitive lines and arcs. MergeCF eliminates extraneous primitives by merging them with their neighboring segments. Our second technique, ShortStraw, works with polyline-only data. Polyline segments are important since many domains use simple polyline symbols formed with squares, triangles, and arrows. Our ShortStraw algorithm is simple to implement, yet more powerful than previous polyline work in the corner finding literature. Lastly, we demonstrate how a combination technique can be used to pull the best corner finding results from multiple segmentation algorithms. This combination segmenter utilizes the best corners found from other segmentation techniques, eliminating many false negatives (missed primitive segmentations) from the final, low-level results. We will present the implementation and results from our new segmentation techniques, showing how they perform better than related work in the corner finding field. We will also discuss limitations of each technique, how we have sought to overcome those limitations, and where we believe the sketch recognition subfield of corner finding is headed

Exploring Methods for Holistically Improving Drawing Ability With Artificial Intelligence

Drawing is a highly useful skill that can make people better at solving problems, communicating ideas to others, collaborating, and producing more creative and novel ideas. It can be a difficult skill to master for many people, however. Like any learned skill, it requires many hours of practice for noticeable improvement, and sufficient motivation is also necessary to keep practicing consistently over a period of time. Utilizing sketch recognition and other forms of artificial intelligence to assist in learning to draw may facilitate the necessary improvements in self-efficacy and motivation students need to improve their drawing ability. While similar tools have been explored, there has been little to no effort at designing a truly holistic approach for teaching drawing skills that includes the basic fundamentals and building blocks for drawing any 3-dimensional object. This dissertation explored the potential of an intelligent tutoring system for teaching drawing skills called SketchTivity along with various other technology probes focused on drawing. We found evidence that individuals could build confidence, build motivation, make measurable improvements to drawing ability, and reduce fixation when ideating concepts through the various studies we conducted. We developed a flexible perspective accuracy recognition algorithm that can help individuals learn perspective. In interviews with students and teachers who used SketchTivity we discovered benefits and limitations of the system. Students were engaged by the interactive lessons, motivated by the gameplay, and saw it as a great warm-up tool. Meanwhile instructors loved that the system could offload grading tasks for them. We hope the nuances of this potential will inform the future development and promise of the approaches described in this dissertation along with similar approaches to impact education at large

Pen-based Methods For Recognition and Animation of Handwritten Physics Solutions

There has been considerable interest in constructing pen-based intelligent tutoring systems due to the natural interaction metaphor and low cognitive load afforded by pen-based interaction. We believe that pen-based intelligent tutoring systems can be further enhanced by integrating animation techniques. In this work, we explore methods for recognizing and animating sketched physics diagrams. Our methodologies enable an Intelligent Tutoring System (ITS) to understand the scenario and requirements posed by a given problem statement and to couple this knowledge with a computational model of the student\u27s handwritten solution. These pieces of information are used to construct meaningful animations and feedback mechanisms that can highlight errors in student solutions. We have constructed a prototype ITS that can recognize mathematics and diagrams in a handwritten solution and infer implicit relationships among diagram elements, mathematics and annotations such as arrows and dotted lines. We use natural language processing to identify the domain of a given problem, and use this information to select one or more of four domain-specific physics simulators to animate the user\u27s sketched diagram. We enable students to use their answers to guide animation behavior and also describe a novel algorithm for checking recognized student solutions. We provide examples of scenarios that can be modeled using our prototype system and discuss the strengths and weaknesses of our current prototype. Additionally, we present the findings of a user study that aimed to identify animation requirements for physics tutoring systems. We describe a taxonomy for categorizing different types of animations for physics problems and highlight how the taxonomy can be used to define requirements for 50 physics problems chosen from a university textbook. We also present a discussion of 56 handwritten solutions acquired from physics students and describe how suitable animations could be constructed for each of them