Ein System für die Online-Erkennung handgeschriebener mathematischer Formeln

Title and Table of Contents 1.Introduction 2. Related Work 3. Preprocessing Techniques for On-Line Handwriting 4. Classification of On-Line Handwritten Symbols 5. Structural Analysis of Mathematical Expressions 6. An Editor for On-Line Handwritten Mathematical Expressions 7. Conclusion BibliographyThis work presents a system for the recognition of on-line handwritten mathematical formulas. The system consists of two main stages: Classification of isolated on-line handwritten symbols and the analysis of spatial relationships among them. We propose a system for the recognition of isolated on-line handwritten characters which is based on support vector classification. We also propose a suitable representation for strokes and symbols which is used to improve the classification rates of the classifier. Our experiments show that our classifier achieved better classification rates in comparison to other popular classification techniques. This could be accomplished by extensive preprocessing of the data and by parameter selection for the support vector classification. We propose a new structural analysis method for the recognition of on-line handwritten mathematical expressions based on a minimum spanning tree construction and symbol dominance. Our method addresses important layout problems frequently encountered in on-line handwritten formula-recognition systems. Our method also aims to handle input as naturally as possible, i.e. using the usual mathematical conventions, without restrictions in the order the symbols are written. Our method handles symbols with non-standard layout, like \sideset{^{*}_{*}}{^{*}_{*}}\prod, as well as tabular layouts, e.g. matrices. Our system for the recognition of on- line handwritten mathematical expressions is used in the Electronic Chalkboard (E-Chalk), a multimedia system for distance-teaching.Die vorliegende Arbeit stellt ein System für die Online-Erkennung handgeschriebener mathematischer Formeln vor. Das System besteht aus zwei verschiedenen Komponenten, einem Klassifikator einzelner handgeschriebener Online-Symbole und einem Analysator mathematischer Strukturen. Die Erkennung der einzelnen Symbole erfolgt mittels Support-Vektor-Maschinen. Aus unserer Experimenten ergab sich, dass unser Klassifikator gegenüber den klassischen Techniken bessere Erkennungsraten erreichte. Diese Ergebnisse wurden durch intensive Vorbearbeitung der Symbole und Suche optimaler Parameter ermöglicht. Unsere Experimente lassen den Schluss zu, dass Support-Vektor-Maschinen den Kompromiss zwischen Trainingszeit und Klassifikationsrate optimieren. In der Arbeit wird eine neue Methode für die Online-Strukturanalyse handgeschriebener mathematischer Ausdrücke besprochen, die sich auf der Aufbau eines minimalen spannenden Baums und Symboldominanz basiert. Diese Technik ermöglicht eine natürliche Eingabe der mathematischen Formeln, d.h., die Symbole und Formeln werden ohne Beschränkungen nach der üblichen mathematischen Notation geschrieben. Unsere Methode lässt sich einfach erweitern, um andere mathematische Strukturen zu erkennen, z.B. Matrizen und andere ungewöhnliche Strukturen, wie die in der LaTeX-Sprache definierte Struktur sideset{^{*}_{*}}{^{*}_{*}} Unser Erkennungssystem wurde in der Programmiersprache Java implementiert und ist das Standard- Formelerkennungssystem des E-Kreide Systems

MathLet v3: recognizing handwritten mathematical expressions

This thesis presents MathLet v3 which is the third version of a system developed to recognize handwritten mathematical expressions. Previous versions were developed by Hakan Büyükbayrak and Mehmet Çelik. MathLet v3 implements two steps to recognize handwritten mathematical expressions; symbol recognition and parsing. In the symbol recognition step, two classifiers are combined. One of these classifiers uses online features while the other one uses offine features. Both classifiers return probability distributions over classes. In the parsing step, probability distributions are used to increase time performance of MathLet v3. Moreover, parallel programming is used in parsing phase. Special handling approach for mistaken symbols is also implemented in the parsing step. MathLet v3 has four applications and two of them can be accessed through the Web. Users write mathematical expressions or upload existing InkML les which contain mathematical expression and get recognition results for them through the Web by using these applications. MathLet has been participating in a competition named CROHME since 2011. The evaluation results of MathLet in CROHME show that the accuracy of MathLet has increased from 0.55% to 8.35% starting from 2011, although recognition task becomes more di cult each year. In addition to accuracy improvements, experiments made in order to measure the time performance of MathLet v3 show that MathLet v3 has become faster

Symbol detection in online handwritten graphics using Faster R-CNN

Symbol detection techniques in online handwritten graphics (e.g. diagrams and mathematical expressions) consist of methods specifically designed for a single graphic type. In this work, we evaluate the Faster R-CNN object detection algorithm as a general method for detection of symbols in handwritten graphics. We evaluate different configurations of the Faster R-CNN method, and point out issues relative to the handwritten nature of the data. Considering the online recognition context, we evaluate efficiency and accuracy trade-offs of using Deep Neural Networks of different complexities as feature extractors. We evaluate the method on publicly available flowchart and mathematical expression (CROHME-2016) datasets. Results show that Faster R-CNN can be effectively used on both datasets, enabling the possibility of developing general methods for symbol detection, and furthermore, general graphic understanding methods that could be built on top of the algorithm.Comment: Submitted to DAS-201

Multi-Scale Attention with Dense Encoder for Handwritten Mathematical Expression Recognition

Handwritten mathematical expression recognition is a challenging problem due to the complicated two-dimensional structures, ambiguous handwriting input and variant scales of handwritten math symbols. To settle this problem, we utilize the attention based encoder-decoder model that recognizes mathematical expression images from two-dimensional layouts to one-dimensional LaTeX strings. We improve the encoder by employing densely connected convolutional networks as they can strengthen feature extraction and facilitate gradient propagation especially on a small training set. We also present a novel multi-scale attention model which is employed to deal with the recognition of math symbols in different scales and save the fine-grained details that will be dropped by pooling operations. Validated on the CROHME competition task, the proposed method significantly outperforms the state-of-the-art methods with an expression recognition accuracy of 52.8% on CROHME 2014 and 50.1% on CROHME 2016, by only using the official training dataset

Math Search for the Masses: Multimodal Search Interfaces and Appearance-Based Retrieval

We summarize math search engines and search interfaces produced by the Document and Pattern Recognition Lab in recent years, and in particular the min math search interface and the Tangent search engine. Source code for both systems are publicly available. "The Masses" refers to our emphasis on creating systems for mathematical non-experts, who may be looking to define unfamiliar notation, or browse documents based on the visual appearance of formulae rather than their mathematical semantics.Comment: Paper for Invited Talk at 2015 Conference on Intelligent Computer Mathematics (July, Washington DC