Symbol detection in online handwritten graphics using Faster R-CNN

Symbol detection techniques in online handwritten graphics (e.g. diagrams and mathematical expressions) consist of methods specifically designed for a single graphic type. In this work, we evaluate the Faster R-CNN object detection algorithm as a general method for detection of symbols in handwritten graphics. We evaluate different configurations of the Faster R-CNN method, and point out issues relative to the handwritten nature of the data. Considering the online recognition context, we evaluate efficiency and accuracy trade-offs of using Deep Neural Networks of different complexities as feature extractors. We evaluate the method on publicly available flowchart and mathematical expression (CROHME-2016) datasets. Results show that Faster R-CNN can be effectively used on both datasets, enabling the possibility of developing general methods for symbol detection, and furthermore, general graphic understanding methods that could be built on top of the algorithm.Comment: Submitted to DAS-201

Integration of Shape Context and Neural Networks for Symbol Recognition

International audienceUsing shape matching within a k–nearest neighbor approach, shape context descriptor has been applied in several classification problems with outstanding results. However , the application of this frame work on large datasets or online scenarios is challenging due to its computational cost. To over come this limitations, we evaluate the use of shape context as input features for neural networks. We test the proposed method in a problem of recognition of handwritten mathematical symbols. For a total of 75 classes of symbols, we obtained a recognition rate of 89.8%, comparable with a k–nearest neighbor approach, but with reduced time complexity.Les descripteurs de contexte de formes ont été utilisés comme caractéristiques dans les classifieurs k–plus proc hes voisins avec des résultats remarquables. Néanmoins, l’utilisation de cette approche sur de grosses bases de symboles ou dans des contextes applicatifs à la volée reste difficile à cause de sa complexité calculatoire . Pour dépasser ces limitations, nous proposons l’utilisation des descripteurs de contexte de formes avec des réseaux de neurones au lieu de l’approche k–ppv. Nous évaluons la méthode proposée dans un problème de reconnaissances de symboles mathématiques en–ligne. Pour un total de 75 classes de symboles, un taux de reconnaissance est de 89,8% est obtenu, ce qui est comparable aux résultats de l’approche initiale mais avec une complexité nettement diminuée