The MUMTDB dataset for evaluating simultaneous composition of structured documents in a multi-user and multi-touch environment

International audienceWe propose in this paper a new online MultiUser Multi-Touch handwritten diagram DataBase (MUMTDB) for evaluating recognition systems under the multiuser situation. The data is collected according to two predefined mind map scenarios which contains 9 classes of graphical symbols. Each scenario is completed by involving two users at the same time. Since the users are given freedom to draw the symbols as they want, the dataset contains a diversity of multi-stroke and even multi-touch symbols. It allows addressing new challenging problems regarding the recognition of simultaneous composition of structured documents. The dataset is freely available on-line

Flow2Code - From Hand-Drawn Flowchart to Code Execution

Flowcharts play an important role when learning to program by conveying algorithms graphically and making them easy to read and understand. When learning how to code with flowcharts and transitioning between the two, people often use computer based software to design and execute the algorithm conveyed by the flowchart. This requires the users to learn how to use the computer-based software first, which often leads to a steep learning curve. We claim that the learning curve can be decremented by using off-line sketch recognition and computer vision algorithms on a mobile device. This can be done by drawing the flowchart on a piece of paper and using a mobile device with a camera to capture an image of the flowchart. Flow2Code is a code flowchart recognizer that allows the users to code simple scripts on a piece of paper by drawing flowcharts. This approach attempts to be more intuitive since the user does not need to learn how to use a system to design the flowchart. Only a pencil, a notebook with white pages, and a mobile device are needed to achieve the same result. The main contribution of this thesis is to provide a more intuitive and easy-to-use tool for people to translate flowcharts into code and then execute the code

Grouping Strokes into Shapes in Hand-Drawn Diagrams

Objects in freely-drawn sketches often have no spatial or temporal separation, making object recognition difficult. We present a two-step stroke-grouping algorithm that first classifies individual strokes according to the type of object to which they belong, then groups strokes with like classifications into clusters representing individual objects. The first step facilitates clustering by naturally separating the strokes, and both steps fluidly integrate spatial and temporal information. Our approach to grouping is unique in its formulation as an efficient classification task rather than, for example, an expensive search task. Our single-stroke classifier performs at least as well as existing single-stroke classifiers on text vs. nontext classification, and we present the first three-way single-stroke classification results. Our stroke grouping results are the first reported of their kind; our grouping algorithm correctly groups between 86% and 91% of the ink in diagrams from two domains, with between 69% and 79% of shapes being perfectly clustered

Sketch recognition of digital ink diagrams : a thesis presented in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Computer Science at Massey University, Palmerston North, New Zealand

Figures are either re-used with permission, or abstracted with permission from the source article.Sketch recognition of digital ink diagrams is the process of automatically identifying hand-drawn elements in a diagram. This research focuses on the simultaneous grouping and recognition of shapes in digital ink diagrams. In order to recognise a shape, we need to group strokes belonging to a shape, however, strokes cannot be grouped until the shape is identified. Therefore, we treat grouping and recognition as a simultaneous task. Our grouping technique uses spatial proximity to hypothesise shape candidates. Many of the hypothesised shape candidates are invalid, therefore we need a way to reject them. We present a novel rejection technique based on novelty detection. The rejection method uses proximity measures to validate a shape candidate. In addition, we investigate on improving the accuracy of the current shape recogniser by adding extra features. We also present a novel connector recognition system that localises connector heads around recognised shapes. We perform a full comparative study on two datasets. The results show that our approach is significantly more accurate in finding shapes and faster on process diagram compared to Stahovich et al. (2014), which the results show the superiority of our approach in terms of computation time and accuracy. Furthermore, we evaluate our system on two public datasets and compare our results with other approaches reported in the literature that have used these dataset. The results show that our approach is more accurate in finding and recognising the shapes in the FC dataset (by finding and recognising 91.7% of the shapes) compared to the reported results in the literature