A unified method for augmented incremental recognition of online handwritten Japanese and English text

We present a unifed method to augmented incremental recognition for online handwritten Japanese and English text, which is used for busy or on-the-fly recognition while writing, and lazy or delayed recognition after writing, without incurring long waiting times. It extends the local context for segmentation and recognition to a range of recent strokes called "segmentation scope" and "recognition scop", respectively. The recognition scope is inside of the segmentation scope. The augmented incremental recognition triggers recognition at every several recent strokes, updates the segmentation and recognition candidate lattice, and searches over the lattice for the best result incrementally. It also incorporates three techniques. The frst is to reuse the segmentation and recognition candidate lattice in the previous recognition scope for the current recognition scope. The second is to fx undecided segmentation points if they are stable between character/word patterns. The third is to skip recognition of partial candidate character/word patterns. The augmented incremental method includes the case of triggering recognition at every new stroke with the above-mentioned techniques. Experiments conducted on TUAT-Kondate and IAM online database show its superiority to batch recognition (recognizing text at one time) and pure incremental recognition (recognizing text at every input stroke) in processing time, waiting time, and recognition accuracy

Augmented incremental recognition of online handwritten mathematical expressions

This paper presents an augmented incremental recognition method for online handwritten mathematical expressions (MEs). If an ME is recognized after all strokes are written (batch recognition), the waiting time increases significantly when the ME becomes longer. On the other hand, the pure incremental recognition method recognizes an ME whenever a new single stroke is input. It shortens the waiting time but degrades the recognition rate due to the limited context. Thus, we propose an augmented incremental recognition method that not only maintains the advantage of the two methods but also reduces their weaknesses. The proposed method has two main features: one is to process the latest stroke, and the other is to find the erroneous segmentations and recognitions in the recent strokes and correct them. In the first process, the segmentation and the recognition by Cocke-Younger-Kasami (CYK) algorithm are only executed for the latest stroke. In the second process, all the previous segmentations are updated if they are significantly changed after the latest stroke is input, and then, all the symbols related to the updated segmentations are updated with their recognition scores. These changes are reflected in the CYK table. In addition, the waiting time is further reduced by employing multi-thread processes. Experiments on our dataset and the CROHME datasets show the effectiveness of this augmented incremental recognition method, which not only maintains recognition rate even compared with the batch recognition method but also reduces the waiting time to a very small level

Corrective Feedback Timing in Kanji Writing Instruction Apps

The focus of this research paper is to determine the correct time to provide corrective feedback to people who are learning how to write Japanese kanji. To do this, we developed a system that is able to recognize Japanese kanji that is handwritten onto an iPad screen and check for errors such as wrong stroke order. Previous research has achieved success in developing similar systems, but this project is unique because the research question involves the timing of corrective feedback. In particular, we are looking at whether immediate or delayed corrective feedback results in better learning

Reconnaissance à la volée de documents structurés manuscrits en-ligne

Dans ce papier, une nouvelle approche pour l'interprétation de documents structurés manuscrits en-ligne est présentée. Elle est basée sur un formalisme flexible et générique permettant la reconnaissance à la volée des éléments d'un document structuré. L'originalité du formalisme est la modélisation du couplage d'une vision globale du document analysé avec une vision locale de l'élément à reconnaître. L'analyseur pilote alors des reconnaisseurs de formes dédiés en fonction du contexte structurel de l'élément analysé. Nous détaillons plus particulièrement le processus de prise de décision en cas d'ambiguïté entre plusieurs interprétations possibles. Nous exploitons la théorie des sous-ensembles flous afin de prendre en compte la nature imprécise du tracé manuscrit et des contextes structurels modélisés. Cette approche a été validée avec le développement de trois systèmes orientés stylo : pour l'édition de partitions musicales, de graphes et de diagrammes de classes UML

The MUMTDB dataset for evaluating simultaneous composition of structured documents in a multi-user and multi-touch environment

International audienceWe propose in this paper a new online MultiUser Multi-Touch handwritten diagram DataBase (MUMTDB) for evaluating recognition systems under the multiuser situation. The data is collected according to two predefined mind map scenarios which contains 9 classes of graphical symbols. Each scenario is completed by involving two users at the same time. Since the users are given freedom to draw the symbols as they want, the dataset contains a diversity of multi-stroke and even multi-touch symbols. It allows addressing new challenging problems regarding the recognition of simultaneous composition of structured documents. The dataset is freely available on-line

A Survey of User Interfaces for Computer Algebra Systems

AbstractThis paper surveys work within the Computer Algebra community (and elsewhere) directed towards improving user interfaces for scientific computation during the period 1963–1994. It is intended to be useful to two groups of people: those who wish to know what work has been done and those who would like to do work in the field. It contains an extensive bibliography to assist readers in exploring the field in more depth. Work related to improving human interaction with computer algebra systems is the main focus of the paper. However, the paper includes additional materials on some closely related issues such as structured document editing, graphics, and communication protocols

Dynamic motion coupling of body movement for input control

Touchless gestures are used for input when touch is unsuitable or unavailable, such as when interacting with displays that are remote, large, public, or when touch is prohibited for hygienic reasons. Traditionally user input is spatially or semantically mapped to system output, however, in the context of touchless gestures these interaction principles suffer from several disadvantages including memorability, fatigue, and ill-defined mappings. This thesis investigates motion correlation as the third interaction principle for touchless gestures, which maps user input to system output based on spatiotemporal matching of reproducible motion. We demonstrate the versatility of motion correlation by using movement as the primary sensing principle, relaxing the restrictions on how a user provides input. Using TraceMatch, a novel computer vision-based system, we show how users can provide effective input through investigation of input performance with different parts of the body, and how users can switch modes of input spontaneously in realistic application scenarios. Secondly, spontaneous spatial coupling shows how motion correlation can bootstrap spatial input, allowing any body movement, or movement of tangible objects, to be appropriated for ad hoc touchless pointing on a per interaction basis. We operationalise the concept in MatchPoint, and demonstrate the unique capabilities through an exploration of the design space with application examples. Finally, we explore how users synchronise with moving targets in the context of motion correlation, revealing how simple harmonic motion leads to better synchronisation. Using the insights gained we explore the robustness of algorithms used for motion correlation, showing how it is possible to successfully detect a user's intent to interact whilst suppressing accidental activations from common spatial and semantic gestures. Finally, we look across our work to distil guidelines for interface design, and further considerations of how motion correlation can be used, both in general and for touchless gestures

Issues in NASA program and project management

This new collection of papers on aerospace management issues contains a history of NASA program and project management, some lessons learned in the areas of management and budget from the Space Shuttle Program, an analysis of tools needed to keep large multilayer programs organized and on track, and an update of resources for NASA managers. A wide variety of opinions and techniques are presented

A multi-modal interface for road planning tasks using vision, haptics and sound

The planning of transportation infrastructure requires analyzing many different types of geo-spatial information in the form of maps. Displaying too many of these maps at the same time can lead to visual clutter or information overload, which results in sub-optimal effectiveness. Multimodal interfaces (MMIs) try to address this visual overload and improve the user\u27s interaction with large amounts of data by combining several sensory modalities. Previous research into MMIs seems to indicate that using multiple sensory modalities leads to more efficient human-computer interactions when used properly. The motivation from this previous work has lead to the creation of this thesis, which describes a novel GIS system for road planning using vision, haptics and sound. The implementation of this virtual environment is discussed, including some of the design decisions used when trying to ascertain how we map visual data to our other senses. A user study was performed to see how this type of system could be utilized, and the results of the study are presented