Introducing Handwriting into a Multimodal LATEX Formula Editor

Handwriting has been shown to be a useful input modality for math. However, math recognizers are imperfect, especially when recognizing complex expressions. Instead of improving the recognizer itself, we explore ways to best visualize the recognizer\u27s output to help the user fix recognition mistakes more efficiently. To do this, we propose changes to the visual editing operations in MathDeck, a math-aware search engine and formula editor, as well as the addition of an n-best list of results for each symbol in the recognizer\u27s output. We present two experiments to help us find good ways to help users fix errors in the recognizer, and to test whether these changes help novices input formulas more efficiently than they would if they did not have handwriting as an input modality. In the first experiment, users had the option to fix errors with an in-place drop-down menu of alternate symbols, a side symbol correction panel, or by typing the symbols themselves or dragging them from a symbol palette. In our experiment, most users preferred to fix the errors manually by typing the correct symbols or using the symbol palette. In the second experiment, participants entered formulas using handwriting and/or LaTeX. We found evidence that suggests that novices can input formulas faster when they have access to handwriting, but experts still do better when they can just type LaTeX

Visual Structure Editing of Math Formulas

Math formulas can be large and complex resulting in correspondingly large and complex LaTeX math strings for expressing them. We design operations to visually edit the typeset LaTeX formulas. The operations are invoked via the formula\u27s control points, which are created as a way to specify an operation associated with the point\u27s location relative to a symbol in the formula. At the control points, formulas can be extended in multiple ways, LaTeX can be inserted locally by typing, an existing formula can be inserted, or part of the formula itself can be moved to that point. Parts of formulas can be selected by clicking on a symbol or dragging a rectangle over an area in the formula, and the subtree for the selection can be replaced, deleted, moved to another point in the formula, or lifted out of the formula into a chip floating above the canvas. Formula chips can be used as arguments to operations, including a set of existing formulas provided in a symbol palette. Operations can be performed either by making a selection, selecting a control point operation, and then specifying an argument, or by dragging an argument to one of the control points in the formula. We perform an online formula editing experiment to examine if these visual editing operations can be used to reduce the time and actions spent in order to make edits to formulas. With 35 participants completing 18 formula editing tasks split between 3 input conditions of LaTeX only, Visual only, or LaTeX and Visual, we find that on average participants spend the least amount of time on the editing tasks when both editing capabilities are available

An initial evaluation of MathPad(2): A tool for creating dynamic mathematical illustrations

MathPad(2) is a pen-based application prototype for creating mathematical sketches. Using a modeless gestural interface, it lets users make dynamic illustrations by associating handwritten mathematics with free-form drawings and provides a set of tools for graphing and evaluating mathematical expressions and solving equations. In this paper, we present the results of an initial evaluation of the MathPad(2) prototype, examining the user interface\u27s intuitiveness and the application\u27s perceived usefulness. Our evaluations are based on both performance and questionnaire results including first attempt gesture performance, interface recall tests, and surveys of user interface satisfaction and perceived usefulness. The results of our evaluation suggest that, although some test subjects had difficulty with our mathematical expression recognizer, they found the interface, in general, intuitive and easy to remember. More importantly, these results suggest the prototype has the potential to assist beginning physics and mathematics students in problem solving and understanding scientific concepts. (c) 2007 Elsevier Ltd. All rights reserved

The Cognitive and Demographic Variables that Underlie Notetaking and Review in Mathematics: Does Quality of Notes Predict Test Performance in Mathematics?

Taking and reviewing lecture notes is an effective and prevalent method of studying employed by students at the post-secondary level (Armbruster, 2000; Armbruster, 2009; Dunkel and Davy, 1989; Peverly et al., 2009). However, few studies have examined the cognitive variables that underlie this skill. In addition, these studies have focused on more verbally based domains, such as history and psychology. The current study examined the practical utility of notes in actual class settings. It is the first study that has attempted to examine the outcomes and cognitive skills associated with note-taking and review in any area of mathematics. It also set out to establish the importance of quality of notes and quality of review sheets to test performance in graduate level probability and statistics courses. Finally, this dissertation sought to explore the extent to which variables besides notes also contribute to test performance in this domain. Participants included 74 graduate students enrolled in introductory probability and statistics courses at a private graduate teacher education college in a large city in the Northeast United States. Participants took notes during class and provided the researcher with a copy of their notes for several lectures. Participants were also required to write down additional information on the back of two formula sheets that were used as an aid on the midterm exam. The independent variables included handwriting speed, gender, spatial visualization ability, background knowledge, verbal ability, and working memory. The dependent variables were quality of lecture notes, quality of supplemental review sheets, and midterm performance. All measures were group administered. Results revealed that gender was the only predictor of quality of lecture notes. Quality of lecture notes was the only significant predictor of quality of supplemental review sheets. Neither quality of lecture notes nor quality of supplemental review sheets predicted overall test performance. Instead, background knowledge and instructor significantly predicted overall test performance. Handwriting speed was a marginally significant predictor of overall test performance. Future research aimed at replicating these findings and expanding the results to include other mathematical domains and educational levels is recommended

Reconocimiento de notación matemática escrita a mano fuera de línea

El reconocimiento automático de expresiones matemáticas es uno de los problemas de reconocimiento de patrones, debido a que las matemáticas representan una fuente valiosa de información en muchos a ́reas de investigación. La escritura de expresiones matemáticas a mano es un medio de comunicación utilizado para la transmisión de información y conocimiento, con la cual se pueden generar de una manera sencilla escritos que contienen notación matemática. Este proceso puede volverse tedioso al ser escrito en lenguaje de composición tipográfica que pueda ser procesada por una computadora, tales como LATEX, MathML, entre otros. En los sistemas de reconocimiento de expresiones matem ́aticas existen dos m ́etodos diferentes a saber: fuera de l ́ınea y en l ́ınea. En esta tesis, se estudia el desempen ̃o de un sistema fuera de l ́ınea en donde se describen los pasos b ́asicos para lograr una mejor precisio ́n en el reconocimiento, las cuales esta ́n divididas en dos pasos principales: recono- cimiento de los s ́ımbolos de las ecuaciones matema ́ticas y el ana ́lisis de la estructura en que est ́an compuestos. Con el fin de convertir una expresi ́on matema ́tica escrita a mano en una expresio ́n equivalente en un sistema de procesador de texto, tal como TEX

Discovering real-world usage scenarios for a multimodal math search interface

To use math expressions in search, current search engines require knowing expression names or using a structure editor or string encoding (e.g., LaTeX) to enter expressions. This is unfortunate for people who are not math experts, as this can lead to an intention gap between the math query they wish to express, and what the interface will allow. min is a search interface that supports drawing expressions on a canvas using a mouse/touch, keyboard and images. We designed a user study to examine how the multimodal interface of min changes search behavior for mathematical non-experts, and discover real-world usage scenarios. Participants demonstrated increased use of math expressions in queries when using min. There was little difference in task success reported by participants using min vs. text-based search, but the majority of participants appreciated the multimodal input, and identified real-world scenarios in which they would like to use systems like min

A Survey of User Interfaces for Computer Algebra Systems

AbstractThis paper surveys work within the Computer Algebra community (and elsewhere) directed towards improving user interfaces for scientific computation during the period 1963–1994. It is intended to be useful to two groups of people: those who wish to know what work has been done and those who would like to do work in the field. It contains an extensive bibliography to assist readers in exploring the field in more depth. Work related to improving human interaction with computer algebra systems is the main focus of the paper. However, the paper includes additional materials on some closely related issues such as structured document editing, graphics, and communication protocols