A Cognitive View of Pandemic Meditation (A Mathematical Visual Poem)

Mathematical visual poetry is a poetic genre whereby metaphorical expressions are created using mathematical structures. Within the structure, the poetics are understood by the cross-mapping of numerous conceptual domains including visual, lexical, and mathematical. Here I focus on one particular mathematical visual poetic structure: what I call a Similar Triangles Poem or Proportional Poem. To illustrate the ideas discussed, I present Pandemic Meditations, a mathematical visual poem; in particular I discuss how this mathematical poem uses the mechanisms of poetic metaphor in the context of the embodied mind. The intent of this paper is not to explain Pandemic Meditation, for explanations of poetry serve only to kill it. Instead, the intent here is to give the reader the tools to access similar triangles poems in general, and this expression in particular, and to show how it functions within the definitions of poetic metaphor. This paper can be used as a template to study all similar triangles visual poems, and more generally, as a source to study visual poetry

Symbolic and Visual Retrieval of Mathematical Notation using Formula Graph Symbol Pair Matching and Structural Alignment

Large data collections containing millions of math formulae in different formats are available on-line. Retrieving math expressions from these collections is challenging. We propose a framework for retrieval of mathematical notation using symbol pairs extracted from visual and semantic representations of mathematical expressions on the symbolic domain for retrieval of text documents. We further adapt our model for retrieval of mathematical notation on images and lecture videos. Graph-based representations are used on each modality to describe math formulas. For symbolic formula retrieval, where the structure is known, we use symbol layout trees and operator trees. For image-based formula retrieval, since the structure is unknown we use a more general Line of Sight graph representation. Paths of these graphs define symbol pairs tuples that are used as the entries for our inverted index of mathematical notation. Our retrieval framework uses a three-stage approach with a fast selection of candidates as the first layer, a more detailed matching algorithm with similarity metric computation in the second stage, and finally when relevance assessments are available, we use an optional third layer with linear regression for estimation of relevance using multiple similarity scores for final re-ranking. Our model has been evaluated using large collections of documents, and preliminary results are presented for videos and cross-modal search. The proposed framework can be adapted for other domains like chemistry or technical diagrams where two visually similar elements from a collection are usually related to each other

VMEXT: A Visualization Tool for Mathematical Expression Trees

Mathematical expressions can be represented as a tree consisting of terminal symbols, such as identifiers or numbers (leaf nodes), and functions or operators (non-leaf nodes). Expression trees are an important mechanism for storing and processing mathematical expressions as well as the most frequently used visualization of the structure of mathematical expressions. Typically, researchers and practitioners manually visualize expression trees using general-purpose tools. This approach is laborious, redundant, and error-prone. Manual visualizations represent a user's notion of what the markup of an expression should be, but not necessarily what the actual markup is. This paper presents VMEXT - a free and open source tool to directly visualize expression trees from parallel MathML. VMEXT simultaneously visualizes the presentation elements and the semantic structure of mathematical expressions to enable users to quickly spot deficiencies in the Content MathML markup that does not affect the presentation of the expression. Identifying such discrepancies previously required reading the verbose and complex MathML markup. VMEXT also allows one to visualize similar and identical elements of two expressions. Visualizing expression similarity can support support developers in designing retrieval approaches and enable improved interaction concepts for users of mathematical information retrieval systems. We demonstrate VMEXT's visualizations in two web-based applications. The first application presents the visualizations alone. The second application shows a possible integration of the visualizations in systems for mathematical knowledge management and mathematical information retrieval. The application converts LaTeX input to parallel MathML, computes basic similarity measures for mathematical expressions, and visualizes the results using VMEXT.Comment: 15 pages, 4 figures, Intelligent Computer Mathematics - 10th International Conference CICM 2017, Edinburgh, UK, July 17-21, 2017, Proceeding

Cross-domain priming from mathematics to relative-clause attachment: a visual-world study in French

Human language processing must rely on a certain degree of abstraction, as we can produce and understand sentences that we have never produced or heard before. One way to establish syntactic abstraction is by investigating structural priming. Structural priming has been shown to be effective within a cognitive domain, in the present case, the linguistic domain. But does priming also work across different domains? In line with previous experiments, we investigated cross-domain structural priming from mathematical expressions to linguistic structures with respect to relative clause attachment in French (e.g., la fille du professeur qui habitait à Paris/the daughter of the teacher who lived in Paris). Testing priming in French is particularly interesting because it will extend earlier results established for English to a language where the baseline for relative clause attachment preferences is different form English: in English, relative clauses (RCs) tend to be attached to the local noun phrase (low attachment) while in French there is a preference for high attachment of relative clauses to the first noun phrase (NP). Moreover, in contrast to earlier studies, we applied an online-technique (visual world eye-tracking). Our results confirm cross-domain priming from mathematics to linguistic structures in French. Most interestingly, different from less mathematically adept participants, we found that in mathematically skilled participants, the effect emerged very early on (at the beginning of the relative clause in the speech stream) and is also present later (at the end of the relative clause). In line with previous findings, our experiment suggests that mathematics and language share aspects of syntactic structure at a very high-level of abstraction

Math Search for the Masses: Multimodal Search Interfaces and Appearance-Based Retrieval

We summarize math search engines and search interfaces produced by the Document and Pattern Recognition Lab in recent years, and in particular the min math search interface and the Tangent search engine. Source code for both systems are publicly available. "The Masses" refers to our emphasis on creating systems for mathematical non-experts, who may be looking to define unfamiliar notation, or browse documents based on the visual appearance of formulae rather than their mathematical semantics.Comment: Paper for Invited Talk at 2015 Conference on Intelligent Computer Mathematics (July, Washington DC

Visual Transformations in Symbolic Elementary Algebra.

Competence in algebra requires knowledge of parsing and transformations. The parsing component specifies the structure of algebraic expressions based on conventional operation hierarchies. The transformational component specifies the real number properties used to transform algebraic expressions into equivalent forms. In standard models of mathematical cognition, both components are conceived as strictly propositional domains (e.g., Anderson, 1983a). Kirshner (1989b) demonstrated that parsing knowledge initially is apprehended in a visual (non-propositional) modality. This dissertation extends that visual analysis beyond parsing to the transformational component. It is proposed that transformations range on a continuum from highly propositional (e.g., x(y\sp{-1}) = ({\rm x \over y}, x\sp2 - y\sp2 = (x - y)(x + y)) to highly visual (e.g., (x\sp{\rm y})\sp{\rm z} = x\sp{\rm yz}, (xy)\sp{\rm n} = x\sp{\rm n}y\sp{\rm n}. It is hypothesized that visual rules are: (1) easier to apprehend initially, but (2) less easily constrained to their proper contexts of application. Thus common errors like ${a+b\over a+c} = {b\over c}$ and {\root n\of {a+b}} = {\root n\of a} + {\root n \of b} are analyzed as overgeneralizations of visual rules like ${ab \over ac} = {b\over c}$ and {\root n \of {AB}} = {\root n\of a}{\root n\of b}, respectively. Two groups of algebra neophytes were taught a mixture of visual and nonvisual rules. One group was taught using ordinary algebraic notation: the other using a syntactic tree notation which distorts the visual structure of ordinary notation, forcing propositional level learning for all rules. Each group was evaluated using recognition tasks, which were applications of rules that had been taught, and rejection tasks, to which no rule applied though one rule nearly applied. Recognition tasks assess the students\u27 initial rule acquisition. Rejection items invite overgeneralization of rules. In tree notation the two rule types were equally difficult to recognize, but in ordinary notation visual rules were significantly easier to recognize than propositional rules. For rejection tasks in tree notation, visual and propositional items were equally difficult. In ordinary notation the visual items tended to be more difficult to constrain than propositional items; though because of basement effects these differences were significant only in some cases. The visual salience construct is more fully analyzed in the Conclusions section

Categorical invariance and structural complexity in human concept learning

An alternative account of human concept learning based on an invariance measure of the categorical\ud stimulus is proposed. The categorical invariance model (CIM) characterizes the degree of structural\ud complexity of a Boolean category as a function of its inherent degree of invariance and its cardinality or\ud size. To do this we introduce a mathematical framework based on the notion of a Boolean differential\ud operator on Boolean categories that generates the degrees of invariance (i.e., logical manifold) of the\ud category in respect to its dimensions. Using this framework, we propose that the structural complexity\ud of a Boolean category is indirectly proportional to its degree of categorical invariance and directly\ud proportional to its cardinality or size. Consequently, complexity and invariance notions are formally\ud unified to account for concept learning difficulty. Beyond developing the above unifying mathematical\ud framework, the CIM is significant in that: (1) it precisely predicts the key learning difficulty ordering of\ud the SHJ [Shepard, R. N., Hovland, C. L.,&Jenkins, H. M. (1961). Learning and memorization of classifications.\ud Psychological Monographs: General and Applied, 75(13), 1-42] Boolean category types consisting of three\ud binary dimensions and four positive examples; (2) it is, in general, a good quantitative predictor of the\ud degree of learning difficulty of a large class of categories (in particular, the 41 category types studied\ud by Feldman [Feldman, J. (2000). Minimization of Boolean complexity in human concept learning. Nature,\ud 407, 630-633]); (3) it is, in general, a good quantitative predictor of parity effects for this large class of\ud categories; (4) it does all of the above without free parameters; and (5) it is cognitively plausible (e.g.,\ud cognitively tractable)

Symbol detection in online handwritten graphics using Faster R-CNN

Symbol detection techniques in online handwritten graphics (e.g. diagrams and mathematical expressions) consist of methods specifically designed for a single graphic type. In this work, we evaluate the Faster R-CNN object detection algorithm as a general method for detection of symbols in handwritten graphics. We evaluate different configurations of the Faster R-CNN method, and point out issues relative to the handwritten nature of the data. Considering the online recognition context, we evaluate efficiency and accuracy trade-offs of using Deep Neural Networks of different complexities as feature extractors. We evaluate the method on publicly available flowchart and mathematical expression (CROHME-2016) datasets. Results show that Faster R-CNN can be effectively used on both datasets, enabling the possibility of developing general methods for symbol detection, and furthermore, general graphic understanding methods that could be built on top of the algorithm.Comment: Submitted to DAS-201