Mathematical Formula Recognition and Transformation to a Linear Format Suitable for Vocalization

Students with vision impairment encounter barriers in studying mathematics particularly in higher education levels. They must have an equal chance with sighted students in mathematics subjects. Making mathematics accessible to the vision impaired users is a complicated process. This accessibility can be static or dynamic, in static accessibility the user is presented with a representation of the entire mathematic expression passively such as using Braille, dynamic accessibility allows the user to navigate the mathematical content in accordance with its structure interactively such as audio format [1]. MATHSPEAK is an application that accepts objects described in LaTeX and converts it to a linear or sequential representation suitable for vocalization, describing functions to people with severe vision impairment. MATHSPEAK provides interactive dynamic access to mathematic expressions by rendering them to audio format. This paper describes a method to create plain text from images of mathematical formulae and convert this text to LaTeX which is used in the earlier developed algorithm, “MATHSPEAK”

Deep Understanding of Technical Documents : Automated Generation of Pseudocode from Digital Diagrams & Analysis/Synthesis of Mathematical Formulas

The technical document is an entity that consists of several essential and interconnected parts, often referred to as modalities. Despite the extensive attention that certain parts have already received, per say the textual information, there are several aspects that severely under researched. Two such modalities are the utility of diagram images and the deep automated understanding of mathematical formulas. Inspired by existing holistic approaches to the deep understanding of technical documents, we develop a novel formal scheme for the modelling of digital diagram images. This extends to a generative framework that allows for the creation of artificial images and their annotation. We contribute on the field with the creation of a novel synthetic dataset and its generation mechanism. We propose the conversion of the pseudocode generation problem to an image captioning task and provide a family of techniques based on adaptive image partitioning. We address the mathematical formulas’ semantic understanding by conducting an evaluating survey on the field, published in May 2021. We then propose a formal synthesis framework that utilized formula graphs as metadata, reaching for novel valuable formulas. The synthesis framework is validated by a deep geometric learning mechanism, that outsources formula data to simulate the missing a priori knowledge. We close with the proof of concept, the description of the overall pipeline and our future aims

Optical Character Recognition for Typeset Mathematics

There is a wealth of mathematical knowledge that could be potentially very useful in many computational applications, but is not available in electronic form. This knowledge comes in the form of mechanically typeset books and journals going back more than a hundred years. Besides these older sources, there are a great many current publications, filled with useful mathematical information, which are difficult if not impossible to obtain in electronic form. What we would like to do is extract character information from these documents, which could then be passed to higher-level parsing routines for further extraction of mathematical content (or any other useful 2-dimensional semantic content). Unfortunately, current commercial OCR (optical character recognition) software packages are quite unable to handle mathematical formulas, since their algorithms at all levels use heuristics developed for other document styles 1 . We are concerned with the development of OCR methods that are able..