280 research outputs found
A Survey of Deep Learning for Mathematical Reasoning
Mathematical reasoning is a fundamental aspect of human intelligence and is
applicable in various fields, including science, engineering, finance, and
everyday life. The development of artificial intelligence (AI) systems capable
of solving math problems and proving theorems has garnered significant interest
in the fields of machine learning and natural language processing. For example,
mathematics serves as a testbed for aspects of reasoning that are challenging
for powerful deep learning models, driving new algorithmic and modeling
advances. On the other hand, recent advances in large-scale neural language
models have opened up new benchmarks and opportunities to use deep learning for
mathematical reasoning. In this survey paper, we review the key tasks,
datasets, and methods at the intersection of mathematical reasoning and deep
learning over the past decade. We also evaluate existing benchmarks and
methods, and discuss future research directions in this domain.Comment: Accepted to ACL 2023. The repository is available at
https://github.com/lupantech/dl4mat
QNRs: toward language for intelligent machines
Impoverished syntax and nondifferentiable vocabularies make natural language a poor medium for neural representation learning and applications. Learned, quasilinguistic neural representations (QNRs) can upgrade words to embeddings and syntax to graphs to provide a more expressive and computationally tractable medium. Graph-structured, embedding-based quasilinguistic representations can support formal and informal reasoning, human and inter-agent communication, and the development of scalable quasilinguistic corpora with characteristics of both literatures and associative memory.
To achieve human-like intellectual competence, machines must be fully literate, able not only to read and learn, but to write things worth retaining as contributions to collective knowledge. In support of this goal, QNR-based systems could translate and process natural language corpora to support the aggregation, refinement, integration, extension, and application of knowledge at scale. Incremental development of QNRbased models can build on current methods in neural machine learning, and as systems mature, could potentially complement or replace todayâs opaque, error-prone âfoundation modelsâ with systems that are more capable, interpretable, and epistemically reliable. Potential applications and implications are broad
Mathematical Language Models: A Survey
In recent years, there has been remarkable progress in leveraging Language
Models (LMs), encompassing Pre-trained Language Models (PLMs) and Large-scale
Language Models (LLMs), within the domain of mathematics. This paper conducts a
comprehensive survey of mathematical LMs, systematically categorizing pivotal
research endeavors from two distinct perspectives: tasks and methodologies. The
landscape reveals a large number of proposed mathematical LLMs, which are
further delineated into instruction learning, tool-based methods, fundamental
CoT techniques, and advanced CoT methodologies. In addition, our survey entails
the compilation of over 60 mathematical datasets, including training datasets,
benchmark datasets, and augmented datasets. Addressing the primary challenges
and delineating future trajectories within the field of mathematical LMs, this
survey is positioned as a valuable resource, poised to facilitate and inspire
future innovation among researchers invested in advancing this domain.Comment: arXiv admin note: text overlap with arXiv:1705.04146,
arXiv:2304.10977, arXiv:2112.00114, arXiv:1905.13319, arXiv:2304.12244,
arXiv:2206.01347, arXiv:2006.09265 by other author
Deep Understanding of Technical Documents : Automated Generation of Pseudocode from Digital Diagrams & Analysis/Synthesis of Mathematical Formulas
The technical document is an entity that consists of several essential and interconnected parts, often referred to as modalities. Despite the extensive attention that certain parts have already received, per say the textual information, there are several aspects that severely under researched. Two such modalities are the utility of diagram images and the deep automated understanding of mathematical formulas. Inspired by existing holistic approaches to the deep understanding of technical documents, we develop a novel formal scheme for the modelling of digital diagram images. This extends to a generative framework that allows for the creation of artificial images and their annotation. We contribute on the field with the creation of a novel synthetic dataset and its generation mechanism. We propose the conversion of the pseudocode generation problem to an image captioning task and provide a family of techniques based on adaptive image partitioning. We address the mathematical formulasâ semantic understanding by conducting an evaluating survey on the field, published in May 2021. We then propose a formal synthesis framework that utilized formula graphs as metadata, reaching for novel valuable formulas. The synthesis framework is validated by a deep geometric learning mechanism, that outsources formula data to simulate the missing a priori knowledge. We close with the proof of concept, the description of the overall pipeline and our future aims
- âŠ