4,041 research outputs found
Programming with a Differentiable Forth Interpreter
Given that in practice training data is scarce for all but a small set of
problems, a core question is how to incorporate prior knowledge into a model.
In this paper, we consider the case of prior procedural knowledge for neural
networks, such as knowing how a program should traverse a sequence, but not
what local actions should be performed at each step. To this end, we present an
end-to-end differentiable interpreter for the programming language Forth which
enables programmers to write program sketches with slots that can be filled
with behaviour trained from program input-output data. We can optimise this
behaviour directly through gradient descent techniques on user-specified
objectives, and also integrate the program into any larger neural computation
graph. We show empirically that our interpreter is able to effectively leverage
different levels of prior program structure and learn complex behaviours such
as sequence sorting and addition. When connected to outputs of an LSTM and
trained jointly, our interpreter achieves state-of-the-art accuracy for
end-to-end reasoning about quantities expressed in natural language stories.Comment: 34th International Conference on Machine Learning (ICML 2017
Dynamic Neuro-Symbolic Knowledge Graph Construction for Zero-shot Commonsense Question Answering
Understanding narratives requires reasoning about implicit world knowledge
related to the causes, effects, and states of situations described in text. At
the core of this challenge is how to access contextually relevant knowledge on
demand and reason over it.
In this paper, we present initial studies toward zero-shot commonsense
question answering by formulating the task as inference over dynamically
generated commonsense knowledge graphs. In contrast to previous studies for
knowledge integration that rely on retrieval of existing knowledge from static
knowledge graphs, our study requires commonsense knowledge integration where
contextually relevant knowledge is often not present in existing knowledge
bases. Therefore, we present a novel approach that generates
contextually-relevant symbolic knowledge structures on demand using generative
neural commonsense knowledge models.
Empirical results on two datasets demonstrate the efficacy of our
neuro-symbolic approach for dynamically constructing knowledge graphs for
reasoning. Our approach achieves significant performance boosts over pretrained
language models and vanilla knowledge models, all while providing interpretable
reasoning paths for its predictions
CoRRPUS: Codex-Leveraged Structured Representations for Neurosymbolic Story Understanding
Story generation and understanding -- as with all NLG/NLU tasks -- has seen a
surge in neurosymbolic work. Researchers have recognized that, while large
language models (LLMs) have tremendous utility, they can be augmented with
symbolic means to be even better and to make up for any flaws that the neural
networks might have. However, symbolic methods are extremely costly in terms of
the amount of time and expertise needed to create them. In this work, we
capitalize on state-of-the-art Code-LLMs, such as Codex, to bootstrap the use
of symbolic methods for tracking the state of stories and aiding in story
understanding. We show that our CoRRPUS system and abstracted prompting
procedures can beat current state-of-the-art structured LLM techniques on
pre-existing story understanding tasks (bAbI task 2 and Re^3) with minimal hand
engineering. We hope that this work can help highlight the importance of
symbolic representations and specialized prompting for LLMs as these models
require some guidance for performing reasoning tasks properly.Comment: Accepted to Findings of ACL 202
Naval Integration into Joint Data Strategies and Architectures in JADC2
NPS NRP Technical ReportAs Joint capabilities mature and shape into the Joint All Domain C2 Concept, Services, COCOMs and Coalition Partners will need to invest into efforts that would seamlessly integrate into Joint capabilities. The objective for the Navy is to study the options for Navy, including Naval Special Warfare Command under SOCOM, on how to integrate Navy's data strategy and architecture under the unifying JADC2 umbrella. The other objectives are to explore alternatives considered by the SOCOM and the Air Force, which are responsible for JADC2 Information Advantage and Digital Mission Command & Control. A major purpose of Joint, Services/COCOMs, agencies and Coalition Partners capabilities is to provide shared core of integrated canonical services for data, information, and knowledge with representations for vertical interoperability across all command levels and JADC2, lateral interoperability between Naval Service/COCOMs, and any combination of JADC2 constituents, agencies, and coalition partners. Our research plan is to explore available data strategy options by leveraging previous NRP work (NPS-20-N313-A). We will participate in emerging data strategy by Navy JADC2 project Overmatch. By working with MITRE our team will explore Air Force JADC2 data strategy implemented in ABMS DataOne component. Our goal is to find a seamless integration between Naval Data Strategy and data strategies behind JADC2 Information Advantage and Digital Mission Command & Control capabilities. Our plan includes studying Service-to-Service and Service-to-COCOM interoperability options required for Joint operations with a goal to minimize OODA's loop latency across sensing, situation discovery & monitoring, and knowledge understanding-for-planning, deciding, and acting. Our team realizes JADC2 requires virtual model allowing interoperability between subordinate C2 for services, agencies, and partner. Without such flexible 'joint' intersection organizational principal hierarchical structure it would be impossible to define necessary temporal and spatial fidelities for each level of organizational command required for implanting JADC2. Research deliverables will document the results of the exploration of Joint, COCOM, Agency and Partner Data Strategies approaches as JADC2 interoperability options to the emerging JADC2. We strive for standard JADC2 interface. Keywords: JADC2, ABMS, DataOne, Information Advantage, Digital Mission Command, IntegrationN2/N6 - Information WarfareThis research is supported by funding from the Naval Postgraduate School, Naval Research Program (PE 0605853N/2098). https://nps.edu/nrpChief of Naval Operations (CNO)Approved for public release. Distribution is unlimited.
Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation
This paper surveys the current state of the art in Natural Language
Generation (NLG), defined as the task of generating text or speech from
non-linguistic input. A survey of NLG is timely in view of the changes that the
field has undergone over the past decade or so, especially in relation to new
(usually data-driven) methods, as well as new applications of NLG technology.
This survey therefore aims to (a) give an up-to-date synthesis of research on
the core tasks in NLG and the architectures adopted in which such tasks are
organised; (b) highlight a number of relatively recent research topics that
have arisen partly as a result of growing synergies between NLG and other areas
of artificial intelligence; (c) draw attention to the challenges in NLG
evaluation, relating them to similar challenges faced in other areas of Natural
Language Processing, with an emphasis on different evaluation methods and the
relationships between them.Comment: Published in Journal of AI Research (JAIR), volume 61, pp 75-170. 118
pages, 8 figures, 1 tabl
- …