4,041 research outputs found

    Programming with a Differentiable Forth Interpreter

    Get PDF
    Given that in practice training data is scarce for all but a small set of problems, a core question is how to incorporate prior knowledge into a model. In this paper, we consider the case of prior procedural knowledge for neural networks, such as knowing how a program should traverse a sequence, but not what local actions should be performed at each step. To this end, we present an end-to-end differentiable interpreter for the programming language Forth which enables programmers to write program sketches with slots that can be filled with behaviour trained from program input-output data. We can optimise this behaviour directly through gradient descent techniques on user-specified objectives, and also integrate the program into any larger neural computation graph. We show empirically that our interpreter is able to effectively leverage different levels of prior program structure and learn complex behaviours such as sequence sorting and addition. When connected to outputs of an LSTM and trained jointly, our interpreter achieves state-of-the-art accuracy for end-to-end reasoning about quantities expressed in natural language stories.Comment: 34th International Conference on Machine Learning (ICML 2017

    Dynamic Neuro-Symbolic Knowledge Graph Construction for Zero-shot Commonsense Question Answering

    Full text link
    Understanding narratives requires reasoning about implicit world knowledge related to the causes, effects, and states of situations described in text. At the core of this challenge is how to access contextually relevant knowledge on demand and reason over it. In this paper, we present initial studies toward zero-shot commonsense question answering by formulating the task as inference over dynamically generated commonsense knowledge graphs. In contrast to previous studies for knowledge integration that rely on retrieval of existing knowledge from static knowledge graphs, our study requires commonsense knowledge integration where contextually relevant knowledge is often not present in existing knowledge bases. Therefore, we present a novel approach that generates contextually-relevant symbolic knowledge structures on demand using generative neural commonsense knowledge models. Empirical results on two datasets demonstrate the efficacy of our neuro-symbolic approach for dynamically constructing knowledge graphs for reasoning. Our approach achieves significant performance boosts over pretrained language models and vanilla knowledge models, all while providing interpretable reasoning paths for its predictions

    CoRRPUS: Codex-Leveraged Structured Representations for Neurosymbolic Story Understanding

    Full text link
    Story generation and understanding -- as with all NLG/NLU tasks -- has seen a surge in neurosymbolic work. Researchers have recognized that, while large language models (LLMs) have tremendous utility, they can be augmented with symbolic means to be even better and to make up for any flaws that the neural networks might have. However, symbolic methods are extremely costly in terms of the amount of time and expertise needed to create them. In this work, we capitalize on state-of-the-art Code-LLMs, such as Codex, to bootstrap the use of symbolic methods for tracking the state of stories and aiding in story understanding. We show that our CoRRPUS system and abstracted prompting procedures can beat current state-of-the-art structured LLM techniques on pre-existing story understanding tasks (bAbI task 2 and Re^3) with minimal hand engineering. We hope that this work can help highlight the importance of symbolic representations and specialized prompting for LLMs as these models require some guidance for performing reasoning tasks properly.Comment: Accepted to Findings of ACL 202

    Naval Integration into Joint Data Strategies and Architectures in JADC2

    Get PDF
    NPS NRP Technical ReportAs Joint capabilities mature and shape into the Joint All Domain C2 Concept, Services, COCOMs and Coalition Partners will need to invest into efforts that would seamlessly integrate into Joint capabilities. The objective for the Navy is to study the options for Navy, including Naval Special Warfare Command under SOCOM, on how to integrate Navy's data strategy and architecture under the unifying JADC2 umbrella. The other objectives are to explore alternatives considered by the SOCOM and the Air Force, which are responsible for JADC2 Information Advantage and Digital Mission Command & Control. A major purpose of Joint, Services/COCOMs, agencies and Coalition Partners capabilities is to provide shared core of integrated canonical services for data, information, and knowledge with representations for vertical interoperability across all command levels and JADC2, lateral interoperability between Naval Service/COCOMs, and any combination of JADC2 constituents, agencies, and coalition partners. Our research plan is to explore available data strategy options by leveraging previous NRP work (NPS-20-N313-A). We will participate in emerging data strategy by Navy JADC2 project Overmatch. By working with MITRE our team will explore Air Force JADC2 data strategy implemented in ABMS DataOne component. Our goal is to find a seamless integration between Naval Data Strategy and data strategies behind JADC2 Information Advantage and Digital Mission Command & Control capabilities. Our plan includes studying Service-to-Service and Service-to-COCOM interoperability options required for Joint operations with a goal to minimize OODA's loop latency across sensing, situation discovery & monitoring, and knowledge understanding-for-planning, deciding, and acting. Our team realizes JADC2 requires virtual model allowing interoperability between subordinate C2 for services, agencies, and partner. Without such flexible 'joint' intersection organizational principal hierarchical structure it would be impossible to define necessary temporal and spatial fidelities for each level of organizational command required for implanting JADC2. Research deliverables will document the results of the exploration of Joint, COCOM, Agency and Partner Data Strategies approaches as JADC2 interoperability options to the emerging JADC2. We strive for standard JADC2 interface. Keywords: JADC2, ABMS, DataOne, Information Advantage, Digital Mission Command, IntegrationN2/N6 - Information WarfareThis research is supported by funding from the Naval Postgraduate School, Naval Research Program (PE 0605853N/2098). https://nps.edu/nrpChief of Naval Operations (CNO)Approved for public release. Distribution is unlimited.

    Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation

    Get PDF
    This paper surveys the current state of the art in Natural Language Generation (NLG), defined as the task of generating text or speech from non-linguistic input. A survey of NLG is timely in view of the changes that the field has undergone over the past decade or so, especially in relation to new (usually data-driven) methods, as well as new applications of NLG technology. This survey therefore aims to (a) give an up-to-date synthesis of research on the core tasks in NLG and the architectures adopted in which such tasks are organised; (b) highlight a number of relatively recent research topics that have arisen partly as a result of growing synergies between NLG and other areas of artificial intelligence; (c) draw attention to the challenges in NLG evaluation, relating them to similar challenges faced in other areas of Natural Language Processing, with an emphasis on different evaluation methods and the relationships between them.Comment: Published in Journal of AI Research (JAIR), volume 61, pp 75-170. 118 pages, 8 figures, 1 tabl
    corecore