175,836 research outputs found
Grammar Filtering For Syntax-Guided Synthesis
Programming-by-example (PBE) is a synthesis paradigm that allows users to
generate functions by simply providing input-output examples. While a promising
interaction paradigm, synthesis is still too slow for realtime interaction and
more widespread adoption. Existing approaches to PBE synthesis have used
automated reasoning tools, such as SMT solvers, as well as works applying
machine learning techniques. At its core, the automated reasoning approach
relies on highly domain specific knowledge of programming languages. On the
other hand, the machine learning approaches utilize the fact that when working
with program code, it is possible to generate arbitrarily large training
datasets. In this work, we propose a system for using machine learning in
tandem with automated reasoning techniques to solve Syntax Guided Synthesis
(SyGuS) style PBE problems. By preprocessing SyGuS PBE problems with a neural
network, we can use a data driven approach to reduce the size of the search
space, then allow automated reasoning-based solvers to more quickly find a
solution analytically. Our system is able to run atop existing SyGuS PBE
synthesis tools, decreasing the runtime of the winner of the 2019 SyGuS
Competition for the PBE Strings track by 47.65% to outperform all of the
competing tools
BizBench: A Quantitative Reasoning Benchmark for Business and Finance
As large language models (LLMs) impact a growing number of complex domains,
it is becoming increasingly important to have fair, accurate, and rigorous
evaluation benchmarks. Evaluating the reasoning skills required for business
and financial NLP stands out as a particularly difficult challenge. We
introduce BizBench, a new benchmark for evaluating models' ability to reason
about realistic financial problems. BizBench comprises 8 quantitative reasoning
tasks. Notably, BizBench targets the complex task of question-answering (QA)
for structured and unstructured financial data via program synthesis (i.e.,
code generation). We introduce three diverse financially-themed code-generation
tasks from newly collected and augmented QA data. Additionally, we isolate
distinct financial reasoning capabilities required to solve these QA tasks:
reading comprehension of financial text and tables, which is required to
extract correct intermediate values; and understanding domain knowledge (e.g.,
financial formulas) needed to calculate complex solutions. Collectively, these
tasks evaluate a model's financial background knowledge, ability to extract
numeric entities from financial documents, and capacity to solve problems with
code. We conduct an in-depth evaluation of open-source and commercial LLMs,
illustrating that BizBench is a challenging benchmark for quantitative
reasoning in the finance and business domain.Comment: Work in progres
Learning Logic Programs by Discovering Higher-Order Abstractions
Discovering novel abstractions is important for human-level AI. We introduce
an approach to discover higher-order abstractions, such as map, filter, and
fold. We focus on inductive logic programming, which induces logic programs
from examples and background knowledge. We introduce the higher-order
refactoring problem, where the goal is to compress a logic program by
introducing higher-order abstractions. We implement our approach in STEVIE,
which formulates the higher-order refactoring problem as a constraint
optimisation problem. Our experimental results on multiple domains, including
program synthesis and visual reasoning, show that, compared to no refactoring,
STEVIE can improve predictive accuracies by 27% and reduce learning times by
47%. We also show that STEVIE can discover abstractions that transfer to
different domain
LILO: Learning Interpretable Libraries by Compressing and Documenting Code
While large language models (LLMs) now excel at code generation, a key aspect
of software development is the art of refactoring: consolidating code into
libraries of reusable and readable programs. In this paper, we introduce LILO,
a neurosymbolic framework that iteratively synthesizes, compresses, and
documents code to build libraries tailored to particular problem domains. LILO
combines LLM-guided program synthesis with recent algorithmic advances in
automated refactoring from Stitch: a symbolic compression system that
efficiently identifies optimal lambda abstractions across large code corpora.
To make these abstractions interpretable, we introduce an auto-documentation
(AutoDoc) procedure that infers natural language names and docstrings based on
contextual examples of usage. In addition to improving human readability, we
find that AutoDoc boosts performance by helping LILO's synthesizer to interpret
and deploy learned abstractions. We evaluate LILO on three inductive program
synthesis benchmarks for string editing, scene reasoning, and graphics
composition. Compared to existing neural and symbolic methods - including the
state-of-the-art library learning algorithm DreamCoder - LILO solves more
complex tasks and learns richer libraries that are grounded in linguistic
knowledge
Az iskolában elsajátított tudás minőségének értékelése = Evaluation of the quality of students' knowledge
Az iskolában elsajátított tudás szerveződésének feltárása érdekében a természettudományos fogalmakat a középpontba állítva vizsgáltuk a fogalmak fejlődését. Megjelent a vizsgálatok eredményeit összegző könyv. Több elemzés mutatta be a tárgyi tudás szerveződésének merevségeit, a többszörös reprezentáció hiányát, és az ebből fakadó alkalmazási problémákat. Sor került a központi jelentőségűnek bizonyult képességek (induktív gondolkodás, analógiás gondolkodás) tanulásban játszott konkrét szerepének kimutatására, a fejlesztési lehetőségek feltérképezésére (pl. problémamegoldás), és fejlesztő kísérletekre (pl. az induktív gondolkodás fejlesztése, az analógiás gondolkodás), továbbá azoknak az iskolában alkalmazható tanítási módszereknek a kipróbálására, amelyek a képességek hatékonyabb fejlesztését segítik (probléma alapú tanítás, projekt módszer, a feladatlapok és az IKT eszközök alkalmazása). Megjelent az analógiás gondolkodás fejlesztésével kapcsolatos kísérleteket bemutató könyv. A komplex problémamegoldás terén a nemzetközi elemzéseknél részletesebb adatfelvételre kerül sor, több korosztály felmérésével lehetőség nyílt a fejlődés elemzésére. Megjelent a vizsgálatokat bemutató könyv. A program több téren továbbfejlesztette a tudás szerveződésével és a képességek fejlődésével kapcsolatos kutatások elméleti kereteit és módszertanát, például a kompetencia-skálák készítésének elméleti megalapozása és a széles életkori intervallumot átfogó fejlődéséi skálák készítése terén. | The main focus of the project has been the study of the organization of knowledge mastered at school. Both propositional knowledge (concept organization, conceptual change) and propositional knowledge (analogical reasoning, inductive reasoning, complex problem solving), as well as the interaction of the two forms of knowledge have been studied. The empirical studies revealed the problems of conceptual change that may be attributed to the lack of deeper understanding and multiple representations. Beyond assessing the development of skills, intervention experiments were also carried out. The intervention methods resulted in significant developmental changes in analogical and inductive reasoning. The research project resulted in a new synthesis of the forms of organization of school knowledge and contributed to the development of methods e.g. to the theoretical bases of creating competency scales and establishing developmental scales by the means of IRT
IRE: A Framework For Inductive Reverse Engineering
abstract: Reverse engineering is critical to reasoning about how a system behaves. While complete access to a system inherently allows for perfect analysis, partial access is inherently uncertain. This is the case foran individual agent in a distributed system. Inductive Reverse Engineering (IRE) enables analysis under
such circumstances. IRE does this by producing program spaces consistent with individual input-output examples for a given domain-specific language. Then, IRE intersects those program spaces to produce a generalized program consistent with all examples. IRE, an easy to use framework, allows this domain-specific language to be specified in the form of Theorist s, which produce Theory s, a succinct way of representing the program space.
Programs are often much more complex than simple string transformations. One of the ways in which they are more complex is in the way that they follow a conversation-like behavior, potentially following some underlying protocol. As a result, IRE represents program interactions as Conversations in order to
more correctly model a distributed system. This, for instance, enables IRE to model dynamically captured inputs received from other agents in the distributed system.
While domain-specific knowledge provided by a user is extremely valuable, such information is not always possible. IRE mitigates this by automatically inferring program grammars, allowing it to still perform efficient searches of the program space. It does this by intersecting conversations prior to synthesis in order to understand what portions of conversations are constant.
IRE exists to be a tool that can aid in automatic reverse engineering across numerous domains. Further, IRE aspires to be a centralized location and interface for implementing program synthesis and automatic black box analysis techniques.Dissertation/ThesisMasters Thesis Computer Science 201
Middle-Out Reasoning for Logic Program Synthesis
We propose a novel approach to automating the synthesis of logic programs: Logic programs are synthesized as a by-product of the planning of a verification proof. The approach is a two-level one: At the object level, we prove program verification conjectures in a sorted, first-order theory. The conjectures are of the form 8args \Gamma\Gamma\Gamma\Gamma! : prog(args \Gamma\Gamma\Gamma\Gamma! ) $ spec(args \Gamma\Gamma\Gamma\Gamma! ). At the meta-level, we plan the object-level verification with an unspecified program definition. The definition is represented with a (second-order) meta-level variable, which becomes instantiated in the course of the planning
Event-Object Reasoning with Curated Knowledge Bases: Deriving Missing Information
The broader goal of our research is to formulate answers to why and how
questions with respect to knowledge bases, such as AURA. One issue we face when
reasoning with many available knowledge bases is that at times needed
information is missing. Examples of this include partially missing information
about next sub-event, first sub-event, last sub-event, result of an event,
input to an event, destination of an event, and raw material involved in an
event. In many cases one can recover part of the missing knowledge through
reasoning. In this paper we give a formal definition about how such missing
information can be recovered and then give an ASP implementation of it. We then
discuss the implication of this with respect to answering why and how
questions.Comment: 13 page
- …