9 research outputs found
Fast Matrix Multiplication Without Tears: A Constraint Programming Approach
It is known that the multiplication of an matrix with an matrix can be performed using fewer multiplications than what the
naive approach suggests. The most famous instance of this is Strassen's
algorithm for multiplying two matrices in 7 instead of 8
multiplications. This gives rise to the constraint satisfaction problem of fast
matrix multiplication, where a set of multiplication terms must be
chosen and combined such that they satisfy correctness constraints on the
output matrix. Despite its highly combinatorial nature, this problem has not
been exhaustively examined from that perspective, as evidenced for example by
the recent deep reinforcement learning approach of AlphaTensor. In this work,
we propose a simple yet novel Constraint Programming approach to find
non-commutative algorithms for fast matrix multiplication or provide proof of
infeasibility otherwise. We propose a set of symmetry-breaking constraints and
valid inequalities that are particularly helpful in proving infeasibility. On
the feasible side, we find that exploiting solver performance variability in
conjunction with a sparsity-based problem decomposition enables finding
solutions for larger (feasible) instances of fast matrix multiplication. Our
experimental results using CP Optimizer demonstrate that we can find fast
matrix multiplication algorithms for matrices up to in a short
amount of time
LLMs and the Abstraction and Reasoning Corpus: Successes, Failures, and the Importance of Object-based Representations
Can a Large Language Model (LLM) solve simple abstract reasoning problems? We
explore this broad question through a systematic analysis of GPT on the
Abstraction and Reasoning Corpus (ARC), a representative benchmark of abstract
reasoning ability from limited examples in which solutions require some "core
knowledge" of concepts such as objects, goal states, counting, and basic
geometry. GPT-4 solves only 13/50 of the most straightforward ARC tasks when
using textual encodings for their two-dimensional input-output grids. Our
failure analysis reveals that GPT-4's capacity to identify objects and reason
about them is significantly influenced by the sequential nature of the text
that represents an object within a text encoding of a task. To test this
hypothesis, we design a new benchmark, the 1D-ARC, which consists of
one-dimensional (array-like) tasks that are more conducive to GPT-based
reasoning, and where it indeed performs better than on the (2D) ARC. To
alleviate this issue, we propose an object-based representation that is
obtained through an external tool, resulting in nearly doubling the performance
on solved ARC tasks and near-perfect scores on the easier 1D-ARC. Although the
state-of-the-art GPT-4 is unable to "reason" perfectly within non-language
domains such as the 1D-ARC or a simple ARC subset, our study reveals that the
use of object-based representations can significantly improve its reasoning
ability. Visualizations, GPT logs, and data are available at
https://khalil-research.github.io/LLM4ARC.Comment: 17 pages, 11 figure
Noisy Symbolic Abstractions for Deep RL: A case study with Reward Machines
Natural and formal languages provide an effective mechanism for humans to
specify instructions and reward functions. We investigate how to generate
policies via RL when reward functions are specified in a symbolic language
captured by Reward Machines, an increasingly popular automaton-inspired
structure. We are interested in the case where the mapping of environment state
to a symbolic (here, Reward Machine) vocabulary -- commonly known as the
labelling function -- is uncertain from the perspective of the agent. We
formulate the problem of policy learning in Reward Machines with noisy symbolic
abstractions as a special class of POMDP optimization problem, and investigate
several methods to address the problem, building on existing and new
techniques, the latter focused on predicting Reward Machine state, rather than
on grounding of individual symbols. We analyze these methods and evaluate them
experimentally under varying degrees of uncertainty in the correct
interpretation of the symbolic vocabulary. We verify the strength of our
approach and the limitation of existing methods via an empirical investigation
on both illustrative, toy domains and partially observable, deep RL domains.Comment: NeurIPS Deep Reinforcement Learning Workshop 202
Learning to Follow Instructions in Text-Based Games
Text-based games present a unique class of sequential decision making problem
in which agents interact with a partially observable, simulated environment via
actions and observations conveyed through natural language. Such observations
typically include instructions that, in a reinforcement learning (RL) setting,
can directly or indirectly guide a player towards completing reward-worthy
tasks. In this work, we study the ability of RL agents to follow such
instructions. We conduct experiments that show that the performance of
state-of-the-art text-based game agents is largely unaffected by the presence
or absence of such instructions, and that these agents are typically unable to
execute tasks to completion. To further study and address the task of
instruction following, we equip RL agents with an internal structured
representation of natural language instructions in the form of Linear Temporal
Logic (LTL), a formal language that is increasingly used for temporally
extended reward specification in RL. Our framework both supports and highlights
the benefit of understanding the temporal semantics of instructions and in
measuring progress towards achievement of such a temporally extended behaviour.
Experiments with 500+ games in TextWorld demonstrate the superior performance
of our approach.Comment: NeurIPS 202
Lifted unit propagation
Recent emergence of effective solvers for propositional satisfiability (SAT) and related problems has led to new methods for solving computationally challenging industrial problems, such as NP-hard search problems in planning, software design, and hardware verification. This has produced a demand for tools which allow users to write high level problem specifications which are automatically reduced to SAT. We consider the case of specifications in first order logic with reduction to SAT by grounding. For realistic problems, the resulting SAT instances can be prohibitively large. A key technique in SAT solvers is unit propagation, which often significantly reduces instance size before search for a solution begins. We define ”lifted unit propagation”, which is executed before grounding. We show that instances produced by a grounding algorithm with lifted unit propagation are never larger than those produced by normal grounding followed by UP, and demonstrate experimentally that they are sometimes much smaller
Learning Branching Heuristics for Propositional Model Counting
Propositional model counting, or #SAT, is the problem of computing the number
of satisfying assignments of a Boolean formula. Many problems from different
application areas, including many discrete probabilistic inference problems,
can be translated into model counting problems to be solved by #SAT solvers.
Exact #SAT solvers, however, are often not scalable to industrial size
instances. In this paper, we present Neuro#, an approach for learning branching
heuristics to improve the performance of exact #SAT solvers on instances from a
given family of problems. We experimentally show that our method reduces the
step count on similarly distributed held-out instances and generalizes to much
larger instances from the same problem family. It is able to achieve these
results on a number of different problem families having very different
structures. In addition to step count improvements, Neuro# can also achieve
orders of magnitude wall-clock speedups over the vanilla solver on larger
instances in some problem families, despite the runtime overhead of querying
the model