1,749 research outputs found
Learning to Infer Graphics Programs from Hand-Drawn Images
We introduce a model that learns to convert simple hand drawings into
graphics programs written in a subset of \LaTeX. The model combines techniques
from deep learning and program synthesis. We learn a convolutional neural
network that proposes plausible drawing primitives that explain an image. These
drawing primitives are like a trace of the set of primitive commands issued by
a graphics program. We learn a model that uses program synthesis techniques to
recover a graphics program from that trace. These programs have constructs like
variable bindings, iterative loops, or simple kinds of conditionals. With a
graphics program in hand, we can correct errors made by the deep network,
measure similarity between drawings by use of similar high-level geometric
structures, and extrapolate drawings. Taken together these results are a step
towards agents that induce useful, human-readable programs from perceptual
input
Improving Unsupervised Visual Program Inference with Code Rewriting Families
Programs offer compactness and structure that makes them an attractive
representation for visual data. We explore how code rewriting can be used to
improve systems for inferring programs from visual data. We first propose
Sparse Intermittent Rewrite Injection (SIRI), a framework for unsupervised
bootstrapped learning. SIRI sparsely applies code rewrite operations over a
dataset of training programs, injecting the improved programs back into the
training set. We design a family of rewriters for visual programming domains:
parameter optimization, code pruning, and code grafting. For three shape
programming languages in 2D and 3D, we show that using SIRI with our family of
rewriters improves performance: better reconstructions and faster convergence
rates, compared with bootstrapped learning methods that do not use rewriters or
use them naively. Finally, we demonstrate that our family of rewriters can be
effectively used at test time to improve the output of SIRI predictions. For 2D
and 3D CSG, we outperform or match the reconstruction performance of recent
domain-specific neural architectures, while producing more parsimonious
programs that use significantly fewer primitives.Comment: Accepted at ICCV 23 (oral). Website:
https://bardofcodes.github.io/coref
Interpretable and Explainable Logical Policies via Neurally Guided Symbolic Abstraction
The limited priors required by neural networks make them the dominating
choice to encode and learn policies using reinforcement learning (RL). However,
they are also black-boxes, making it hard to understand the agent's behaviour,
especially when working on the image level. Therefore, neuro-symbolic RL aims
at creating policies that are interpretable in the first place. Unfortunately,
interpretability is not explainability. To achieve both, we introduce Neurally
gUided Differentiable loGic policiEs (NUDGE). NUDGE exploits trained neural
network-based agents to guide the search of candidate-weighted logic rules,
then uses differentiable logic to train the logic agents. Our experimental
evaluation demonstrates that NUDGE agents can induce interpretable and
explainable policies while outperforming purely neural ones and showing good
flexibility to environments of different initial states and problem sizes.Comment: 9 main pages + appendix (19 in total
A Connectionist Theory of Phenomenal Experience
When cognitive scientists apply computational theory to the problem of phenomenal consciousness, as
many of them have been doing recently, there are two fundamentally distinct approaches available. Either
consciousness is to be explained in terms of the nature of the representational vehicles the brain deploys; or
it is to be explained in terms of the computational processes defined over these vehicles. We call versions of
these two approaches vehicle and process theories of consciousness, respectively. However, while there may
be space for vehicle theories of consciousness in cognitive science, they are relatively rare. This is because
of the influence exerted, on the one hand, by a large body of research which purports to show that the
explicit representation of information in the brain and conscious experience are dissociable, and on the
other, by the classical computational theory of mind – the theory that takes human cognition to be a species
of symbol manipulation. But two recent developments in cognitive science combine to suggest that a
reappraisal of this situation is in order. First, a number of theorists have recently been highly critical of the
experimental methodologies employed in the dissociation studies – so critical, in fact, it’s no longer
reasonable to assume that the dissociability of conscious experience and explicit representation has been
adequately demonstrated. Second, classicism, as a theory of human cognition, is no longer as dominant in
cognitive science as it once was. It now has a lively competitor in the form of connectionism; and
connectionism, unlike classicism, does have the computational resources to support a robust vehicle theory
of consciousness. In this paper we develop and defend this connectionist vehicle theory of consciousness. It
takes the form of the following simple empirical hypothesis: phenomenal experience consists in the explicit
representation of information in neurally realized PDP networks. This hypothesis leads us to re-assess some
common wisdom about consciousness, but, we will argue, in fruitful and ultimately plausible ways
A Defence of Cartesian Materialism
One of the principal tasks Dennett sets himself in "Consciousness Explained" is to demolish the Cartesian theatre model of phenomenal consciousness, which in its contemporary garb takes the form of Cartesian materialism: the idea that conscious experience is a process of presentation realized in the physical materials of the brain. The now standard response to Dennett is that, in focusing on Cartesian materialism, he attacks an impossibly naive account of consciousness held by no one currently working in cognitive science or the philosophy of mind. Our response is quite different. We believe that, once properly formulated, Cartesian materialism is no straw man. Rather, it is an attractive hypothesis about the relationship between the computational architecture of the brain and phenomenal consciousness, and hence one that is worthy of further exploration. Consequently, our primary aim in this paper is to defend Cartesian materialism from Dennett's assault. We do this by showing that Dennett's argument against this position is founded on an implicit assumption (about the relationship between phenomenal experience and information coding in the brain), which while valid in the context of classical cognitive science, is not forced on connectionism
Leveraging Language to Learn Program Abstractions and Search Heuristics
Inductive program synthesis, or inferring programs from examples of desired
behavior, offers a general paradigm for building interpretable, robust, and
generalizable machine learning systems. Effective program synthesis depends on
two key ingredients: a strong library of functions from which to build
programs, and an efficient search strategy for finding programs that solve a
given task. We introduce LAPS (Language for Abstraction and Program Search), a
technique for using natural language annotations to guide joint learning of
libraries and neurally-guided search models for synthesis. When integrated into
a state-of-the-art library learning system (DreamCoder), LAPS produces
higher-quality libraries and improves search efficiency and generalization on
three domains -- string editing, image composition, and abstract reasoning
about scenes -- even when no natural language hints are available at test time.Comment: appeared in Thirty-eighth International Conference on Machine
Learning (ICML 2021
Approaching human 3D shape perception with neurally mappable models
Humans effortlessly infer the 3D shape of objects. What computations underlie
this ability? Although various computational models have been proposed, none of
them capture the human ability to match object shape across viewpoints. Here,
we ask whether and how this gap might be closed. We begin with a relatively
novel class of computational models, 3D neural fields, which encapsulate the
basic principles of classic analysis-by-synthesis in a deep neural network
(DNN). First, we find that a 3D Light Field Network (3D-LFN) supports 3D
matching judgments well aligned to humans for within-category comparisons,
adversarially-defined comparisons that accentuate the 3D failure cases of
standard DNN models, and adversarially-defined comparisons for algorithmically
generated shapes with no category structure. We then investigate the source of
the 3D-LFN's ability to achieve human-aligned performance through a series of
computational experiments. Exposure to multiple viewpoints of objects during
training and a multi-view learning objective are the primary factors behind
model-human alignment; even conventional DNN architectures come much closer to
human behavior when trained with multi-view objectives. Finally, we find that
while the models trained with multi-view learning objectives are able to
partially generalize to new object categories, they fall short of human
alignment. This work provides a foundation for understanding human shape
inferences within neurally mappable computational architectures and highlights
important questions for future work
- …