7,580 research outputs found
Building Machines That Learn and Think Like People
Recent progress in artificial intelligence (AI) has renewed interest in
building systems that learn and think like people. Many advances have come from
using deep neural networks trained end-to-end in tasks such as object
recognition, video games, and board games, achieving performance that equals or
even beats humans in some respects. Despite their biological inspiration and
performance achievements, these systems differ from human intelligence in
crucial ways. We review progress in cognitive science suggesting that truly
human-like learning and thinking machines will have to reach beyond current
engineering trends in both what they learn, and how they learn it.
Specifically, we argue that these machines should (a) build causal models of
the world that support explanation and understanding, rather than merely
solving pattern recognition problems; (b) ground learning in intuitive theories
of physics and psychology, to support and enrich the knowledge that is learned;
and (c) harness compositionality and learning-to-learn to rapidly acquire and
generalize knowledge to new tasks and situations. We suggest concrete
challenges and promising routes towards these goals that can combine the
strengths of recent neural network advances with more structured cognitive
models.Comment: In press at Behavioral and Brain Sciences. Open call for commentary
proposals (until Nov. 22, 2016).
https://www.cambridge.org/core/journals/behavioral-and-brain-sciences/information/calls-for-commentary/open-calls-for-commentar
Reconstruction Bottlenecks in Object-Centric Generative Models
A range of methods with suitable inductive biases exist to learn
interpretable object-centric representations of images without supervision.
However, these are largely restricted to visually simple images; robust object
discovery in real-world sensory datasets remains elusive. To increase the
understanding of such inductive biases, we empirically investigate the role of
"reconstruction bottlenecks" for scene decomposition in GENESIS, a recent
VAE-based model. We show such bottlenecks determine reconstruction and
segmentation quality and critically influence model behaviour.Comment: 10 pages, 7 Figures, Workshop on Object-Oriented Learning at ICML
202
Backwards is the way forward: feedback in the cortical hierarchy predicts the expected future
Clark offers a powerful description of the brain as a prediction machine, which offers progress on two distinct levels. First, on an abstract conceptual level, it provides a unifying framework for perception, action, and cognition (including subdivisions such as attention, expectation, and imagination). Second, hierarchical prediction offers progress on a concrete descriptive level for testing and constraining conceptual elements and mechanisms of predictive coding models (estimation of predictions, prediction errors, and internal models)
Image Representations in Deep Neural Networks and their Applications to Neural Data Modelling
Over the last decade, deep neural networks (DNNs) have become a standard tool in computer vision, allowing us to tackle a variety of problems from classifying objects in natural images to generating new images to predicting brain activity. Such a wide applicability of DNNs is something that these models have in common with the human vision, and exploring some of these similarities is the goal of this thesis.
DNNs much like human vision are hierarchical models that process an input scene with a series of sequential computations. It has been shown that typically only a few final computations in this hierarchy are problem-specific, while the rest of them are quite general and applicable to a number of problems. The results of intermediate computations in the DNN are often referred to as image representations and their generality is another similarity to human vision which also has general visual areas (e.g. primary visual cortex) projecting further to the specialised ones solving specific visual tasks.
We focus on studying DNN image representations with the goal of understanding what makes them so useful for a variety of visual problems. To do so, we discuss DNNs solving a number of specific computer vision problems and analyse similarities and differences of their image representations. Moreover, we discuss how to build DNNs providing image representations with specific properties which enables us to build a "digital twin" of the mouse primary visual system to be used as a tool for studying the computations in the brain.
Taking these results together, we concluded that in general we are still lacking a good understanding of DNN representations. Despite the progress on some specific problems, it still remains largely an open question how the image information is organised in these representations and how to use it for solving arbitrary visual problems. However, we also argue that thinking of DNNs as "digital twins" might be a promising framework for addressing these issues in the future DNN research as they allow us to study image representations by means of computational experiments rather than rely on a priori ideas of how these representations are structured which has proven to be quite challenging
Neural Block-Slot Representations
In this paper, we propose a novel object-centric representation, called
Block-Slot Representation. Unlike the conventional slot representation, the
Block-Slot Representation provides concept-level disentanglement within a slot.
A block-slot is constructed by composing a set of modular concept
representations, called blocks, generated from a learned memory of abstract
concept prototypes. We call this block-slot construction process Block-Slot
Attention. Block-Slot Attention facilitates the emergence of abstract concept
blocks within a slot such as color, position, and texture, without any
supervision. This brings the benefits of disentanglement into slots and the
representation becomes more interpretable. Similar to Slot Attention, this
mechanism can be used as a drop-in module in any arbitrary neural architecture.
In experiments, we show that our model disentangles object properties
significantly better than the previous methods, including complex textured
scenes. We also demonstrate the ability to compose novel scenes by composing
slots at the block-level
Object-Centric Learning with Slot Attention
Learning object-centric representations of complex scenes is a promising step
towards enabling efficient abstract reasoning from low-level perceptual
features. Yet, most deep learning approaches learn distributed representations
that do not capture the compositional properties of natural scenes. In this
paper, we present the Slot Attention module, an architectural component that
interfaces with perceptual representations such as the output of a
convolutional neural network and produces a set of task-dependent abstract
representations which we call slots. These slots are exchangeable and can bind
to any object in the input by specializing through a competitive procedure over
multiple rounds of attention. We empirically demonstrate that Slot Attention
can extract object-centric representations that enable generalization to unseen
compositions when trained on unsupervised object discovery and supervised
property prediction tasks
- …