1,816 research outputs found
Empiricism without Magic: Transformational Abstraction in Deep Convolutional Neural Networks
In artificial intelligence, recent research has demonstrated the remarkable potential of Deep Convolutional Neural Networks (DCNNs), which seem to exceed state-of-the-art performance in new domains weekly, especially on the sorts of very difficult perceptual discrimination tasks that skeptics thought would remain beyond the reach of artificial intelligence. However, it has proven difficult to explain why DCNNs perform so well. In philosophy of mind, empiricists have long suggested that complex cognition is based on information derived from sensory experience, often appealing to a faculty of abstraction. Rationalists have frequently complained, however, that empiricists never adequately explained how this faculty of abstraction actually works. In this paper, I tie these two questions together, to the mutual benefit of both disciplines. I argue that the architectural features that distinguish DCNNs from earlier neural networks allow them to implement a form of hierarchical processing that I call âtransformational abstractionâ. Transformational abstraction iteratively converts sensory-based representations of category exemplars into new formats that are increasingly tolerant to ânuisance variationâ in input. Reflecting upon the way that DCNNs leverage a combination of linear and non-linear processing to efficiently accomplish this feat allows us to understand how the brain is capable of bi-directional travel between exemplars and abstractions, addressing longstanding problems in empiricist philosophy of mind. I end by considering the prospects for future research on DCNNs, arguing that rather than simply implementing 80s connectionism with more brute-force computation, transformational abstraction counts as a qualitatively distinct form of processing ripe with philosophical and psychological significance, because it is significantly better suited to depict the generic mechanism responsible for this important kind of psychological processing in the brain
Active inference, stressors, and psychological trauma: A neuroethological model of (mal)adaptive explore-exploit dynamics in ecological context
This paper offers a formal account of emotional inference and stress-related behaviour, using the notion of active inference. We formulate responses to stressful scenarios in terms of Bayesian belief-updating and subsequent policy selection; namely, planning as (active) inference. Using a minimal model of how creatures or subjects account for their sensations (and subsequent action), we deconstruct the sequences of belief updating and behaviour that underwrite stress-related responses â and simulate the aberrant responses of the sort seen in post-traumatic stress disorder (PTSD). Crucially, the model used for belief-updating generates predictions in multiple (exteroceptive, proprioceptive and interoceptive) modalities, to provide an integrated account of evidence accumulation and multimodal integration that has consequences for both motor and autonomic responses. The ensuing phenomenology speaks to many constructs in the ecological and clinical literature on stress, which we unpack with reference to simulated inference processes and accompanying neuronal responses. A key insight afforded by this formal approach rests on the trade-off between the epistemic affordance of certain cues (that resolve uncertainty about states of affairs in the environment) and the consequences of epistemic foraging (that may be in conflict with the instrumental or pragmatic value of âfleeingâ or âfreezingâ). Starting from first principles, we show how this trade-off is nuanced by prior (subpersonal) beliefs about the outcomes of behaviour â beliefs that, when held with unduly high precision, can lead to (Bayes optimal) responses that closely resemble PTSD
Object Detection Through Exploration With A Foveated Visual Field
We present a foveated object detector (FOD) as a biologically-inspired
alternative to the sliding window (SW) approach which is the dominant method of
search in computer vision object detection. Similar to the human visual system,
the FOD has higher resolution at the fovea and lower resolution at the visual
periphery. Consequently, more computational resources are allocated at the
fovea and relatively fewer at the periphery. The FOD processes the entire
scene, uses retino-specific object detection classifiers to guide eye
movements, aligns its fovea with regions of interest in the input image and
integrates observations across multiple fixations. Our approach combines modern
object detectors from computer vision with a recent model of peripheral pooling
regions found at the V1 layer of the human visual system. We assessed various
eye movement strategies on the PASCAL VOC 2007 dataset and show that the FOD
performs on par with the SW detector while bringing significant computational
cost savings.Comment: An extended version of this manuscript was published in PLOS
Computational Biology (October 2017) at
https://doi.org/10.1371/journal.pcbi.100574
Categorical Perception: A Groundwork for Deep Learning
A well-known perceptual consequence of categorization in humans and other
animals, called categorical perception, is notably characterized by a
within-category compression and a between-category separation: two items, close
in input space, are perceived closer if they belong to the same category than
if they belong to different categories. Elaborating on experimental and
theoretical results in cognitive science, here we study categorical effects in
artificial neural networks. We combine a theoretical analysis that makes use of
mutual and Fisher information quantities, and a series of numerical simulations
on networks of increasing complexity. These formal and numerical analyses
provide insights into the geometry of the neural representation in deep layers,
with expansion of space near category boundaries and contraction far from
category boundaries. We investigate categorical representation by using two
complementary approaches: one mimics experiments in psychophysics and cognitive
neuroscience by means of morphed continua between stimuli of different
categories, while the other introduces a categoricality index that, for each
layer in the network, quantifies the separability of the categories at the
neural population level. We show on both shallow and deep neural networks that
category learning automatically induces categorical perception. We further show
that the deeper a layer, the stronger the categorical effects. As an outcome of
our study, we propose a coherent view of the efficacy of different heuristic
practices of the dropout regularization technique. More generally, our view,
which finds echoes in the neuroscience literature, insists on the differential
impact of noise in any given layer depending on the geometry of the neural
representation that is being learned, i.e. on how this geometry reflects the
structure of the categories
- âŠ