Visual Causal Feature Learning

We provide a rigorous definition of the visual cause of a behavior that is broadly applicable to the visually driven behavior in humans, animals, neurons, robots and other perceiving systems. Our framework generalizes standard accounts of causal learning to settings in which the causal variables need to be constructed from micro-variables. We prove the Causal Coarsening Theorem, which allows us to gain causal knowledge from observational data with minimal experimental effort. The theorem provides a connection to standard inference techniques in machine learning that identify features of an image that correlate with, but may not cause, the target behavior. Finally, we propose an active learning scheme to learn a manipulator function that performs optimal manipulations on the image to automatically identify the visual cause of a target behavior. We illustrate our inference and learning algorithms in experiments based on both synthetic and real data.Comment: Accepted at UAI 201

Reversal of age-related learning deficiency by the vertebrate PACAP and IGF-1 in a novel invertebrate model of aging: the pond snail (Lymnaea Stagnalis)

With the increase of life span, nonpathological age-related memory decline is affecting an increasing number of people. However, there is evidence that age-associated memory impairment only suspends, rather than irreversibly extinguishes, the intrinsic capacity of the aging nervous system for plasticity (1). Here, using a molluscan model system, we show that the age-related decline in memory performance can be reversed by administration of the pituitary adenylate cyclase activating polypeptide (PACAP). Our earlier findings showed that a homolog of the vertebrate PACAP38 and its receptors exist in the pond snail (Lymnaea stagnalis) brain (2), and it is both necessary and instructive for memory formation after reward conditioning in young animals (3). Here we show that exogenous PACAP38 boosts memory formation in aged Lymnaea, where endogenous PACAP38 levels are low in the brain. Treatment with insulin-like growth factor-1, which in vertebrates was shown to transactivate PACAP type I (PAC1) receptors (4) also boosts memory formation in aged pond snails. Due to the evolutionarily conserved nature of these polypeptides and their established role in memory and synaptic plasticity, there is a very high probability that they could also act as “memory rejuvenating” agents in humans

To Fall Or Not To Fall: A Visual Approach to Physical Stability Prediction

Understanding physical phenomena is a key competence that enables humans and animals to act and interact under uncertain perception in previously unseen environments containing novel object and their configurations. Developmental psychology has shown that such skills are acquired by infants from observations at a very early stage. In this paper, we contrast a more traditional approach of taking a model-based route with explicit 3D representations and physical simulation by an end-to-end approach that directly predicts stability and related quantities from appearance. We ask the question if and to what extent and quality such a skill can directly be acquired in a data-driven way bypassing the need for an explicit simulation. We present a learning-based approach based on simulated data that predicts stability of towers comprised of wooden blocks under different conditions and quantities related to the potential fall of the towers. The evaluation is carried out on synthetic data and compared to human judgments on the same stimuli

Subitizing with Variational Autoencoders

Numerosity, the number of objects in a set, is a basic property of a given visual scene. Many animals develop the perceptual ability to subitize: the near-instantaneous identification of the numerosity in small sets of visual items. In computer vision, it has been shown that numerosity emerges as a statistical property in neural networks during unsupervised learning from simple synthetic images. In this work, we focus on more complex natural images using unsupervised hierarchical neural networks. Specifically, we show that variational autoencoders are able to spontaneously perform subitizing after training without supervision on a large amount images from the Salient Object Subitizing dataset. While our method is unable to outperform supervised convolutional networks for subitizing, we observe that the networks learn to encode numerosity as basic visual property. Moreover, we find that the learned representations are likely invariant to object area; an observation in alignment with studies on biological neural networks in cognitive neuroscience

Reasoning on transition from manipulative strategies to general procedures in solving counting problems

We describe the procedures used by 11- to 12-year-old students for solving basic counting problems in order to analyse the transition from manipulative strategies involving direct counting to the use of the multiplication principle as a general procedure in combinatorial problems. In this transition, the students sometimes spontaneously use tree diagrams and sometimes use numerical thinking strategies. We relate the findings of our research to recent research on the representational formats on the learning of combinatorics, and reflect on the didactic implications of these investigations

A Full Probabilistic Model for Yes/No Type Crowdsourcing in Multi-Class Classification

Crowdsourcing has become widely used in supervised scenarios where training sets are scarce and difficult to obtain. Most crowdsourcing models in the literature assume labelers can provide answers to full questions. In classification contexts, full questions require a labeler to discern among all possible classes. Unfortunately, discernment is not always easy in realistic scenarios. Labelers may not be experts in differentiating all classes. In this work, we provide a full probabilistic model for a shorter type of queries. Our shorter queries only require "yes" or "no" responses. Our model estimates a joint posterior distribution of matrices related to labelers' confusions and the posterior probability of the class of every object. We developed an approximate inference approach, using Monte Carlo Sampling and Black Box Variational Inference, which provides the derivation of the necessary gradients. We built two realistic crowdsourcing scenarios to test our model. The first scenario queries for irregular astronomical time-series. The second scenario relies on the image classification of animals. We achieved results that are comparable with those of full query crowdsourcing. Furthermore, we show that modeling labelers' failures plays an important role in estimating true classes. Finally, we provide the community with two real datasets obtained from our crowdsourcing experiments. All our code is publicly available.Comment: SIAM International Conference on Data Mining (SDM19), 9 official pages, 5 supplementary page

Salient object subitizing

We study the problem of salient object subitizing, i.e. predicting the existence and the number of salient objects in an image using holistic cues. This task is inspired by the ability of people to quickly and accurately identify the number of items within the subitizing range (1–4). To this end, we present a salient object subitizing image dataset of about 14 K everyday images which are annotated using an online crowdsourcing marketplace. We show that using an end-to-end trained convolutional neural network (CNN) model, we achieve prediction accuracy comparable to human performance in identifying images with zero or one salient object. For images with multiple salient objects, our model also provides significantly better than chance performance without requiring any localization process. Moreover, we propose a method to improve the training of the CNN subitizing model by leveraging synthetic images. In experiments, we demonstrate the accuracy and generalizability of our CNN subitizing model and its applications in salient object detection and image retrieval.This research was supported in part by US NSF Grants 0910908 and 1029430, and gifts from Adobe and NVIDIA. (0910908 - US NSF; 1029430 - US NSF)https://arxiv.org/abs/1607.07525https://arxiv.org/pdf/1607.07525.pdfAccepted manuscrip