125 research outputs found
Improvements to context based self-supervised learning
We develop a set of methods to improve on the results of self-supervised
learning using context. We start with a baseline of patch based arrangement
context learning and go from there. Our methods address some overt problems
such as chromatic aberration as well as other potential problems such as
spatial skew and mid-level feature neglect. We prevent problems with testing
generalization on common self-supervised benchmark tests by using different
datasets during our development. The results of our methods combined yield top
scores on all standard self-supervised benchmarks, including classification and
detection on PASCAL VOC 2007, segmentation on PASCAL VOC 2012, and "linear
tests" on the ImageNet and CSAIL Places datasets. We obtain an improvement over
our baseline method of between 4.0 to 7.1 percentage points on transfer
learning classification tests. We also show results on different standard
network architectures to demonstrate generalization as well as portability. All
data, models and programs are available at:
https://gdo-datasci.llnl.gov/selfsupervised/.Comment: Accepted paper at CVPR 201
A bottom–up model of spatial attention predicts human error patterns in rapid scene recognition
Humans demonstrate a peculiar ability to detect complex targets in rapidly presented natural scenes. Recent studies suggest that (nearly) no focal attention is required for overall performance in such tasks. Little is known, however, of how detection performance varies from trial to trial and which stages in the processing hierarchy limit performance: bottom–up visual processing (attentional selection and/or recognition) or top–down factors (e.g., decision-making, memory, or alertness fluctuations)? To investigate the relative contribution of these factors, eight human observers performed an animal detection task in natural scenes presented at 20 Hz. Trial-by-trial performance was highly consistent across observers, far exceeding the prediction of independent errors. This consistency demonstrates that performance is not primarily limited by idiosyncratic factors but by visual processing. Two statistical stimulus properties, contrast variation in the target image and the information-theoretical measure of “surprise” in adjacent images, predict performance on a trial-by-trial basis. These measures are tightly related to spatial attention, demonstrating that spatial attention and rapid target detection share common mechanisms. To isolate the causal contribution of the surprise measure, eight additional observers performed the animal detection task in sequences that were reordered versions of those all subjects had correctly recognized in the first experiment. Reordering increased surprise before and/or after the target while keeping the target and distractors themselves unchanged. Surprise enhancement impaired target detection in all observers. Consequently, and contrary to several previously published findings, our results demonstrate that attentional limitations, rather than target recognition alone, affect the detection of targets in rapidly presented visual sequences
Student Self-Assessment: A Tool for Engaging Management Students in Their Learning
This article discusses the use of student self-assessment (SSA) for formative and summative assessment in two undergraduate programs, a management program and a leadership program, to encourage students to become more engaged in their learning. Using action research, we used an iterative process of changing or refining our methods to accommodate the differences in our teaching environments, concluding that different methods may be desirable in different environments, and that students appear to benefit from SSA regardless of the method used. Five overlapping themes emerged in the data we collected: how SSA 1) provided students with the opportunity to see the transformative impact their educations had on them, 2) acted as a motivator to their performance, 3) encouraged them to take personal responsibility for their learning, 4) had impact on their reflections as learners, and 5) encouraged them to be more honest and self-critical about their performance
Class-Agnostic Counting
Nearly all existing counting methods are designed for a specific object
class. Our work, however, aims to create a counting model able to count any
class of object. To achieve this goal, we formulate counting as a matching
problem, enabling us to exploit the image self-similarity property that
naturally exists in object counting problems. We make the following three
contributions: first, a Generic Matching Network (GMN) architecture that can
potentially count any object in a class-agnostic manner; second, by
reformulating the counting problem as one of matching objects, we can take
advantage of the abundance of video data labeled for tracking, which contains
natural repetitions suitable for training a counting model. Such data enables
us to train the GMN. Third, to customize the GMN to different user
requirements, an adapter module is used to specialize the model with minimal
effort, i.e. using a few labeled examples, and adapting only a small fraction
of the trained parameters. This is a form of few-shot learning, which is
practical for domains where labels are limited due to requiring expert
knowledge (e.g. microbiology). We demonstrate the flexibility of our method on
a diverse set of existing counting benchmarks: specifically cells, cars, and
human crowds. The model achieves competitive performance on cell and crowd
counting datasets, and surpasses the state-of-the-art on the car dataset using
only three training images. When training on the entire dataset, the proposed
method outperforms all previous methods by a large margin.Comment: Asian Conference on Computer Vision (ACCV), 201
Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients
Discovering the underlying mathematical expressions describing a dataset is a
core challenge for artificial intelligence. This is the problem of
. Despite recent advances in training neural
networks to solve complex tasks, deep learning approaches to symbolic
regression are underexplored. We propose a framework that leverages deep
learning for symbolic regression via a simple idea: use a large model to search
the space of small models. Specifically, we use a recurrent neural network to
emit a distribution over tractable mathematical expressions and employ a novel
risk-seeking policy gradient to train the network to generate better-fitting
expressions. Our algorithm outperforms several baseline methods (including
Eureqa, the gold standard for symbolic regression) in its ability to exactly
recover symbolic expressions on a series of benchmark problems, both with and
without added noise. More broadly, our contributions include a framework that
can be applied to optimize hierarchical, variable-length objects under a
black-box performance metric, with the ability to incorporate constraints in
situ, and a risk-seeking policy gradient formulation that optimizes for
best-case performance instead of expected performance.Comment: Published at International Conference on Learning Representations,
202
Improving exploration in policy gradient search: Application to symbolic optimization
Many machine learning strategies designed to automate mathematical tasks
leverage neural networks to search large combinatorial spaces of mathematical
symbols. In contrast to traditional evolutionary approaches, using a neural
network at the core of the search allows learning higher-level symbolic
patterns, providing an informed direction to guide the search. When no labeled
data is available, such networks can still be trained using reinforcement
learning. However, we demonstrate that this approach can suffer from an early
commitment phenomenon and from initialization bias, both of which limit
exploration. We present two exploration methods to tackle these issues,
building upon ideas of entropy regularization and distribution initialization.
We show that these techniques can improve the performance, increase sample
efficiency, and lower the complexity of solutions for the task of symbolic
regression.Comment: Published in 1st Mathematical Reasoning in General Artificial
Intelligence Workshop, ICLR 202
- …