3,417 research outputs found
How direct is the link between words and images?
Current word embedding models despite their success, still suffer from their
lack of grounding in the real world. In this line of research, Gunther et al.
2022 proposed a behavioral experiment to investigate the relationship between
words and images. In their setup, participants were presented with a target
noun and a pair of images, one chosen by their model and another chosen
randomly. Participants were asked to select the image that best matched the
target noun. In most cases, participants preferred the image selected by the
model. Gunther et al., therefore, concluded the possibility of a direct link
between words and embodied experience. We took their experiment as a point of
departure and addressed the following questions. 1. Apart from utilizing
visually embodied simulation of given images, what other strategies might
subjects have used to solve this task? To what extent does this setup rely on
visual information from images? Can it be solved using purely textual
representations? 2. Do current visually grounded embeddings explain subjects'
selection behavior better than textual embeddings? 3. Does visual grounding
improve the semantic representations of both concrete and abstract words? To
address these questions, we designed novel experiments by using pre-trained
textual and visually grounded word embeddings. Our experiments reveal that
subjects' selection behavior is explained to a large extent based on purely
text-based embeddings and word-based similarities, suggesting a minor
involvement of active embodied experiences. Visually grounded embeddings
offered modest advantages over textual embeddings only in certain cases. These
findings indicate that the experiment by Gunther et al. may not be well suited
for tapping into the perceptual experience of participants, and therefore the
extent to which it measures visually grounded knowledge is unclear.Comment: Accepted in the Mental Lexicon Journal:
https://benjamins.com/catalog/m
Visual Knowledge Tracing
Each year, thousands of people learn new visual categorization tasks --
radiologists learn to recognize tumors, birdwatchers learn to distinguish
similar species, and crowd workers learn how to annotate valuable data for
applications like autonomous driving. As humans learn, their brain updates the
visual features it extracts and attend to, which ultimately informs their final
classification decisions. In this work, we propose a novel task of tracing the
evolving classification behavior of human learners as they engage in
challenging visual classification tasks. We propose models that jointly extract
the visual features used by learners as well as predicting the classification
functions they utilize. We collect three challenging new datasets from real
human learners in order to evaluate the performance of different visual
knowledge tracing methods. Our results show that our recurrent models are able
to predict the classification behavior of human learners on three challenging
medical image and species identification tasks.Comment: 14 pages, 4 figures, 14 supplemental pages, 11 supplemental figures,
accepted to European Conference on Computer Vision (ECCV) 202
Connecting Levels of Analysis in Educational Neuroscience: A Review of Multi-level Structure of Educational Neuroscience with Concrete Examples
In its origins educational neuroscience has started as an endeavor to discuss implications of neuroscience studies for education. However, it is now on its way to become a transdisciplinary field, incorporating findings, theoretical frameworks and methodologies from education, and cognitive and brain sciences. Given the differences and diversity in the originating disciplines, it has been a challenge for educational neuroscience to integrate both theoretical and methodological perspective in education and neuroscience in a coherent way. We present a multi-level framework for educational neuroscience, which argues for integration of multiple levels of analysis, some originating in brain and cognitive sciences, others in education, as a roadmap for the future of educational neuroscience with concrete examples in moral education
Recommended from our members
Interactive Prediction and Planning for Autonomous Driving: from Algorithms to Fundamental Aspects
Inevitably, autonomous vehicles need to interact with other road participants in a variety of highly complex or critical driving scenarios. It is still an extremely challenging task even for the forefront companies or institutes to enable autonomous vehicles to interactively predict the behavior of others, and plan safe and high-quality motions accordingly. The major obstacles are not just originated from prediction and planning algorithms with insufficient performances. Several fundamental problems in the fields of interactive prediction and planning still remain open, such as formulation, representation and evaluation of interactive prediction methods, motion dataset with densely interactive driving behavior, as well as interface of interactive prediction and planning algorithms. The aforementioned fundamental aspects of interactive prediction and planning are addressed in this dissertation along with various kinds of algorithms. First, generic environmental representation for various scenarios with topological decomposition is constructed, and a corresponding planning algorithm is designed by combining graph search and optimization. Hard constraints in optimization-based planners are also incorporated into the training loss of imitation learning so that the policy net can generate safe and feasible motions in highly constrained scenarios. Unified problem formulation and motion representation are designed for different paradigms of interactive predictors such as planning-based prediction (inverse reinforcement learning), as well as probabilistic graphical models (hidden Markov model) and deep neural networks (mixture density network), which are utilized for the prediction/planning interface design and prediction benchmark. A framework combing decision network and graph-search/optimization/sample-based planner is proposed to achieve a driving strategy which is defensive to potential violations of others, but not overly conservatively to threats of low probabilities. Such driving strategy is achieved via experiments based on the aforementioned interactive prediction and planning algorithms with proper interface designed. These predictors are also evaluated from closed loop perspective considering planning fatality when using the prediction results instead of pure data approximation metrics. Finally, INTERACTION (INTERnational, Adversarial and Cooperative moTION) dataset with highly interactive driving scenarios and behavior from international locations is constructed with interaction density metric defined to compare different datasets. The dataset has been utilized for various behavior-related research areas such as prediction, planning, imitation learning and behavior modeling, and is inspiring new research fields such as representation learning, interaction extraction and scenario generation
- …