761,223 research outputs found
Separable mechanisms underlying global feature-based attention
Feature-based attention is known to operate in a spatially global manner, in that the selection of attended features is not bound to the spatial focus of attention. Here we used electromagnetic recordings in human observers to characterize the spatiotemporal signature of such global selection of an orientation feature. Observers performed a simple orientation-discrimination task while ignoring task-irrelevant orientation probes outside the focus of attention. We observed that global feature-based selection, indexed by the brain response to unattended orientation probes, is composed of separable functional components. One such component reflects global selection based on the similarity of the probe with task-relevant orientation values ("template matching"), which is followed by a component reflecting selection based on the similarity of the probe with the orientation value under discrimination in the focus of attention ("discrimination matching"). Importantly, template matching occurs at similar to 150 ms after stimulus onset, similar to 80 ms before the onset of discrimination matching. Moreover, source activity underlying template matching and discrimination matching was found to originate from ventral extrastriate cortex, with the former being generated in more anterolateral and the latter in more posteromedial parts, suggesting template matching to occur in visual cortex higher up in the visual processing hierarchy than discrimination matching. We take these observations to indicate that the population-level signature of global feature-based selection reflects a sequence of hierarchically ordered operations in extrastriate visual cortex, in which the selection based on task relevance has temporal priority over the selection based on the sensory similarity between input representations
BattRAE: Bidimensional Attention-Based Recursive Autoencoders for Learning Bilingual Phrase Embeddings
In this paper, we propose a bidimensional attention based recursive
autoencoder (BattRAE) to integrate clues and sourcetarget interactions at
multiple levels of granularity into bilingual phrase representations. We employ
recursive autoencoders to generate tree structures of phrases with embeddings
at different levels of granularity (e.g., words, sub-phrases and phrases). Over
these embeddings on the source and target side, we introduce a bidimensional
attention network to learn their interactions encoded in a bidimensional
attention matrix, from which we extract two soft attention weight distributions
simultaneously. These weight distributions enable BattRAE to generate
compositive phrase representations via convolution. Based on the learned phrase
representations, we further use a bilinear neural model, trained via a
max-margin method, to measure bilingual semantic similarity. To evaluate the
effectiveness of BattRAE, we incorporate this semantic similarity as an
additional feature into a state-of-the-art SMT system. Extensive experiments on
NIST Chinese-English test sets show that our model achieves a substantial
improvement of up to 1.63 BLEU points on average over the baseline.Comment: 7 pages, accepted by AAAI 201
NAIS: Neural Attentive Item Similarity Model for Recommendation
Item-to-item collaborative filtering (aka. item-based CF) has been long used
for building recommender systems in industrial settings, owing to its
interpretability and efficiency in real-time personalization. It builds a
user's profile as her historically interacted items, recommending new items
that are similar to the user's profile. As such, the key to an item-based CF
method is in the estimation of item similarities. Early approaches use
statistical measures such as cosine similarity and Pearson coefficient to
estimate item similarities, which are less accurate since they lack tailored
optimization for the recommendation task. In recent years, several works
attempt to learn item similarities from data, by expressing the similarity as
an underlying model and estimating model parameters by optimizing a
recommendation-aware objective function. While extensive efforts have been made
to use shallow linear models for learning item similarities, there has been
relatively less work exploring nonlinear neural network models for item-based
CF.
In this work, we propose a neural network model named Neural Attentive Item
Similarity model (NAIS) for item-based CF. The key to our design of NAIS is an
attention network, which is capable of distinguishing which historical items in
a user profile are more important for a prediction. Compared to the
state-of-the-art item-based CF method Factored Item Similarity Model (FISM),
our NAIS has stronger representation power with only a few additional
parameters brought by the attention network. Extensive experiments on two
public benchmarks demonstrate the effectiveness of NAIS. This work is the first
attempt that designs neural network models for item-based CF, opening up new
research possibilities for future developments of neural recommender systems
Temporal similarity metrics for latent network reconstruction: The role of time-lag decay
When investigating the spreading of a piece of information or the diffusion
of an innovation, we often lack information on the underlying propagation
network. Reconstructing the hidden propagation paths based on the observed
diffusion process is a challenging problem which has recently attracted
attention from diverse research fields. To address this reconstruction problem,
based on static similarity metrics commonly used in the link prediction
literature, we introduce new node-node temporal similarity metrics. The new
metrics take as input the time-series of multiple independent spreading
processes, based on the hypothesis that two nodes are more likely to be
connected if they were often infected at similar points in time. This
hypothesis is implemented by introducing a time-lag function which penalizes
distant infection times. We find that the choice of this time-lag strongly
affects the metrics' reconstruction accuracy, depending on the network's
clustering coefficient and we provide an extensive comparative analysis of
static and temporal similarity metrics for network reconstruction. Our findings
shed new light on the notion of similarity between pairs of nodes in complex
networks
- …