73 research outputs found
Recommended from our members
Generation of Transfer Functions with Stochastic Search Techniques
This paper presents a novel approach to assist the
user in exploring appropriate transfer functions for the
visualization of volumetric datasets. The search for a
transfer function is treated as a parameter optimization problem and addressed with stochastic search techniques. Starting from an initial population of (random or pre-defined) transfer functions, the evolution of the stochastic algorithms is controlled by either direct user selection of intermediate images or automatic fitness evaluation using user-specified objective functions. This approach essentially shields the user from the complex and tedious "trial and error" approach, and demonstrates effective and convenient generation of transfer functions.Engineering and Applied Science
Improving Multi-Task Generalization via Regularizing Spurious Correlation
Multi-Task Learning (MTL) is a powerful learning paradigm to improve
generalization performance via knowledge sharing. However, existing studies
find that MTL could sometimes hurt generalization, especially when two tasks
are less correlated. One possible reason that hurts generalization is spurious
correlation, i.e., some knowledge is spurious and not causally related to task
labels, but the model could mistakenly utilize them and thus fail when such
correlation changes. In MTL setup, there exist several unique challenges of
spurious correlation. First, the risk of having non-causal knowledge is higher,
as the shared MTL model needs to encode all knowledge from different tasks, and
causal knowledge for one task could be potentially spurious to the other.
Second, the confounder between task labels brings in a different type of
spurious correlation to MTL. We theoretically prove that MTL is more prone to
taking non-causal knowledge from other tasks than single-task learning, and
thus generalize worse. To solve this problem, we propose Multi-Task Causal
Representation Learning framework, aiming to represent multi-task knowledge via
disentangled neural modules, and learn which module is causally related to each
task via MTL-specific invariant regularization. Experiments show that it could
enhance MTL model's performance by 5.5% on average over Multi-MNIST, MovieLens,
Taskonomy, CityScape, and NYUv2, via alleviating spurious correlation problem.Comment: Published on NeurIPS 202
Hiformer: Heterogeneous Feature Interactions Learning with Transformers for Recommender Systems
Learning feature interaction is the critical backbone to building recommender
systems. In web-scale applications, learning feature interaction is extremely
challenging due to the sparse and large input feature space; meanwhile,
manually crafting effective feature interactions is infeasible because of the
exponential solution space. We propose to leverage a Transformer-based
architecture with attention layers to automatically capture feature
interactions. Transformer architectures have witnessed great success in many
domains, such as natural language processing and computer vision. However,
there has not been much adoption of Transformer architecture for feature
interaction modeling in industry. We aim at closing the gap. We identify two
key challenges for applying the vanilla Transformer architecture to web-scale
recommender systems: (1) Transformer architecture fails to capture the
heterogeneous feature interactions in the self-attention layer; (2) The serving
latency of Transformer architecture might be too high to be deployed in
web-scale recommender systems. We first propose a heterogeneous self-attention
layer, which is a simple yet effective modification to the self-attention layer
in Transformer, to take into account the heterogeneity of feature interactions.
We then introduce \textsc{Hiformer} (\textbf{H}eterogeneous
\textbf{I}nteraction Trans\textbf{former}) to further improve the model
expressiveness. With low-rank approximation and model pruning, \hiformer enjoys
fast inference for online deployment. Extensive offline experiment results
corroborates the effectiveness and efficiency of the \textsc{Hiformer} model.
We have successfully deployed the \textsc{Hiformer} model to a real world large
scale App ranking model at Google Play, with significant improvement in key
engagement metrics (up to +2.66\%)
- …