12 research outputs found
On Sampling Strategies for Neural Network-based Collaborative Filtering
Recent advances in neural networks have inspired people to design hybrid
recommendation algorithms that can incorporate both (1) user-item interaction
information and (2) content information including image, audio, and text.
Despite their promising results, neural network-based recommendation algorithms
pose extensive computational costs, making it challenging to scale and improve
upon. In this paper, we propose a general neural network-based recommendation
framework, which subsumes several existing state-of-the-art recommendation
algorithms, and address the efficiency issue by investigating sampling
strategies in the stochastic gradient descent training for the framework. We
tackle this issue by first establishing a connection between the loss functions
and the user-item interaction bipartite graph, where the loss function terms
are defined on links while major computation burdens are located at nodes. We
call this type of loss functions "graph-based" loss functions, for which varied
mini-batch sampling strategies can have different computational costs. Based on
the insight, three novel sampling strategies are proposed, which can
significantly improve the training efficiency of the proposed framework (up to
times speedup in our experiments), as well as improving the
recommendation performance. Theoretical analysis is also provided for both the
computational cost and the convergence. We believe the study of sampling
strategies have further implications on general graph-based loss functions, and
would also enable more research under the neural network-based recommendation
framework.Comment: This is a longer version (with supplementary attached) of the KDD'17
pape
Targeted collapse regularized autoencoder for anomaly detection: black hole at the center
Autoencoders have been extensively used in the development of recent anomaly
detection techniques. The premise of their application is based on the notion
that after training the autoencoder on normal training data, anomalous inputs
will exhibit a significant reconstruction error. Consequently, this enables a
clear differentiation between normal and anomalous samples. In practice,
however, it is observed that autoencoders can generalize beyond the normal
class and achieve a small reconstruction error on some of the anomalous
samples. To improve the performance, various techniques propose additional
components and more sophisticated training procedures. In this work, we propose
a remarkably straightforward alternative: instead of adding neural network
components, involved computations, and cumbersome training, we complement the
reconstruction loss with a computationally light term that regulates the norm
of representations in the latent space. The simplicity of our approach
minimizes the requirement for hyperparameter tuning and customization for new
applications which, paired with its permissive data modality constraint,
enhances the potential for successful adoption across a broad range of
applications. We test the method on various visual and tabular benchmarks and
demonstrate that the technique matches and frequently outperforms alternatives.
We also provide a theoretical analysis and numerical simulations that help
demonstrate the underlying process that unfolds during training and how it can
help with anomaly detection. This mitigates the black-box nature of
autoencoder-based anomaly detection algorithms and offers an avenue for further
investigation of advantages, fail cases, and potential new directions.Comment: 16 pages, 4 figures, 4 table