239,111 research outputs found
On Sampling Strategies for Neural Network-based Collaborative Filtering
Recent advances in neural networks have inspired people to design hybrid
recommendation algorithms that can incorporate both (1) user-item interaction
information and (2) content information including image, audio, and text.
Despite their promising results, neural network-based recommendation algorithms
pose extensive computational costs, making it challenging to scale and improve
upon. In this paper, we propose a general neural network-based recommendation
framework, which subsumes several existing state-of-the-art recommendation
algorithms, and address the efficiency issue by investigating sampling
strategies in the stochastic gradient descent training for the framework. We
tackle this issue by first establishing a connection between the loss functions
and the user-item interaction bipartite graph, where the loss function terms
are defined on links while major computation burdens are located at nodes. We
call this type of loss functions "graph-based" loss functions, for which varied
mini-batch sampling strategies can have different computational costs. Based on
the insight, three novel sampling strategies are proposed, which can
significantly improve the training efficiency of the proposed framework (up to
times speedup in our experiments), as well as improving the
recommendation performance. Theoretical analysis is also provided for both the
computational cost and the convergence. We believe the study of sampling
strategies have further implications on general graph-based loss functions, and
would also enable more research under the neural network-based recommendation
framework.Comment: This is a longer version (with supplementary attached) of the KDD'17
pape
Advocacy spurs innovation: promoting synergy between physical and biomedical sciences
Despite dramatic advances in decoding the genes, proteins, and pathways that drive cancer, the disease has evaded the reductionist approaches to defeat it. Recent work has highlighted cancer’s heterogeneity, complexity, and ability to develop resistance as major barriers to progress. To better understand and control the processes that govern the initiation, behavior, and progression of cancer, the National Cancer Institute (NCI) created the Physical Sciences-Oncology Center (PS-OC) Network in 2009. As a hub for scientific innovation and as an example of the transdisciplinary research model, the twelve centers within the PS-OC strive for the systematic convergence of the physical sciences with cancer biology. Promoting collaboration between biologists, physicists, mathematicians, chemists, biomedical engineers, and oncologists, the program offers a compelling vision of how new frontiers in physical sciences and oncology will permit the emergence of new scientific principles and opportunities, and of how the benefits of the current convergence revolution would be enhanced by vigorous public/advocacy support
A New PHO-rmula for Improved Performance of Semi-Structured Networks
Recent advances to combine structured regression models and deep neural
networks for better interpretability, more expressiveness, and statistically
valid uncertainty quantification demonstrate the versatility of semi-structured
neural networks (SSNs). We show that techniques to properly identify the
contributions of the different model components in SSNs, however, lead to
suboptimal network estimation, slower convergence, and degenerated or erroneous
predictions. In order to solve these problems while preserving favorable model
properties, we propose a non-invasive post-hoc orthogonalization (PHO) that
guarantees identifiability of model components and provides better estimation
and prediction quality. Our theoretical findings are supported by numerical
experiments, a benchmark comparison as well as a real-world application to
COVID-19 infections.Comment: ICML 202
Reward-Predictive Clustering
Recent advances in reinforcement-learning research have demonstrated
impressive results in building algorithms that can out-perform humans in
complex tasks. Nevertheless, creating reinforcement-learning systems that can
build abstractions of their experience to accelerate learning in new contexts
still remains an active area of research. Previous work showed that
reward-predictive state abstractions fulfill this goal, but have only be
applied to tabular settings. Here, we provide a clustering algorithm that
enables the application of such state abstractions to deep learning settings,
providing compressed representations of an agent's inputs that preserve the
ability to predict sequences of reward. A convergence theorem and simulations
show that the resulting reward-predictive deep network maximally compresses the
agent's inputs, significantly speeding up learning in high dimensional visual
control tasks. Furthermore, we present different generalization experiments and
analyze under which conditions a pre-trained reward-predictive representation
network can be re-used without re-training to accelerate learning -- a form of
systematic out-of-distribution transfer
Low Rank Optimization for Efficient Deep Learning: Making A Balance between Compact Architecture and Fast Training
Deep neural networks have achieved great success in many data processing
applications. However, the high computational complexity and storage cost makes
deep learning hard to be used on resource-constrained devices, and it is not
environmental-friendly with much power cost. In this paper, we focus on
low-rank optimization for efficient deep learning techniques. In the space
domain, deep neural networks are compressed by low rank approximation of the
network parameters, which directly reduces the storage requirement with a
smaller number of network parameters. In the time domain, the network
parameters can be trained in a few subspaces, which enables efficient training
for fast convergence. The model compression in the spatial domain is summarized
into three categories as pre-train, pre-set, and compression-aware methods,
respectively. With a series of integrable techniques discussed, such as sparse
pruning, quantization, and entropy coding, we can ensemble them in an
integration framework with lower computational complexity and storage. Besides
of summary of recent technical advances, we have two findings for motivating
future works: one is that the effective rank outperforms other sparse measures
for network compression. The other is a spatial and temporal balance for
tensorized neural networks
- …