12 research outputs found
A Survey of Matrix Completion Methods for Recommendation Systems
In recent years, the recommendation systems have become increasingly popular and have been used in a broad variety of applications. Here, we investigate the matrix completion techniques for the recommendation systems that are based on collaborative filtering. The collaborative filtering problem can be viewed as predicting the favorability of a user with respect to new items of commodities. When a rating matrix is constructed with users as rows, items as columns, and entries as ratings, the collaborative filtering problem can then be modeled as a matrix completion problem by filling out the unknown elements in the rating matrix. This article presents a comprehensive survey of the matrix completion methods used in recommendation systems. We focus on the mathematical models for matrix completion and the corresponding computational algorithms as well as their characteristics and potential issues. Several applications other than the traditional user-item association prediction are also discussed
Latent State Models of Training Dynamics
The impact of randomness on model training is poorly understood. How do
differences in data order and initialization actually manifest in the model,
such that some training runs outperform others or converge faster? Furthermore,
how can we interpret the resulting training dynamics and the phase transitions
that characterize different trajectories? To understand the effect of
randomness on the dynamics and outcomes of neural network training, we train
models multiple times with different random seeds and compute a variety of
metrics throughout training, such as the norm, mean, and variance of the
neural network's weights. We then fit a hidden Markov model (HMM) over the
resulting sequences of metrics. The HMM represents training as a stochastic
process of transitions between latent states, providing an intuitive overview
of significant changes during training. Using our method, we produce a
low-dimensional, discrete representation of training dynamics on grokking
tasks, image classification, and masked language modeling. We use the HMM
representation to study phase transitions and identify latent "detour" states
that slow down convergence.Comment: Accepted at TMLR 2023. Updated Jan 19, 2024 with erratu