SUPERT: Towards New Frontiers in Unsupervised Evaluation Metrics for Multi-Document Summarization

We study unsupervised multi-document summarization evaluation metrics, which require neither human-written reference summaries nor human annotations (e.g. preferences, ratings, etc.). We propose SUPERT, which rates the quality of a summary by measuring its semantic similarity with a pseudo reference summary, i.e. selected salient sentences from the source documents, using contextualized embeddings and soft token alignment techniques. Compared to the state-of-the-art unsupervised evaluation metrics, SUPERT correlates better with human ratings by 18-39%. Furthermore, we use SUPERT as rewards to guide a neural-based reinforcement learning summarizer, yielding favorable performance compared to the state-of-the-art unsupervised summarizers. All source code is available at https://github.com/yg211/acl20-ref-free-eval.Comment: ACL 202

Occlusion Aware Unsupervised Learning of Optical Flow

It has been recently shown that a convolutional neural network can learn optical flow estimation with unsupervised learning. However, the performance of the unsupervised methods still has a relatively large gap compared to its supervised counterpart. Occlusion and large motion are some of the major factors that limit the current unsupervised learning of optical flow methods. In this work we introduce a new method which models occlusion explicitly and a new warping way that facilitates the learning of large motion. Our method shows promising results on Flying Chairs, MPI-Sintel and KITTI benchmark datasets. Especially on KITTI dataset where abundant unlabeled samples exist, our unsupervised method outperforms its counterpart trained with supervised learning.Comment: CVPR 2018 Camera-read

On Some Aspects of Model Selection Variability

In this thesis, we investigate the data analytic approach to integrate the model selection uncertainty into the statistical inferences of high dimensional estimators. Two closed-form formulae of covariance matrices are derived for high dimensional bagging estimators, one for the nonparametric bootstrapping and the other for the parametric bootstrapping. Two simulation studies are completed in detail for demonstrating the validity of the new formulae. Several model selection methods --- the hypothesis testing, the Mallows' $C_p$, AIC, BIC and LASSO --- are compared in terms of the effects on the accuracy of bagging estimators in the context of multivariate linear regression. The confidence region and its coverage probability are also estimated for the bagging estimators with those model selection methods