45,913 research outputs found
ChebNet: Efficient and Stable Constructions of Deep Neural Networks with Rectified Power Units using Chebyshev Approximations
In a recent paper [B. Li, S. Tang and H. Yu, arXiv:1903.05858], it was shown
that deep neural networks built with rectified power units (RePU) can give
better approximation for sufficient smooth functions than those with rectified
linear units, by converting polynomial approximation given in power series into
deep neural networks with optimal complexity and no approximation error.
However, in practice, power series are not easy to compute. In this paper, we
propose a new and more stable way to construct deep RePU neural networks based
on Chebyshev polynomial approximations. By using a hierarchical structure of
Chebyshev polynomial approximation in frequency domain, we build efficient and
stable deep neural network constructions. In theory, ChebNets and the deep RePU
nets based on Power series have the same upper error bounds for general
function approximations. But numerically, ChebNets are much more stable.
Numerical results show that the constructed ChebNets can be further trained and
obtain much better results than those obtained by training deep RePU nets
constructed basing on power series.Comment: 18 pages, 6 figures, 2 table
Learning how to be robust: Deep polynomial regression
Polynomial regression is a recurrent problem with a large number of
applications. In computer vision it often appears in motion analysis. Whatever
the application, standard methods for regression of polynomial models tend to
deliver biased results when the input data is heavily contaminated by outliers.
Moreover, the problem is even harder when outliers have strong structure.
Departing from problem-tailored heuristics for robust estimation of parametric
models, we explore deep convolutional neural networks. Our work aims to find a
generic approach for training deep regression models without the explicit need
of supervised annotation. We bypass the need for a tailored loss function on
the regression parameters by attaching to our model a differentiable hard-wired
decoder corresponding to the polynomial operation at hand. We demonstrate the
value of our findings by comparing with standard robust regression methods.
Furthermore, we demonstrate how to use such models for a real computer vision
problem, i.e., video stabilization. The qualitative and quantitative
experiments show that neural networks are able to learn robustness for general
polynomial regression, with results that well overpass scores of traditional
robust estimation methods.Comment: 18 pages, conferenc
- …