825 research outputs found
Inertial Stochastic PALM (iSPALM) and Applications in Machine Learning
Inertial algorithms for minimizing nonsmooth and nonconvex functions as the
inertial proximal alternating linearized minimization algorithm (iPALM) have
demonstrated their superiority with respect to computation time over their non
inertial variants. In many problems in imaging and machine learning, the
objective functions have a special form involving huge data which encourage the
application of stochastic algorithms. While algorithms based on stochastic
gradient descent are still used in the majority of applications, recently also
stochastic algorithms for minimizing nonsmooth and nonconvex functions were
proposed. In this paper, we derive an inertial variant of a stochastic PALM
algorithm with variance-reduced gradient estimator, called iSPALM, and prove
linear convergence of the algorithm under certain assumptions. Our inertial
approach can be seen as generalization of momentum methods widely used to speed
up and stabilize optimization algorithms, in particular in machine learning, to
nonsmooth problems. Numerical experiments for learning the weights of a
so-called proximal neural network and the parameters of Student-t mixture
models show that our new algorithm outperforms both stochastic PALM and its
deterministic counterparts
Provably Good Solutions to the Knapsack Problem via Neural Networks of Bounded Size
The development of a satisfying and rigorous mathematical understanding of
the performance of neural networks is a major challenge in artificial
intelligence. Against this background, we study the expressive power of neural
networks through the example of the classical NP-hard Knapsack Problem. Our
main contribution is a class of recurrent neural networks (RNNs) with rectified
linear units that are iteratively applied to each item of a Knapsack instance
and thereby compute optimal or provably good solution values. We show that an
RNN of depth four and width depending quadratically on the profit of an optimum
Knapsack solution is sufficient to find optimum Knapsack solutions. We also
prove the following tradeoff between the size of an RNN and the quality of the
computed Knapsack solution: for Knapsack instances consisting of items, an
RNN of depth five and width computes a solution of value at least
times the optimum solution value. Our results
build upon a classical dynamic programming formulation of the Knapsack Problem
as well as a careful rounding of profit values that are also at the core of the
well-known fully polynomial-time approximation scheme for the Knapsack Problem.
A carefully conducted computational study qualitatively supports our
theoretical size bounds. Finally, we point out that our results can be
generalized to many other combinatorial optimization problems that admit
dynamic programming solution methods, such as various Shortest Path Problems,
the Longest Common Subsequence Problem, and the Traveling Salesperson Problem.Comment: A short version of this paper appears in the proceedings of AAAI 202
Conformally Flat Circle Bundles over Surfaces
We classify conformally flat Riemannian manifolds which possesses a free
isometric action.Comment: 12 pages, Part of the author's PhD thesi
Harmonic maps in unfashionable geometries
We describe some general constructions on a real smooth projective 4-quadric
which provide analogues of the Willmore functional and conformal Gauss map in
both Lie sphere and projective differential geometry. Extrema of these
functionals are characterized by harmonicity of this Gauss map.Comment: plain TeX, uses bbmsl for blackboard bold, 20 page
A discrete version of the Darboux transform for isothermic surfaces
We study Christoffel and Darboux transforms of discrete isothermic nets in
4-dimensional Euclidean space: definitions and basic properties are derived.
Analogies with the smooth case are discussed and a definition for discrete
Ribaucour congruences is given. Surfaces of constant mean curvature are special
among all isothermic surfaces: they can be characterized by the fact that their
parallel constant mean curvature surfaces are Christoffel and Darboux
transforms at the same time. This characterization is used to define discrete
nets of constant mean curvature. Basic properties of discrete nets of constant
mean curvature are derived.Comment: 30 pages, LaTeX, a version with high quality figures is available at
http://www-sfb288.math.tu-berlin.de/preprints.htm
Discrete constant mean curvature nets in space forms: Steiner's formula and Christoffel duality
We show that the discrete principal nets in quadrics of constant curvature
that have constant mixed area mean curvature can be characterized by the
existence of a K\"onigs dual in a concentric quadric.Comment: 12 pages, 10 figures, pdfLaTeX (plain pdfTeX source included as bak
file
- β¦