8,197 research outputs found
Composite Optimization Algorithms for Sigmoid Networks
In this paper, we use composite optimization algorithms to solve sigmoid
networks. We equivalently transfer the sigmoid networks to a convex composite
optimization and propose the composite optimization algorithms based on the
linearized proximal algorithms and the alternating direction method of
multipliers. Under the assumptions of the weak sharp minima and the regularity
condition, the algorithm is guaranteed to converge to a globally optimal
solution of the objective function even in the case of non-convex and
non-smooth problems. Furthermore, the convergence results can be directly
related to the amount of training data and provide a general guide for setting
the size of sigmoid networks. Numerical experiments on Franke's function
fitting and handwritten digit recognition show that the proposed algorithms
perform satisfactorily and robustly
- …