9,259 research outputs found
Multiplicative Nonholonomic/Newton -like Algorithm
We construct new algorithms from scratch, which use the fourth order cumulant
of stochastic variables for the cost function. The multiplicative updating rule
here constructed is natural from the homogeneous nature of the Lie group and
has numerous merits for the rigorous treatment of the dynamics. As one
consequence, the second order convergence is shown. For the cost function,
functions invariant under the componentwise scaling are choosen. By identifying
points which can be transformed to each other by the scaling, we assume that
the dynamics is in a coset space. In our method, a point can move toward any
direction in this coset. Thus, no prewhitening is required.Comment: 12 page
Computing Optimal Experimental Designs via Interior Point Method
In this paper, we study optimal experimental design problems with a broad
class of smooth convex optimality criteria, including the classical A-, D- and
p th mean criterion. In particular, we propose an interior point (IP) method
for them and establish its global convergence. Furthermore, by exploiting the
structure of the Hessian matrix of the aforementioned optimality criteria, we
derive an explicit formula for computing its rank. Using this result, we then
show that the Newton direction arising in the IP method can be computed
efficiently via Sherman-Morrison-Woodbury formula when the size of the moment
matrix is small relative to the sample size. Finally, we compare our IP method
with the widely used multiplicative algorithm introduced by Silvey et al. [29].
The computational results show that the IP method generally outperforms the
multiplicative algorithm both in speed and solution quality
Non-negative mixtures
This is the author's accepted pre-print of the article, first published as M. D. Plumbley, A. Cichocki and R. Bro. Non-negative mixtures. In P. Comon and C. Jutten (Ed), Handbook of Blind Source Separation: Independent Component Analysis and Applications. Chapter 13, pp. 515-547. Academic Press, Feb 2010. ISBN 978-0-12-374726-6 DOI: 10.1016/B978-0-12-374726-6.00018-7file: Proof:p\PlumbleyCichockiBro10-non-negative.pdf:PDF owner: markp timestamp: 2011.04.26file: Proof:p\PlumbleyCichockiBro10-non-negative.pdf:PDF owner: markp timestamp: 2011.04.2
The Diagonalized Newton Algorithm for Nonnegative Matrix Factorization
Non-negative matrix factorization (NMF) has become a popular machine learning
approach to many problems in text mining, speech and image processing,
bio-informatics and seismic data analysis to name a few. In NMF, a matrix of
non-negative data is approximated by the low-rank product of two matrices with
non-negative entries. In this paper, the approximation quality is measured by
the Kullback-Leibler divergence between the data and its low-rank
reconstruction. The existence of the simple multiplicative update (MU)
algorithm for computing the matrix factors has contributed to the success of
NMF. Despite the availability of algorithms showing faster convergence, MU
remains popular due to its simplicity. In this paper, a diagonalized Newton
algorithm (DNA) is proposed showing faster convergence while the implementation
remains simple and suitable for high-rank problems. The DNA algorithm is
applied to various publicly available data sets, showing a substantial speed-up
on modern hardware.Comment: 8 pages + references; International Conference on Learning
Representations, 201
- …