26 research outputs found
Factor Analysis and Alternating Minimization
In this paper we make a first attempt at understanding how to build an
optimal approximate normal factor analysis model. The criterion we have chosen
to evaluate the distance between different models is the I-divergence between
the corresponding normal laws. The algorithm that we propose for the
construction of the best approximation is of an the alternating minimization
kind
Approximate Nonnegative Matrix Factorization via Alternating Minimization
In this paper we consider the Nonnegative Matrix Factorization (NMF) problem:
given an (elementwise) nonnegative matrix find, for
assigned , nonnegative matrices and
such that . Exact, non trivial, nonnegative
factorizations do not always exist, hence it is interesting to pose the
approximate NMF problem. The criterion which is commonly employed is
I-divergence between nonnegative matrices. The problem becomes that of finding,
for assigned , the factorization closest to in I-divergence. An
iterative algorithm, EM like, for the construction of the best pair
has been proposed in the literature. In this paper we interpret the algorithm
as an alternating minimization procedure \`a la Csisz\'ar-Tusn\'ady and
investigate some of its stability properties. NMF is widespreading as a data
analysis method in applications for which the positivity constraint is
relevant. There are other data analysis methods which impose some form of
nonnegativity: we discuss here the connections between NMF and Archetypal
Analysis. An interesting system theoretic application of NMF is to the problem
of approximate realization of Hidden Markov Models
Consistent Estimation of the Order for Markov and Hidden Markov Chains
The structural parameters of many statistical models can be estimated maximizing a penalized version of the likelihood function. We use this idea to construct strongly consistent estimators of the order for Markov Chains and Hidden Markov Chain models. The specification of the penalty term requires precise information on the rate of growth of the maximized likelihood ratio. For Markov chain models we determine the rate using the Law of the Iterated Logarithm. For Hidden Markov chain models we find an upper bound to the rate using results from Information Theory. We give sufficient conditions on the penalty term to avoid overestimation and underestimation of the order. Examples of penalty terms that generate strongly consitent estimators are also given
Order Determination for Probabilistic Functions of Finite Markov Chains
Let {Y sub t} be a stationary stochastic process with values in the finite set YY. We model {Y sub t} as a probabilistic function of a finite state Markov Chain {X sub t} i.e. X sub t is such that: P[Y sub t | X sup t, Y sup t-1] = P[Y sub t | X sub t] Define the cardinality of the state space of {X sub t} as the order of the model. The problem is to determine the order given the observations {y sub 1, y sub 2, y sub T}. We show that under mild conditions on the probability distribution function P sub Y (.) of {Y sub t} the order is identifiable and can be consistently determined from the data
Approximation of nonnegative systems by moving averages of fixed order
Contains fulltext :
206990.pdf (preprint version ) (Open Access