1 research outputs found
A Dynamical Systems Approach for Convergence of the Bayesian EM Algorithm
Out of the recent advances in systems and control (S\&C)-based analysis of
optimization algorithms, not enough work has been specifically dedicated to
machine learning (ML) algorithms and its applications. This paper addresses
this gap by illustrating how (discrete-time) Lyapunov stability theory can
serve as a powerful tool to aid, or even lead, in the analysis (and potential
design) of optimization algorithms that are not necessarily gradient-based. The
particular ML problem that this paper focuses on is that of parameter
estimation in an incomplete-data Bayesian framework via the popular
optimization algorithm known as maximum a posteriori expectation-maximization
(MAP-EM). Following first principles from dynamical systems stability theory,
conditions for convergence of MAP-EM are developed. Furthermore, if additional
assumptions are met, we show that fast convergence (linear or quadratic) is
achieved, which could have been difficult to unveil without our adopted S\&C
approach. The convergence guarantees in this paper effectively expand the set
of sufficient conditions for EM applications, thereby demonstrating the
potential of similar S\&C-based convergence analysis of other ML algorithms