1 research outputs found
Resonant Machine Learning Based on Complex Growth Transform Dynamical Systems
Traditional energy-based learning models associate a single energy metric to
each configuration of variables involved in the underlying optimization
process. Such models associate the lowest energy state to the optimal
configuration of variables under consideration, and are thus inherently
dissipative. In this paper we propose an energy-efficient learning framework
that exploits structural and functional similarities between a machine learning
network and a general electrical network satisfying the Tellegen's theorem. In
contrast to the standard energy-based models, the proposed formulation
associates two energy components, namely, active and reactive energy to the
network. This ensures that the network's active-power is dissipated only during
the process of learning, whereas the reactive-power is maintained to be zero at
all times. As a result, in steady-state, the learned parameters are stored and
self-sustained by electrical resonance determined by the network's nodal
inductances and capacitances. Based on this approach, this paper introduces
three novel concepts: (a) A learning framework where the network's active-power
dissipation is used as a regularization for a learning objective function that
is subjected to zero total reactive-power constraint; (b) A dynamical system
based on complex-domain, continuous-time growth transforms which optimizes the
learning objective function and drives the network towards electrical resonance
under steady-state operation; and (c) An annealing procedure that controls the
trade-off between active-power dissipation and the speed of convergence. As a
representative example, we show how the proposed framework can be used for
designing resonant support vector machines (SVMs), where we show that the
support-vectors correspond to an LC network with self-sustained oscillations.Comment: Version3, accepted in IEEE TNNLS, March 202