With this paper we aim to present a new class of learning models for linear as well as non-linear neural layers, deriving from the study of the dynamics of an abstract rigid mechanical system. The set of equations describing the motion of this system may be readily interpreted as a learning rule for orthogonal networks. As a simple example of how to use the new learning theory, a case of Orthonormal Independent Component Analysis based on the Bell-Sejnowski's InfoMax principle is discussed through simulations