A Connection Between GRBF and MLP

Both multilayer perceptrons (MLP) and Generalized Radial Basis Functions (GRBF) have good approximation properties, theoretically and experimentally. Are they related? The main point of this paper is to show that for normalized inputs, multilayer perceptron networks are radial function networks (albeit with a non-standard radial function). This provides an interpretation of the weights w as centers t of the radial function network, and therefore as equivalent to templates. This insight may be useful for practical applications, including better initialization procedures for MLP. In the remainder of the paper, we discuss the relation between the radial functions that correspond to the sigmoid for normalized inputs and well-behaved radial basis functions, such as the Gaussian. In particular, we observe that the radial function associated with the sigmoid is an activation function that is good approximation to Gaussian basis functions for a range of values of the bias parameter. The implication is that a MLP network can always simulate a Gaussian GRBF network (with the same number of units but less parameters); the converse is true only for certain values of the bias parameter. Numerical experiments indicate that this constraint is not always satisfied in practice by MLP networks trained with backpropagation. Multiscale GRBF networks, on the other hand, can approximate MLP networks with a similar number of parameters

Relative Resolution: A Multipole Approximation at Appropriate Distances

Recently, we introduced Relative Resolution as a hybrid formalism for fluid mixtures [1]. The essence of this approach is that it switches molecular resolution in terms or relative separation: While nearest neighbors are characterized by a detailed fine-grained model, other neighbors are characterized by a simplified coarse-grained model. Once the two models are analytically connected with each other via energy conservation, Relative Resolution can capture the structural and thermal behavior of (nonpolar) multi-component and multi-phase systems across state space. The current work is a natural continuation of our original communication [1]. Most importantly, we present the comprehensive mathematics of Relative Resolution, basically casting it as a multipole approximation at appropriate distances; the current set of equations importantly applies for all systems (e.g, polar molecules). Besides, we continue examining the capability of our multiscale approach in molecular simulations, importantly showing that we can successfully retrieve not just the statics but also the dynamics of liquid systems. We finally conclude by discussing how Relative Resolution is the inherent variant of the famous "cell-multipole" approach for molecular simulations