Analysis of Natural Gradient Descent for Multilayer Neural Networks

Natural gradient descent is a principled method for adapting the parameters of a statistical model on-line using an underlying Riemannian parameter space to redefine the direction of steepest descent. The algorithm is examined via methods of statistical physics which accurately characterize both transient and asymptotic behavior. A solution of the learning dynamics is obtained for the case of multilayer neural network training in the limit of large input dimension. We find that natural gradient learning leads to optimal asymptotic performance and outperforms gradient descent in the transient, significantly shortening or even removing plateaus in the transient generalization performance which typically hamper gradient descent training.Comment: 14 pages including figures. To appear in Physical Review

The Competition for Shortest Paths on Sparse Graphs

Optimal paths connecting randomly selected network nodes and fixed routers are studied analytically in the presence of non-linear overlap cost that penalizes congestion. Routing becomes increasingly more difficult as the number of selected nodes increases and exhibits ergodicity breaking in the case of multiple routers. A distributed linearly-scalable routing algorithm is devised. The ground state of such systems reveals non-monotonic complex behaviors in both average path-length and algorithmic convergence, depending on the network topology, and densities of communicating nodes and routers.Comment: 4 pages, 4 figure

Sustainable Competitive Advantage Through Servitization: An Investigation Into Servitization Strategy In the Real Estate Development Sector

Achieving sustainable competitive advantage, based upon services provision, is often claimed to be viable for businesses. There has, however, been little evidence captured on the application of aspects of servitization within the real estate development. By applying the RVB theory, the research propositions of this study include understanding how superior performance is driven from organization capabilities and how different organizations develop and position those capabilities to gain competitive advantage. This research was conducted using an exploratory research and in-depth case study. The study suggests that more strategic alignment between servitization decisions and operations management is required to create competitive advantage

Statistical Mechanics of Low-Density Parity Check Error-Correcting Codes over Galois Fields

A variation of low density parity check (LDPC) error correcting codes defined over Galois fields ($GF(q)$) is investigated using statistical physics. A code of this type is characterised by a sparse random parity check matrix composed of $C$ nonzero elements per column. We examine the dependence of the code performance on the value of $q$, for finite and infinite $C$ values, both in terms of the thermodynamical transition point and the practical decoding phase characterised by the existence of a unique (ferromagnetic) solution. We find different $q$-dependencies in the cases of C=2 and $C \ge 3$; the analytical solutions are in agreement with simulation results, providing a quantitative measure to the improvement in performance obtained using non-binary alphabets.Comment: 7 pages, 1 figur

Public key cryptography and error correcting codes as Ising models

We employ the methods of statistical physics to study the performance of Gallager type error-correcting codes. In this approach, the transmitted codeword comprises Boolean sums of the original message bits selected by two randomly-constructed sparse matrices. We show that a broad range of these codes potentially saturate Shannon's bound but are limited due to the decoding dynamics used. Other codes show sub-optimal performance but are not restricted by the decoding dynamics. We show how these codes may also be employed as a practical public-key cryptosystem and are of competitive performance to modern cyptographical methods.Comment: 6 page