Functional Optimisation of Online Algorithms in Multilayer Neural Networks

We study the online dynamics of learning in fully connected soft committee machines in the student-teacher scenario. The locally optimal modulation function, which determines the learning algorithm, is obtained from a variational argument in such a manner as to maximise the average generalisation error decay per example. Simulations results for the resulting algorithm are presented for a few cases. The symmetric phase plateaux are found to be vastly reduced in comparison to those found when online backpropagation algorithms are used. A discussion of the implementation of these ideas as practical algorithms is given

Inflation, interest rates, and seasonality

Inflation (Finance) ; Interest rates ; Seasonal variations (Economics)

Phase transitions in soft-committee machines

Equilibrium statistical physics is applied to layered neural networks with differentiable activation functions. A first analysis of off-line learning in soft-committee machines with a finite number (K) of hidden units learning a perfectly matching rule is performed. Our results are exact in the limit of high training temperatures. For K=2 we find a second order phase transition from unspecialized to specialized student configurations at a critical size P of the training set, whereas for K > 2 the transition is first order. Monte Carlo simulations indicate that our results are also valid for moderately low temperatures qualitatively. The limit K to infinity can be performed analytically, the transition occurs after presenting on the order of N K examples. However, an unspecialized metastable state persists up to P= O (N K^2).Comment: 8 pages, 4 figure

Modeling one-dimensional island growth with mass-dependent detachment rates

We study one-dimensional models of particle diffusion and attachment/detachment from islands where the detachment rates gamma(m) of particles at the cluster edges increase with cluster mass m. They are expected to mimic the effects of lattice mismatch with the substrate and/or long-range repulsive interactions that work against the formation of long islands. Short-range attraction is represented by an overall factor epsilon<<1 in the detachment rates relatively to isolated particle hopping rates [epsilon ~ exp(-E/T), with binding energy E and temperature T]. We consider various gamma(m), from rapidly increasing forms such as gamma(m) ~ m to slowly increasing ones, such as gamma(m) ~ [m/(m+1)]^b. A mapping onto a column problem shows that these systems are zero-range processes, whose steady states properties are exactly calculated under the assumption of independent column heights in the Master equation. Simulation provides island size distributions which confirm analytic reductions and are useful whenever the analytical tools cannot provide results in closed form. The shape of island size distributions can be changed from monomodal to monotonically decreasing by tuning the temperature or changing the particle density rho. Small values of the scaling variable X=epsilon^{-1}rho/(1-rho) favour the monotonically decreasing ones. However, for large X, rapidly increasing gamma(m) lead to distributions with peaks very close to and rapidly decreasing tails, while slowly increasing gamma(m) provide peaks close to /2$ and fat right tails.Comment: 16 pages, 6 figure

'I'm as much an anarchist in theory as I am in practice': Fernando Pessoa's 'Anarchist banker' in a management education context

The performance of Fernando Pessoa?s novel The Anarchist Banker serves as an example for critical management education and allows for further insights into how anarchist theories may be reflected upon and practiced in a business school context. We explore elements of an ?anarchist aesthetics? that are created through dramaturgy, narration, and collective production and reception. The Anarchist Banker fits well with arts-based education in business schools and efforts to learn lessons for leadership through the use of drama. The literary source encourages to rethink salient issues in today?s global and finance-dominated capitalism and offers opportunities to search for alternative forms of organizing society and the economy by questioning charismatic leadership and managerial rhetoric in favor of collective reasoning. Elements of an anarchist aesthetic include the deconstruction of the hero and authoritarian discourse, dialogue and polyphony, collectivity and obstructionism that are at play artistically and socially, integrating anarchist theory and practice in content and form. The topic links to new forms of resistance, with critical artists opposing the business world and academics attempting to play out the ?banker? versus the ?anarchist?

Online Learning with Ensembles

Supervised online learning with an ensemble of students randomized by the choice of initial conditions is analyzed. For the case of the perceptron learning rule, asymptotically the same improvement in the generalization error of the ensemble compared to the performance of a single student is found as in Gibbs learning. For more optimized learning rules, however, using an ensemble yields no improvement. This is explained by showing that for any learning rule $f$ a transform $\tilde{f}$ exists, such that a single student using $\tilde{f}$ has the same generalization behaviour as an ensemble of $f$-students.Comment: 8 pages, 1 figure. Submitted to J.Phys.

Atlas of soil reflectance properties

A compendium of soil spectral reflectance curves together with soil test results and site information is presented in an abbreviated manner listing those soil properties most important in influencing soil reflectance. Results are presented for 251 soils from 39 states and Brazil. A narrative key describes relationships between soil parameters and reflectance curves. All soils are classified according to the U.S. soil taxonomy and soil series name for ease of identification

Noisy regression and classification with continuous multilayer networks

We investigate zero temperature Gibbs learning for two classes of unrealizable rules which play an important role in practical applications of multilayer neural networks with differentiable activation functions: classification problems and noisy regression problems. Considering one step of replica symmetry breaking, we surprisingly find that for sufficiently large training sets the stable state is replica symmetric even though the target rule is unrealizable. Further, the classification problem is shown to be formally equivalent to the noisy regression problem.Comment: 7 pages, including 2 figure