Dynamics of Supervised Learning with Restricted Training Sets

We study the dynamics of supervised learning in layered neural networks, in the regime where the size $p$ of the training set is proportional to the number $N$ of inputs. Here the local fields are no longer described by Gaussian probability distributions. We show how dynamical replica theory can be used to predict the evolution of macroscopic observables, including the relevant performance measures, incorporating the old formalism in the limit $\alpha=p/N\to\infty$ as a special case. For simplicity we restrict ourselves to single-layer networks and realizable tasks.Comment: 36 pages, latex2e, 12 eps figures (to be publ in: Proc Newton Inst Workshop on On-Line Learning '97

Price and tax policy for semi-subsistence agriculture in Ethiopia

In the case of semi-subsistence agriculture where wage employment is not available, the role played by prices and taxes in determining production and consumption decisions is not clearly established by economic theories of household choice. This study demonstrates that where choices in production, consumption, and leisure can be made independently, farmers will decide what to grow on the basis of their preferences for marketed goods. The paper also points out that the choice will be affected by the level and type of taxation imposed. The paper shows the impact of four taxes -- agricultural revenue, land, production and marketed goods consumption -- on crop production and tax revenues. This paper also reports on a model of production in Ethiopia. The results of this study give strong evidence of the role of producer and consumer prices in semi-subsistence agriculture. In addition, the results show the importance of production capacity, household and climatic factors in agricultural development.Environmental Economics&Policies,Economic Theory&Research,Crops&Crop Management Systems,Agricultural Knowledge&Information Systems,Consumption

Superfluidity of Dense 4^4He in Vycor

We calculate properties of a model of $^4$He in Vycor using the Path Integral Monte Carlo method. We find that $^4$He forms a distinct layered structure with a highly localized first layer, a disordered second layer with some atoms delocalized and able to give rise to the observed superfluid response, and higher layers nearly perfect crystals. The addition of a single $^3$He atom was enough to bring down the total superfluidity by blocking the exchange in the second layer. Our results are consistent with the persistent liquid layer model to explain the observations. Such a model may be relevant to the experiments on bulk solid $^4$He, if there is a fine network of grain boundaries in those systems.Comment: 4 pages, 4 figure

Optimisation of on-line principal component analysis

Different techniques, used to optimise on-line principal component analysis, are investigated by methods of statistical mechanics. These include local and global optimisation of node-dependent learning-rates which are shown to be very efficient in speeding up the learning process. They are investigated further for gaining insight into the learning rates' time-dependence, which is then employed for devising simple practical methods to improve training performance. Simulations demonstrate the benefit gained from using the new methods.Comment: 10 pages, 5 figure

Exact normalized eigenfunctions for general deformed Hulth\'en potentials

The exact solutions of Schr\"odinger's equation with the deformed Hulth\'en potential $V_q(x)=-{\mu\, e^{-\delta\,x }}/({1-q\,e^{-\delta\,x}}),~ \delta,\mu, q>0$ are given, along with a closed--form formula for the normalization constants of the eigenfunctions for arbitrary $q>0$. The Crum-Darboux transformation is then used to derive the corresponding exact solutions for the extended Hulth\'en potentials $V(x)= -{\mu\, e^{-\delta\,x }}/({1-q\,e^{-\delta\,x}})+ {q\,j(j+1)\, e^{-\delta\,x }}/({1-q\,e^{-\delta\,x}})^2, j=0,1,2,\dots.$ A general formula for the new normalization condition is also provided.Comment: 14 pages, two figure

Solutions for certain classes of Riccati differential equation

We derive some analytic closed-form solutions for a class of Riccati equation y'(x)-\lambda_0(x)y(x)\pm y^2(x)=\pm s_0(x), where \lambda_0(x), s_0(x) are C^{\infty}-functions. We show that if \delta_n=\lambda_n s_{n-1}-\lambda_{n-1}s_n=0, where \lambda_{n}= \lambda_{n-1}^\prime+s_{n-1}+\lambda_0\lambda_{n-1} and s_{n}=s_{n-1}^\prime+s_0\lambda_{k-1}, n=1,2,..., then The Riccati equation has a solution given by y(x)=\mp s_{n-1}(x)/\lambda_{n-1}(x). Extension to the generalized Riccati equation y'(x)+P(x)y(x)+Q(x)y^2(x)=R(x) is also investigated.Comment: 10 page

On-Line Learning Theory of Soft Committee Machines with Correlated Hidden Units - Steepest Gradient Descent and Natural Gradient Descent -

The permutation symmetry of the hidden units in multilayer perceptrons causes the saddle structure and plateaus of the learning dynamics in gradient learning methods. The correlation of the weight vectors of hidden units in a teacher network is thought to affect this saddle structure, resulting in a prolonged learning time, but this mechanism is still unclear. In this paper, we discuss it with regard to soft committee machines and on-line learning using statistical mechanics. Conventional gradient descent needs more time to break the symmetry as the correlation of the teacher weight vectors rises. On the other hand, no plateaus occur with natural gradient descent regardless of the correlation for the limit of a low learning rate. Analytical results support these dynamics around the saddle point.Comment: 7 pages, 6 figure

The great turn-of-the-century housing boom

This article explains the recent high levels of residential investment and rates of homeownership.Home ownership ; Housing - Finance ; Housing - Prices