Optimisation of on-line principal component analysis

Different techniques, used to optimise on-line principal component analysis, are investigated by methods of statistical mechanics. These include local and global optimisation of node-dependent learning-rates which are shown to be very efficient in speeding up the learning process. They are investigated further for gaining insight into the learning rates' time-dependence, which is then employed for devising simple practical methods to improve training performance. Simulations demonstrate the benefit gained from using the new methods.Comment: 10 pages, 5 figure

Vegetation and soils field research data base: Experiment summaries

Understanding of the relationships between the optical, spectral characteristics and important biological-physical parameters of earth-surface features can best be obtained by carefully controlled studies over fields and plots where complete data describing the condition of targets are attainable and where frequent, timely spectral measurement can be obtained. Development of a vegetation and soils field research data base was initiated in 1972 at Purdue University's Laboratory for Applications of Remote Sensing and expanded in the fall of 1974 by NASA as part of LACIE. Since then, over 250,000 truck-mounted and helicopter-borne spectrometer/multiband radiometer observations have been obtained of more than 50 soil series and 20 species of crops, grasses, and trees. These data are supplemented by an extensive set of biophysical and meteorological data acquired during each mission. The field research data form one of the most complete and best-documented data sets acquired for agricultural remote sensing research. Thus, they are well-suited to serve as a data base for research to: (1) quantiatively determine the relationships of spectral and biophysical characteristics of vegetation, (2) define future sensor systems, and (3) develop advanced data analysis techniques

Dynamics of Learning with Restricted Training Sets I: General Theory

We study the dynamics of supervised learning in layered neural networks, in the regime where the size $p$ of the training set is proportional to the number $N$ of inputs. Here the local fields are no longer described by Gaussian probability distributions and the learning dynamics is of a spin-glass nature, with the composition of the training set playing the role of quenched disorder. We show how dynamical replica theory can be used to predict the evolution of macroscopic observables, including the two relevant performance measures (training error and generalization error), incorporating the old formalism developed for complete training sets in the limit $\alpha=p/N\to\infty$ as a special case. For simplicity we restrict ourselves in this paper to single-layer networks and realizable tasks.Comment: 39 pages, LaTe

Phase transitions in optimal unsupervised learning

We determine the optimal performance of learning the orientation of the symmetry axis of a set of P = alpha N points that are uniformly distributed in all the directions but one on the N-dimensional sphere. The components along the symmetry breaking direction, of unitary vector B, are sampled from a mixture of two gaussians of variable separation and width. The typical optimal performance is measured through the overlap Ropt=B.J* where J* is the optimal guess of the symmetry breaking direction. Within this general scenario, the learning curves Ropt(alpha) may present first order transitions if the clusters are narrow enough. Close to these transitions, high performance states can be obtained through the minimization of the corresponding optimal potential, although these solutions are metastable, and therefore not learnable, within the usual bayesian scenario.Comment: 9 pages, 8 figures, submitted to PRE, This new version of the paper contains one new section, Bayesian versus optimal solutions, where we explain in detail the results supporting our claim that bayesian learning may not be optimal. Figures 4 of the first submission was difficult to understand. We replaced it by two new figures (Figs. 4 and 5 in this new version) containing more detail

The Impact of School-Connected Behavioral and Emotional Health Interventions on Student Academic Performance

This annotated bibliography provides a systematic review of current literature published between 2001 and 2013 and summarizes findings on the relationship between prevention-focused behavioral health interventions, such as social and emotional (SEL) learning programs, and their impact on academic outcomes

Noisy regression and classification with continuous multilayer networks

We investigate zero temperature Gibbs learning for two classes of unrealizable rules which play an important role in practical applications of multilayer neural networks with differentiable activation functions: classification problems and noisy regression problems. Considering one step of replica symmetry breaking, we surprisingly find that for sufficiently large training sets the stable state is replica symmetric even though the target rule is unrealizable. Further, the classification problem is shown to be formally equivalent to the noisy regression problem.Comment: 7 pages, including 2 figure

Linear polarization of light by two wheat canopies measured at many view angles

The linear polarization and reflection of visible light by wheat as a function of sun-view directions, crop development stage, and wavelength were examined. Two-hundred spectra were taken continuously in wave-lengths from 0.45 to 0.72 Micron in 33 view directions using an Exotech model 20C spectroradiometer six meters above two wheat canopies in the boot and fully headed maturity stages. The analysis results show that the amount of linearly polarized light from the wheat canopies is greatest in the blue spectral region and decreases gradually with increasing wavelength. The results also show that the linearly polarized light from the canopies is generally greatest in the azimuth direction of the Sun and tends toward zero as the view direction tends toward the direction of the hot spot or anti-solar point. It is demonstrated that the single, angle of incidence of sunlight on the leaf, explains almost all of the variation of the amount of polarized light with Sun-view direction

A two step algorithm for learning from unspecific reinforcement

We study a simple learning model based on the Hebb rule to cope with "delayed", unspecific reinforcement. In spite of the unspecific nature of the information-feedback, convergence to asymptotically perfect generalization is observed, with a rate depending, however, in a non- universal way on learning parameters. Asymptotic convergence can be as fast as that of Hebbian learning, but may be slower. Moreover, for a certain range of parameter settings, it depends on initial conditions whether the system can reach the regime of asymptotically perfect generalization, or rather approaches a stationary state of poor generalization.Comment: 13 pages LaTeX, 4 figures, note on biologically motivated stochastic variant of the algorithm adde

Simulated response of a multispectral scanner over wheat as a function of wavelength and view/illumination direction

The reflectance response with view angle of wheat, was analyzed. The analyses, which assumes there are no atmospheric effects, and otherwise simulates the response of a multispectral scanner, is based upon spectra taken continuously in wavelength from 0.45 to 2.4 micrometers at more than 1200 view/illumination directions using an Exotech model 20C spectra radiometer. Data were acquired six meters above four wheat canopies, each at a different growth stage. The analysis shows that the canopy reflective response is a pronounced function of illumination angle, scanner view angle and wavelength. The variation is greater at low solar elevations compared to high solar elevations