Analysis of dropout learning regarded as ensemble learning

Deep learning is the state-of-the-art in fields such as visual object recognition and speech recognition. This learning uses a large number of layers, huge number of units, and connections. Therefore, overfitting is a serious problem. To avoid this problem, dropout learning is proposed. Dropout learning neglects some inputs and hidden units in the learning process with a probability, p, and then, the neglected inputs and hidden units are combined with the learned network to express the final output. We find that the process of combining the neglected hidden units with the learned network can be regarded as ensemble learning, so we analyze dropout learning from this point of view.Comment: 9 pages, 8 figures, submitted to Conferenc

A Low-Cost 3-D Printed Smartphone Add-on Spectrometer for Diagnosis of Crop Diseases in Field

We present our initial proof of concept study towards the development of a low-cost 3-D printed smartphone add-on spectrometer. The study aimed at developing a cheap technology (less than 5 USD) to be used for detection of crop diseases in the field using spectrometry. Previously, we experimented with the problem of disease diagnosis using an off-the-shelf and expensive spectrometer (approximately 1000 USD). However, in real world practice, this off-the-shelf device can not be used by typical users (smallholder farmers). Therefore, the study presents a tool that is cheap and user friendly. We present preliminary results and identify requirements for a future version aiming at an accurate diagnostic technology to be used in the field before disease symptoms are visibly seen by the naked eye. Evaluation shows performance of the tool is better than random however below performance of an industry grade spectrometer

Generalizing with perceptrons in case of structured phase- and pattern-spaces

We investigate the influence of different kinds of structure on the learning behaviour of a perceptron performing a classification task defined by a teacher rule. The underlying pattern distribution is permitted to have spatial correlations. The prior distribution for the teacher coupling vectors itself is assumed to be nonuniform. Thus classification tasks of quite different difficulty are included. As learning algorithms we discuss Hebbian learning, Gibbs learning, and Bayesian learning with different priors, using methods from statistics and the replica formalism. We find that the Hebb rule is quite sensitive to the structure of the actual learning problem, failing asymptotically in most cases. Contrarily, the behaviour of the more sophisticated methods of Gibbs and Bayes learning is influenced by the spatial correlations only in an intermediate regime of $\alpha$, where $\alpha$ specifies the size of the training set. Concerning the Bayesian case we show, how enhanced prior knowledge improves the performance.Comment: LaTeX, 32 pages with eps-figs, accepted by J Phys

On-line learning with adaptive back-propagation in two-layer networks

An adaptive back-propagation algorithm parameterized by an inverse temperature 1/T is studied and compared with gradient descent (standard back-propagation) for on-line learning in two-layer neural networks with an arbitrary number of hidden units. Within a statistical mechanics framework, we analyse these learning algorithms in both the symmetric and the convergence phase for finite learning rates in the case of uncorrelated teachers of similar but arbitrary length T. These analyses show that adaptive back-propagation results generally in faster training by breaking the symmetry between hidden units more efficiently and by providing faster convergence to optimal generalization than gradient descent

On-Line AdaTron Learning of Unlearnable Rules

We study the on-line AdaTron learning of linearly non-separable rules by a simple perceptron. Training examples are provided by a perceptron with a non-monotonic transfer function which reduces to the usual monotonic relation in a certain limit. We find that, although the on-line AdaTron learning is a powerful algorithm for the learnable rule, it does not give the best possible generalization error for unlearnable problems. Optimization of the learning rate is shown to greatly improve the performance of the AdaTron algorithm, leading to the best possible generalization error for a wide range of the parameter which controls the shape of the transfer function.)Comment: RevTeX 17 pages, 8 figures, to appear in Phys.Rev.

A lattice gas model of II-VI(001) semiconductor surfaces

We introduce an anisotropic two-dimensional lattice gas model of metal terminated II-IV(001) seminconductor surfaces. Important properties of this class of materials are represented by effective NN and NNN interactions, which result in the competition of two vacancy structures on the surface. We demonstrate that the experimentally observed c(2x2)-(2x1) transition of the CdTe(001) surface can be understood as a phase transition in thermal equilbrium. The model is studied by means of transfer matrix and Monte Carlo techniques. The analysis shows that the small energy difference of the competing reconstructions determines to a large extent the nature of the different phases. Possible implications for further experimental research are discussed.Comment: 7 pages, 2 figure