634 research outputs found
Analysis of dropout learning regarded as ensemble learning
Deep learning is the state-of-the-art in fields such as visual object
recognition and speech recognition. This learning uses a large number of
layers, huge number of units, and connections. Therefore, overfitting is a
serious problem. To avoid this problem, dropout learning is proposed. Dropout
learning neglects some inputs and hidden units in the learning process with a
probability, p, and then, the neglected inputs and hidden units are combined
with the learned network to express the final output. We find that the process
of combining the neglected hidden units with the learned network can be
regarded as ensemble learning, so we analyze dropout learning from this point
of view.Comment: 9 pages, 8 figures, submitted to Conferenc
Exploratory Observation Machine (XOM) with Kullback-Leibler Divergence for Dimensionality Reduction and Visualization
Exploratory Observation Machine (XOM) with Kullback-Leibler Divergence for Dimensionality Reduction and Visualization
A Low-Cost 3-D Printed Smartphone Add-on Spectrometer for Diagnosis of Crop Diseases in Field
We present our initial proof of concept study towards the development of a low-cost 3-D printed smartphone add-on spectrometer. The study aimed at developing a cheap technology (less than 5 USD) to be used for detection of crop diseases in the field using spectrometry. Previously, we experimented with the problem of disease diagnosis using an off-the-shelf and expensive spectrometer (approximately 1000 USD). However, in real world practice, this off-the-shelf device can not be used by typical users (smallholder farmers). Therefore, the study presents a tool that is cheap and user friendly. We present preliminary results and identify requirements for a future version aiming at an accurate diagnostic technology to be used in the field before disease symptoms are visibly seen by the naked eye. Evaluation shows performance of the tool is better than random however below performance of an industry grade spectrometer
Generalizing with perceptrons in case of structured phase- and pattern-spaces
We investigate the influence of different kinds of structure on the learning
behaviour of a perceptron performing a classification task defined by a teacher
rule. The underlying pattern distribution is permitted to have spatial
correlations. The prior distribution for the teacher coupling vectors itself is
assumed to be nonuniform. Thus classification tasks of quite different
difficulty are included. As learning algorithms we discuss Hebbian learning,
Gibbs learning, and Bayesian learning with different priors, using methods from
statistics and the replica formalism. We find that the Hebb rule is quite
sensitive to the structure of the actual learning problem, failing
asymptotically in most cases. Contrarily, the behaviour of the more
sophisticated methods of Gibbs and Bayes learning is influenced by the spatial
correlations only in an intermediate regime of , where
specifies the size of the training set. Concerning the Bayesian case we show,
how enhanced prior knowledge improves the performance.Comment: LaTeX, 32 pages with eps-figs, accepted by J Phys
Exploratory Observation Machine (XOM) with Kullback-Leibler Divergence for Dimensionality Reduction and Visualization
On-line learning with adaptive back-propagation in two-layer networks
An adaptive back-propagation algorithm parameterized by an inverse temperature 1/T is studied and compared with gradient descent (standard back-propagation) for on-line learning in two-layer neural networks with an arbitrary number of hidden units. Within a statistical mechanics framework, we analyse these learning algorithms in both the symmetric and the convergence phase for finite learning rates in the case of uncorrelated teachers of similar but arbitrary length T. These analyses show that adaptive back-propagation results generally in faster training by breaking the symmetry between hidden units more efficiently and by providing faster convergence to optimal generalization than gradient descent
On-Line AdaTron Learning of Unlearnable Rules
We study the on-line AdaTron learning of linearly non-separable rules by a
simple perceptron. Training examples are provided by a perceptron with a
non-monotonic transfer function which reduces to the usual monotonic relation
in a certain limit. We find that, although the on-line AdaTron learning is a
powerful algorithm for the learnable rule, it does not give the best possible
generalization error for unlearnable problems. Optimization of the learning
rate is shown to greatly improve the performance of the AdaTron algorithm,
leading to the best possible generalization error for a wide range of the
parameter which controls the shape of the transfer function.)Comment: RevTeX 17 pages, 8 figures, to appear in Phys.Rev.
A lattice gas model of II-VI(001) semiconductor surfaces
We introduce an anisotropic two-dimensional lattice gas model of metal
terminated II-IV(001) seminconductor surfaces. Important properties of this
class of materials are represented by effective NN and NNN interactions, which
result in the competition of two vacancy structures on the surface. We
demonstrate that the experimentally observed c(2x2)-(2x1) transition of the
CdTe(001) surface can be understood as a phase transition in thermal
equilbrium. The model is studied by means of transfer matrix and Monte Carlo
techniques. The analysis shows that the small energy difference of the
competing reconstructions determines to a large extent the nature of the
different phases. Possible implications for further experimental research are
discussed.Comment: 7 pages, 2 figure
- …