4,549 research outputs found
COMET: A Recipe for Learning and Using Large Ensembles on Massive Data
COMET is a single-pass MapReduce algorithm for learning on large-scale data.
It builds multiple random forest ensembles on distributed blocks of data and
merges them into a mega-ensemble. This approach is appropriate when learning
from massive-scale data that is too large to fit on a single machine. To get
the best accuracy, IVoting should be used instead of bagging to generate the
training subset for each decision tree in the random forest. Experiments with
two large datasets (5GB and 50GB compressed) show that COMET compares favorably
(in both accuracy and training time) to learning on a subsample of data using a
serial algorithm. Finally, we propose a new Gaussian approach for lazy ensemble
evaluation which dynamically decides how many ensemble members to evaluate per
data point; this can reduce evaluation cost by 100X or more
Compression and Classification Methods for Galaxy Spectra in Large Redshift Surveys
Methods for compression and classification of galaxy spectra, which are
useful for large galaxy redshift surveys (such as the SDSS, 2dF, 6dF and
VIRMOS), are reviewed. In particular, we describe and contrast three methods:
(i) Principal Component Analysis, (ii) Information Bottleneck, and (iii) Fisher
Matrix. We show applications to 2dF galaxy spectra and to mock semi-analytic
spectra, and we discuss how these methods can be used to study physical
processes of galaxy formation, clustering and galaxy biasing in the new large
redshift surveys.Comment: Review talk, proceedings of MPA/MPE/ESO Conference "Mining the Sky",
2000, Garching, Germany; 20 pages, 5 figure
Objective Classification of Galaxy Spectra using the Information Bottleneck Method
A new method for classification of galaxy spectra is presented, based on a
recently introduced information theoretical principle, the `Information
Bottleneck'. For any desired number of classes, galaxies are classified such
that the information content about the spectra is maximally preserved. The
result is classes of galaxies with similar spectra, where the similarity is
determined via a measure of information. We apply our method to approximately
6000 galaxy spectra from the ongoing 2dF redshift survey, and a mock-2dF
catalogue produced by a Cold Dark Matter-based semi-analytic model of galaxy
formation. We find a good match between the mean spectra of the classes found
in the data and in the models. For the mock catalogue, we find that the classes
produced by our algorithm form an intuitively sensible sequence in terms of
physical properties such as colour, star formation activity, morphology, and
internal velocity dispersion. We also show the correlation of the classes with
the projections resulting from a Principal Component Analysis.Comment: submitted to MNRAS, 17 pages, Latex, with 14 figures embedde
Neural networks with late-phase weights
The largely successful method of training neural networks is to learn their
weights using some variant of stochastic gradient descent (SGD). Here, we show
that the solutions found by SGD can be further improved by ensembling a subset
of the weights in late stages of learning. At the end of learning, we obtain
back a single model by taking a spatial average in weight space. To avoid
incurring increased computational costs, we investigate a family of
low-dimensional late-phase weight models which interact multiplicatively with
the remaining parameters. Our results show that augmenting standard models with
late-phase weights improves generalization in established benchmarks such as
CIFAR-10/100, ImageNet and enwik8. These findings are complemented with a
theoretical analysis of a noisy quadratic problem which provides a simplified
picture of the late phases of neural network learning.Comment: 25 pages, 6 figure
- …