72,913 research outputs found
Machine Learning for Intrusion Detection: Modeling the Distribution Shift
This paper addresses two important issue that arise in formulating and solving computer intrusion detection as a machine learning problem, a topic that has attracted considerable attention in recent years including a community wide competition using a common data set known as the KDD Cup â99. The first of these problems we address is the size of the data set, 5 Ă 106 by 41 features, which makes conventional learning algorithms impractical. In previous work, we introduced a one-pass non-parametric classification technique called Voted Spheres, which carves up the input space into a series of overlapping hyperspheres. Training data seen within each hypersphere is used in a voting scheme during testing on unseen data. Secondly, we address the problem of distribution shift whereby the training and test data may be drawn from slightly different probability densities, while the conditional densities of class membership for a given datum remains the same. We adopt two recent techniques from the literature, density weighting and kernel mean matching, to enhance the Voted Spheres technique to deal with such distribution disparities. We demonstrate that substantial performance gains can be achieved using these techniques on the KDD cup data set
An Active Instance-based Machine Learning method for Stellar Population Studies
We have developed a method for fast and accurate stellar population
parameters determination in order to apply it to high resolution galaxy
spectra. The method is based on an optimization technique that combines active
learning with an instance-based machine learning algorithm. We tested the
method with the retrieval of the star-formation history and dust content in
"synthetic" galaxies with a wide range of S/N ratios. The "synthetic" galaxies
where constructed using two different grids of high resolution theoretical
population synthesis models. The results of our controlled experiment shows
that our method can estimate with good speed and accuracy the parameters of the
stellar populations that make up the galaxy even for very low S/N input. For a
spectrum with S/N=5 the typical average deviation between the input and fitted
spectrum is less than 10**{-5}. Additional improvements are achieved using
prior knowledge.Comment: 14 pages, 25 figures, accepted by Monthly Notice
Gaussian Artmap: A Neural Network for Fast Incremental Learning of Noisy Multidimensional Maps
A new neural network architecture for incremental supervised learning of analalog multidimensional maps is introduced. The architecture, called Gaussian ARTMAP, is a synthesis of a Gaussian classifier and an Adaptive Resonance Theory (ART) neural network, achieved by defining the ART choice function as the discriminant function of a Gaussian classifer with separable distributions, and the ART match function as the same, but with the a priori probabilities of the distributions discounted. While Gaussian ARTMAP retains the attractive parallel computing and fast learning properties of fuzzy ARTMAP, it learns a more efficient internal representation of a mapping while being more resistant to noise than fuzzy ARTMAP on a number of benchmark databases. Several simulations are presented which demonstrate that Gaussian ARTMAP consistently obtains a better trade-off of classification rate to number of categories than fuzzy ARTMAP. Results on a vowel classiflcation problem are also presented which demonstrate that Gaussian ARTMAP outperforms many other classifiers.National Science Foundation (IRI 90-00530); Office of Naval Research (N00014-92-J-4015, 40014-91-J-4100
Geoeffectiveness of Coronal Mass Ejections in the SOHO era
The main objective of the study is to determine the probability distributions
of the geomagnetic Dst index as a function of the coronal mass ejection (CME)
and solar flare parameters for the purpose of establishing a probabilistic
forecast tool for the geomagnetic storm intensity. Several CME and flare
parameters as well as the effect of successive-CME occurrence in changing the
probability for a certain range of Dst index values, were examined. The results
confirm some of already known relationships between remotely-observed
properties of solar eruptive events and geomagnetic storms, namely the
importance of initial CME speed, apparent width, source position, and the
associated solar flare class. In this paper we quantify these relationships in
a form to be used for space weather forecasting in future. The results of the
statistical study are employed to construct an empirical statistical model for
predicting the probability of the geomagnetic storm intensity based on remote
solar observations of CMEs and flares
Looking Beyond Label Noise: Shifted Label Distribution Matters in Distantly Supervised Relation Extraction
In recent years there is a surge of interest in applying distant supervision
(DS) to automatically generate training data for relation extraction (RE). In
this paper, we study the problem what limits the performance of DS-trained
neural models, conduct thorough analyses, and identify a factor that can
influence the performance greatly, shifted label distribution. Specifically, we
found this problem commonly exists in real-world DS datasets, and without
special handing, typical DS-RE models cannot automatically adapt to this shift,
thus achieving deteriorated performance. To further validate our intuition, we
develop a simple yet effective adaptation method for DS-trained models, bias
adjustment, which updates models learned over the source domain (i.e., DS
training set) with a label distribution estimated on the target domain (i.e.,
test set). Experiments demonstrate that bias adjustment achieves consistent
performance gains on DS-trained models, especially on neural models, with an up
to 23% relative F1 improvement, which verifies our assumptions. Our code and
data can be found at
\url{https://github.com/INK-USC/shifted-label-distribution}.Comment: 13 pages: 10 pages paper, 3 pages appendix. Appears at EMNLP 201
Discriminative Density-ratio Estimation
The covariate shift is a challenging problem in supervised learning that
results from the discrepancy between the training and test distributions. An
effective approach which recently drew a considerable attention in the research
community is to reweight the training samples to minimize that discrepancy. In
specific, many methods are based on developing Density-ratio (DR) estimation
techniques that apply to both regression and classification problems. Although
these methods work well for regression problems, their performance on
classification problems is not satisfactory. This is due to a key observation
that these methods focus on matching the sample marginal distributions without
paying attention to preserving the separation between classes in the reweighted
space. In this paper, we propose a novel method for Discriminative
Density-ratio (DDR) estimation that addresses the aforementioned problem and
aims at estimating the density-ratio of joint distributions in a class-wise
manner. The proposed algorithm is an iterative procedure that alternates
between estimating the class information for the test data and estimating new
density ratio for each class. To incorporate the estimated class information of
the test data, a soft matching technique is proposed. In addition, we employ an
effective criterion which adopts mutual information as an indicator to stop the
iterative procedure while resulting in a decision boundary that lies in a
sparse region. Experiments on synthetic and benchmark datasets demonstrate the
superiority of the proposed method in terms of both accuracy and robustness
A novel neural prediction error found in anterior cingulate cortex ensembles
The function of the anterior cingulate cortex (ACC) remains controversial, yet many theories suggest a role in behavioral adaptation, partly because a robust event-related potential, the feedback-related negativity (FN), is evoked over the ACC whenever expectations are violated. We recorded from the ACC as rats performed a task identical to one that reliably evokes an FN in humans. A subset of neurons was found that encoded expected outcomes as abstract outcome representations. The degree to which a reward/non-reward outcome representation emerged during a trial depended on the history of outcomes that preceded it. A prediction error was generated on incongruent trials as the ensembles shifted from representing the expected to the actual outcome, at the same time point we have previously reported an FN in the local field potential. The results describe a novel mode of prediction error signaling by ACC neurons that is associated with the generation of an FN
- âŠ