96,603 research outputs found
Distributed Learning over Unreliable Networks
Most of today's distributed machine learning systems assume {\em reliable
networks}: whenever two machines exchange information (e.g., gradients or
models), the network should guarantee the delivery of the message. At the same
time, recent work exhibits the impressive tolerance of machine learning
algorithms to errors or noise arising from relaxed communication or
synchronization. In this paper, we connect these two trends, and consider the
following question: {\em Can we design machine learning systems that are
tolerant to network unreliability during training?} With this motivation, we
focus on a theoretical problem of independent interest---given a standard
distributed parameter server architecture, if every communication between the
worker and the server has a non-zero probability of being dropped, does
there exist an algorithm that still converges, and at what speed? The technical
contribution of this paper is a novel theoretical analysis proving that
distributed learning over unreliable network can achieve comparable convergence
rate to centralized or distributed learning over reliable networks. Further, we
prove that the influence of the packet drop rate diminishes with the growth of
the number of \textcolor{black}{parameter servers}. We map this theoretical
result onto a real-world scenario, training deep neural networks over an
unreliable network layer, and conduct network simulation to validate the system
improvement by allowing the networks to be unreliable
Anomaly detection for machine learning redshifts applied to SDSS galaxies
We present an analysis of anomaly detection for machine learning redshift
estimation. Anomaly detection allows the removal of poor training examples,
which can adversely influence redshift estimates. Anomalous training examples
may be photometric galaxies with incorrect spectroscopic redshifts, or galaxies
with one or more poorly measured photometric quantity. We select 2.5 million
'clean' SDSS DR12 galaxies with reliable spectroscopic redshifts, and 6730
'anomalous' galaxies with spectroscopic redshift measurements which are flagged
as unreliable. We contaminate the clean base galaxy sample with galaxies with
unreliable redshifts and attempt to recover the contaminating galaxies using
the Elliptical Envelope technique. We then train four machine learning
architectures for redshift analysis on both the contaminated sample and on the
preprocessed 'anomaly-removed' sample and measure redshift statistics on a
clean validation sample generated without any preprocessing. We find an
improvement on all measured statistics of up to 80% when training on the
anomaly removed sample as compared with training on the contaminated sample for
each of the machine learning routines explored. We further describe a method to
estimate the contamination fraction of a base data sample.Comment: 13 pages, 8 figures, 1 table, minor text updates to macth MNRAS
accepted versio
Adding Neural Network Controllers to Behavior Trees without Destroying Performance Guarantees
In this paper, we show how Behavior Trees that have performance guarantees,
in terms of safety and goal convergence, can be extended with components that
were designed using machine learning, without destroying those performance
guarantees.
Machine learning approaches such as reinforcement learning or learning from
demonstration can be very appealing to AI designers that want efficient and
realistic behaviors in their agents. However, those algorithms seldom provide
guarantees for solving the given task in all different situations while keeping
the agent safe. Instead, such guarantees are often easier to find for manually
designed model based approaches. In this paper we exploit the modularity of
Behavior trees to extend a given design with an efficient, but possibly
unreliable, machine learning component in a way that preserves the guarantees.
The approach is illustrated with an inverted pendulum example.Comment: Submitted to IEEE Transactions on Game
Q-Class Authentication System for Double Arbiter PUF
Physically Unclonable Function (PUF) is a cryptographic primitive that is based on physical property of each entity or Integrated Circuit (IC) chip. It is expected that PUF be used in security applications such as ID generation and authentication. Some responses from PUF are unreliable, and they are usually discarded. In this paper, we propose a new PUF-based authentication system that exploits information of unreliable responses. In the proposed method, each response is categorized into multiple classes by its unreliability evaluated by feeding the same challenges several times. This authentication system is named Q-class authentication, where Q is the number of classes. We perform experiments assuming a challenge-response authentication system with a certain threshold of errors. Considering 4-class separation for 4-1 Double Arbiter PUF, it is figured out that the advantage of a legitimate prover against a clone is improved form 24% to 36% in terms of success rate. In other words, it is possible to improve the tolerance of machine-learning attack by using unreliable information that was previously regarded disadvantageous to authentication systems
Reliability and validity in comparative studies of software prediction models
Empirical studies on software prediction models do not converge with respect to the question "which prediction model is best?" The reason for this lack of convergence is poorly understood. In this simulation study, we have examined a frequently used research procedure comprising three main ingredients: a single data sample, an accuracy indicator, and cross validation. Typically, these empirical studies compare a machine learning model with a regression model. In our study, we use simulation and compare a machine learning and a regression model. The results suggest that it is the research procedure itself that is unreliable. This lack of reliability may strongly contribute to the lack of convergence. Our findings thus cast some doubt on the conclusions of any study of competing software prediction models that used this research procedure as a basis of model comparison. Thus, we need to develop more reliable research procedures before we can have confidence in the conclusions of comparative studies of software prediction models
Is Big Data Sufficient for a Reliable Detection of Non-Technical Losses?
Non-technical losses (NTL) occur during the distribution of electricity in
power grids and include, but are not limited to, electricity theft and faulty
meters. In emerging countries, they may range up to 40% of the total
electricity distributed. In order to detect NTLs, machine learning methods are
used that learn irregular consumption patterns from customer data and
inspection results. The Big Data paradigm followed in modern machine learning
reflects the desire of deriving better conclusions from simply analyzing more
data, without the necessity of looking at theory and models. However, the
sample of inspected customers may be biased, i.e. it does not represent the
population of all customers. As a consequence, machine learning models trained
on these inspection results are biased as well and therefore lead to unreliable
predictions of whether customers cause NTL or not. In machine learning, this
issue is called covariate shift and has not been addressed in the literature on
NTL detection yet. In this work, we present a novel framework for quantifying
and visualizing covariate shift. We apply it to a commercial data set from
Brazil that consists of 3.6M customers and 820K inspection results. We show
that some features have a stronger covariate shift than others, making
predictions less reliable. In particular, previous inspections were focused on
certain neighborhoods or customer classes and that they were not sufficiently
spread among the population of customers. This framework is about to be
deployed in a commercial product for NTL detection.Comment: Proceedings of the 19th International Conference on Intelligent
System Applications to Power Systems (ISAP 2017
On the Reduction of Biases in Big Data Sets for the Detection of Irregular Power Usage
In machine learning, a bias occurs whenever training sets are not
representative for the test data, which results in unreliable models. The most
common biases in data are arguably class imbalance and covariate shift. In this
work, we aim to shed light on this topic in order to increase the overall
attention to this issue in the field of machine learning. We propose a scalable
novel framework for reducing multiple biases in high-dimensional data sets in
order to train more reliable predictors. We apply our methodology to the
detection of irregular power usage from real, noisy industrial data. In
emerging markets, irregular power usage, and electricity theft in particular,
may range up to 40% of the total electricity distributed. Biased data sets are
of particular issue in this domain. We show that reducing these biases
increases the accuracy of the trained predictors. Our models have the potential
to generate significant economic value in a real world application, as they are
being deployed in a commercial software for the detection of irregular power
usage
- …