682,597 research outputs found
Ethical Testing in the Real World: Evaluating Physical Testing of Adversarial Machine Learning
This paper critically assesses the adequacy and representativeness of
physical domain testing for various adversarial machine learning (ML) attacks
against computer vision systems involving human subjects. Many papers that
deploy such attacks characterize themselves as "real world." Despite this
framing, however, we found the physical or real-world testing conducted was
minimal, provided few details about testing subjects and was often conducted as
an afterthought or demonstration. Adversarial ML research without
representative trials or testing is an ethical, scientific, and health/safety
issue that can cause real harms. We introduce the problem and our methodology,
and then critique the physical domain testing methodologies employed by papers
in the field. We then explore various barriers to more inclusive physical
testing in adversarial ML and offer recommendations to improve such testing
notwithstanding these challenges.Comment: Accepted to NeurIPS 2020 Workshop on Dataset Curation and Security;
Also accepted at Navigating the Broader Impacts of AI Research Workshop. All
authors contributed equally. The list of authors is arranged alphabeticall
A Kernel Independence Test for Random Processes
A new non parametric approach to the problem of testing the independence of
two random process is developed. The test statistic is the Hilbert Schmidt
Independence Criterion (HSIC), which was used previously in testing
independence for i.i.d pairs of variables. The asymptotic behaviour of HSIC is
established when computed from samples drawn from random processes. It is shown
that earlier bootstrap procedures which worked in the i.i.d. case will fail for
random processes, and an alternative consistent estimate of the p-values is
proposed. Tests on artificial data and real-world Forex data indicate that the
new test procedure discovers dependence which is missed by linear approaches,
while the earlier bootstrap procedure returns an elevated number of false
positives. The code is available online:
https://github.com/kacperChwialkowski/HSIC .Comment: In Proceedings of The 31st International Conference on Machine
Learnin
Experimental Design for Bathymetry Editing
We describe an application of machine learning to a real-world computer
assisted labeling task. Our experimental results expose significant deviations
from the IID assumption commonly used in machine learning. These results
suggest that the common random split of all data into training and testing can
often lead to poor performance.Comment: Published as a workshop paper at ICML 2020 Workshop on Real World
Experiment Design and Active Learnin
- …