9,145 research outputs found
Fairness Testing: Testing Software for Discrimination
This paper defines software fairness and discrimination and develops a
testing-based method for measuring if and how much software discriminates,
focusing on causality in discriminatory behavior. Evidence of software
discrimination has been found in modern software systems that recommend
criminal sentences, grant access to financial products, and determine who is
allowed to participate in promotions. Our approach, Themis, generates efficient
test suites to measure discrimination. Given a schema describing valid system
inputs, Themis generates discrimination tests automatically and does not
require an oracle. We evaluate Themis on 20 software systems, 12 of which come
from prior work with explicit focus on avoiding discrimination. We find that
(1) Themis is effective at discovering software discrimination, (2)
state-of-the-art techniques for removing discrimination from algorithms fail in
many situations, at times discriminating against as much as 98% of an input
subdomain, (3) Themis optimizations are effective at producing efficient test
suites for measuring discrimination, and (4) Themis is more efficient on
systems that exhibit more discrimination. We thus demonstrate that fairness
testing is a critical aspect of the software development cycle in domains with
possible discrimination and provide initial tools for measuring software
discrimination.Comment: Sainyam Galhotra, Yuriy Brun, and Alexandra Meliou. 2017. Fairness
Testing: Testing Software for Discrimination. In Proceedings of 2017 11th
Joint Meeting of the European Software Engineering Conference and the ACM
SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE),
Paderborn, Germany, September 4-8, 2017 (ESEC/FSE'17).
https://doi.org/10.1145/3106237.3106277, ESEC/FSE, 201
Detecting multineuronal temporal patterns in parallel spike trains
We present a non-parametric and computationally efficient method that detects spatiotemporal firing patterns and pattern sequences in parallel spike trains and tests whether the observed numbers of repeating patterns and sequences on a given timescale are significantly different from those expected by chance. The method is generally applicable and uncovers coordinated activity with arbitrary precision by comparing it to appropriate surrogate data. The analysis of coherent patterns of spatially and temporally distributed spiking activity on various timescales enables the immediate tracking of diverse qualities of coordinated firing related to neuronal state changes and information processing. We apply the method to simulated data and multineuronal recordings from rat visual cortex and show that it reliably discriminates between data sets with random pattern occurrences and with additional exactly repeating spatiotemporal patterns and pattern sequences. Multineuronal cortical spiking activity appears to be precisely coordinated and exhibits a sequential organization beyond the cell assembly concept
A Systematic Review of the Application and Empirical Investigation of Search-Based Test Case Generation
Otsingupõhine tarkvara testimine kasutab metaheuristilisi algoritme, et automatiseerida testide genereerimist. Selle töö eesmärgiks on osaliselt taasluua 2010. aastal kirjutatud Ali et al. artikkel, et uurida, kuidas on aastatel 2008-2015 kasutatud metaheuristilisi algoritme testide loomiseks. See töö analüüsib, kuidas on antud artiklid koostatud ning kuidas neis on algoritmide maksumust ja efektiivsust hinnatud. Kogutud tulemusi võrreldakse Ali et al. tulemustega.Search based software testing uses metaheuristic algorithms to automate the generation of test cases. This thesis partially replicates a literature study published in 2010 by Ali et al. to determine how studies published in 2008-2015 use metaheuristic algorithms to automate the generation of test cases. The thesis analyses how these studies were conducted and how the cost-effectiveness is assessed in these papers. The trends detected in the new publications are compared to those presented in Ali et al
Sampling From A Manifold
We develop algorithms for sampling from a probability distribution on a
submanifold embedded in Rn. Applications are given to the evaluation of
algorithms in 'Topological Statistics'; to goodness of fit tests in exponential
families and to Neyman's smooth test. This article is partially expository,
giving an introduction to the tools of geometric measure theory
Detecting Activations over Graphs using Spanning Tree Wavelet Bases
We consider the detection of activations over graphs under Gaussian noise,
where signals are piece-wise constant over the graph. Despite the wide
applicability of such a detection algorithm, there has been little success in
the development of computationally feasible methods with proveable theoretical
guarantees for general graph topologies. We cast this as a hypothesis testing
problem, and first provide a universal necessary condition for asymptotic
distinguishability of the null and alternative hypotheses. We then introduce
the spanning tree wavelet basis over graphs, a localized basis that reflects
the topology of the graph, and prove that for any spanning tree, this approach
can distinguish null from alternative in a low signal-to-noise regime. Lastly,
we improve on this result and show that using the uniform spanning tree in the
basis construction yields a randomized test with stronger theoretical
guarantees that in many cases matches our necessary conditions. Specifically,
we obtain near-optimal performance in edge transitive graphs, -nearest
neighbor graphs, and -graphs
Adapting Quality Assurance to Adaptive Systems: The Scenario Coevolution Paradigm
From formal and practical analysis, we identify new challenges that
self-adaptive systems pose to the process of quality assurance. When tackling
these, the effort spent on various tasks in the process of software engineering
is naturally re-distributed. We claim that all steps related to testing need to
become self-adaptive to match the capabilities of the self-adaptive
system-under-test. Otherwise, the adaptive system's behavior might elude
traditional variants of quality assurance. We thus propose the paradigm of
scenario coevolution, which describes a pool of test cases and other
constraints on system behavior that evolves in parallel to the (in part
autonomous) development of behavior in the system-under-test. Scenario
coevolution offers a simple structure for the organization of adaptive testing
that allows for both human-controlled and autonomous intervention, supporting
software engineering for adaptive systems on a procedural as well as technical
level.Comment: 17 pages, published at ISOLA 201
Do Judge a Test by its Cover: Combining Combinatorial and Property-Based Testing
Property-based testing uses randomly generated inputs to validate high-level program specifications. It can be shockingly effective at finding bugs, but it often requires generating a very large number of inputs to do so. In this paper, we apply ideas from combinatorial testing, a powerful and widely studied testing methodology, to modify the distributions of our random generators so as to find bugs with fewer tests. The key concept is combinatorial coverage, which measures the degree to which a given set of tests exercises every possible choice of values for every small combination of input features. In its “classical” form, combinatorial coverage only applies to programs whose inputs have a very particular shape—essentially, a Cartesian product of finite sets. We generalize combinatorial coverage to the richer world of algebraic data types by formalizing a class of sparse test descriptions based on regular tree expressions. This new definition of coverage inspires a novel combinatorial thinning algorithm for improving the coverage of random test generators, requiring many fewer tests to catch bugs. We evaluate this algorithm on two case studies, a typed evaluator for System F terms and a Haskell compiler, showing significant improvements in both
- …