4,599 research outputs found
Fairness Testing: Testing Software for Discrimination
This paper defines software fairness and discrimination and develops a
testing-based method for measuring if and how much software discriminates,
focusing on causality in discriminatory behavior. Evidence of software
discrimination has been found in modern software systems that recommend
criminal sentences, grant access to financial products, and determine who is
allowed to participate in promotions. Our approach, Themis, generates efficient
test suites to measure discrimination. Given a schema describing valid system
inputs, Themis generates discrimination tests automatically and does not
require an oracle. We evaluate Themis on 20 software systems, 12 of which come
from prior work with explicit focus on avoiding discrimination. We find that
(1) Themis is effective at discovering software discrimination, (2)
state-of-the-art techniques for removing discrimination from algorithms fail in
many situations, at times discriminating against as much as 98% of an input
subdomain, (3) Themis optimizations are effective at producing efficient test
suites for measuring discrimination, and (4) Themis is more efficient on
systems that exhibit more discrimination. We thus demonstrate that fairness
testing is a critical aspect of the software development cycle in domains with
possible discrimination and provide initial tools for measuring software
discrimination.Comment: Sainyam Galhotra, Yuriy Brun, and Alexandra Meliou. 2017. Fairness
Testing: Testing Software for Discrimination. In Proceedings of 2017 11th
Joint Meeting of the European Software Engineering Conference and the ACM
SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE),
Paderborn, Germany, September 4-8, 2017 (ESEC/FSE'17).
https://doi.org/10.1145/3106237.3106277, ESEC/FSE, 201
Towards a Theory of Software Development Expertise
Software development includes diverse tasks such as implementing new
features, analyzing requirements, and fixing bugs. Being an expert in those
tasks requires a certain set of skills, knowledge, and experience. Several
studies investigated individual aspects of software development expertise, but
what is missing is a comprehensive theory. We present a first conceptual theory
of software development expertise that is grounded in data from a mixed-methods
survey with 335 software developers and in literature on expertise and expert
performance. Our theory currently focuses on programming, but already provides
valuable insights for researchers, developers, and employers. The theory
describes important properties of software development expertise and which
factors foster or hinder its formation, including how developers' performance
may decline over time. Moreover, our quantitative results show that developers'
expertise self-assessments are context-dependent and that experience is not
necessarily related to expertise.Comment: 14 pages, 5 figures, 26th ACM Joint European Software Engineering
Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE
2018), ACM, 201
Efficient and Reasonable Object-Oriented Concurrency
Making threaded programs safe and easy to reason about is one of the chief
difficulties in modern programming. This work provides an efficient execution
model for SCOOP, a concurrency approach that provides not only data race
freedom but also pre/postcondition reasoning guarantees between threads. The
extensions we propose influence both the underlying semantics to increase the
amount of concurrent execution that is possible, exclude certain classes of
deadlocks, and enable greater performance. These extensions are used as the
basis an efficient runtime and optimization pass that improve performance 15x
over a baseline implementation. This new implementation of SCOOP is also 2x
faster than other well-known safe concurrent languages. The measurements are
based on both coordination-intensive and data-manipulation-intensive benchmarks
designed to offer a mixture of workloads.Comment: Proceedings of the 10th Joint Meeting of the European Software
Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of
Software Engineering (ESEC/FSE '15). ACM, 201
Software Architecture in Practice: Challenges and Opportunities
Software architecture has been an active research field for nearly four
decades, in which previous studies make significant progress such as creating
methods and techniques and building tools to support software architecture
practice. Despite past efforts, we have little understanding of how
practitioners perform software architecture related activities, and what
challenges they face. Through interviews with 32 practitioners from 21
organizations across three continents, we identified challenges that
practitioners face in software architecture practice during software
development and maintenance. We reported on common software architecture
activities at software requirements, design, construction and testing, and
maintenance stages, as well as corresponding challenges. Our study uncovers
that most of these challenges center around management, documentation, tooling
and process, and collects recommendations to address these challenges.Comment: Preprint of Full Research Paper, the 31st ACM Joint European Software
Engineering Conference and Symposium on the Foundations of Software
Engineering (ESEC/FSE '23
PRF: A Framework for Building Automatic Program Repair Prototypes for JVM-Based Languages
PRF is a Java-based framework that allows researchers to build prototypes of
test-based generate-and-validate automatic program repair techniques for JVM
languages by simply extending it with their patch generation plugins. The
framework also provides other useful components for constructing automatic
program repair tools, e.g., a fault localization component that provides
spectrum-based fault localization information at different levels of
granularity, a configurable and safe patch validation component that is 11+X
faster than vanilla testing, and a customizable post-processing component to
generate fix reports. A demo video of PRF is available at
https://bit.ly/3ehduSS.Comment: Proceedings of the 28th ACM Joint European Software Engineering
Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE
'20
Reflections on Software Failure Analysis
Failure studies are important in revealing the root causes, behaviors, and
life cycle of defects in software systems. These studies either focus on
understanding the characteristics of defects in specific classes of systems or
the characteristics of a specific type of defect in the systems it manifests
in. Failure studies have influenced various software engineering research
directions, especially in the area of software evolution, defect detection, and
program repair.
In this paper, we reflect on the conduct of failure studies in software
engineering. We reviewed a sample of 52 failure study papers. We identified
several recurring problems in these studies, some of which hinder the ability
of the engineering community to trust or replicate the results. Based on our
findings, we suggest future research directions, including identifying and
analyzing failure causal chains, standardizing the conduct of failure studies,
and tool support for faster defect analysis.Comment: 6 pages, 4 figures To be published in: Proceedings of the 30th ACM
Joint European Software Engineering Conference and Symposium on the
Foundations of Software Engineering (ESEC/FSE '22
Neural-Augmented Static Analysis of Android Communication
We address the problem of discovering communication links between
applications in the popular Android mobile operating system, an important
problem for security and privacy in Android. Any scalable static analysis in
this complex setting is bound to produce an excessive amount of
false-positives, rendering it impractical. To improve precision, we propose to
augment static analysis with a trained neural-network model that estimates the
probability that a communication link truly exists. We describe a
neural-network architecture that encodes abstractions of communicating objects
in two applications and estimates the probability with which a link indeed
exists. At the heart of our architecture are type-directed encoders (TDE), a
general framework for elegantly constructing encoders of a compound data type
by recursively composing encoders for its constituent types. We evaluate our
approach on a large corpus of Android applications, and demonstrate that it
achieves very high accuracy. Further, we conduct thorough interpretability
studies to understand the internals of the learned neural networks.Comment: Appears in Proceedings of the 2018 ACM Joint European Software
Engineering Conference and Symposium on the Foundations of Software
Engineering (ESEC/FSE
Assessing the Quality of the Steps to Reproduce in Bug Reports
A major problem with user-written bug reports, indicated by developers and
documented by researchers, is the (lack of high) quality of the reported steps
to reproduce the bugs. Low-quality steps to reproduce lead to excessive manual
effort spent on bug triage and resolution. This paper proposes Euler, an
approach that automatically identifies and assesses the quality of the steps to
reproduce in a bug report, providing feedback to the reporters, which they can
use to improve the bug report. The feedback provided by Euler was assessed by
external evaluators and the results indicate that Euler correctly identified
98% of the existing steps to reproduce and 58% of the missing ones, while 73%
of its quality annotations are correct.Comment: In Proceedings of the 27th ACM Joint European Software Engineering
Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE
'19), August 26-30, 2019, Tallinn, Estoni
On the Relationship between Code Verifiability and Understandability
Proponents of software verification have argued that simpler code is easier
to verify: that is, that verification tools issue fewer false positives and
require less human intervention when analyzing simpler code. We empirically
validate this assumption by comparing the number of warnings produced by four
state-of-the-art verification tools on 211 snippets of Java code with 20
metrics of code comprehensibility from human subjects in six prior studies. Our
experiments, based on a statistical (meta-)analysis, show that, in aggregate,
there is a small correlation (r = 0.23) between understandability and
verifiability. The results support the claim that easy-to-verify code is often
easier to understand than code that requires more effort to verify. Our work
has implications for the users and designers of verification tools and for
future attempts to automatically measure code comprehensibility: verification
tools may have ancillary benefits to understandability, and measuring
understandability may require reasoning about semantic, not just syntactic,
code properties.Comment: to appear at Proceedings of the 31st ACM Joint European Software
Engineering Conference and Symposium on the Foundations of Software
Engineering (ESEC/FSE'23
- …