44 research outputs found
LittleDarwin: a Feature-Rich and Extensible Mutation Testing Framework for Large and Complex Java Systems
Mutation testing is a well-studied method for increasing the quality of a
test suite. We designed LittleDarwin as a mutation testing framework able to
cope with large and complex Java software systems, while still being easily
extensible with new experimental components. LittleDarwin addresses two
existing problems in the domain of mutation testing: having a tool able to work
within an industrial setting, and yet, be open to extension for cutting edge
techniques provided by academia. LittleDarwin already offers higher-order
mutation, null type mutants, mutant sampling, manual mutation, and mutant
subsumption analysis. There is no tool today available with all these features
that is able to work with typical industrial software systems.Comment: Pre-proceedings of the 7th IPM International Conference on
Fundamentals of Software Engineerin
Modelling Realistic User Behaviour in Information Systems Simulations as Fuzzing Aspects
In this paper we contend that the engineering of information systems is hampered by a paucity of tools to tractably model, simulate and predict the impact of realistic user behaviours on the emergent properties of the wider socio-technical system, evidenced by the plethora of case studies of system failure in the literature. We address this gap by presenting a novel approach that models ideal user behaviour as workflows, and introduces irregularities in that behaviour as aspects which fuzz the model. We demonstrate the success of this approach through a case study of software development workflows, showing that the introduction of realistic user behaviour to idealised workflows better simulates outcomes reported in the empirical software engineering literature
Spotting Trees with Few Leaves
We show two results related to the Hamiltonicity and -Path algorithms in
undirected graphs by Bj\"orklund [FOCS'10], and Bj\"orklund et al., [arXiv'10].
First, we demonstrate that the technique used can be generalized to finding
some -vertex tree with leaves in an -vertex undirected graph in
time. It can be applied as a subroutine to solve the
-Internal Spanning Tree (-IST) problem in
time using polynomial space, improving upon previous algorithms for this
problem. In particular, for the first time we break the natural barrier of
. Second, we show that the iterated random bipartition employed by
the algorithm can be improved whenever the host graph admits a vertex coloring
with few colors; it can be an ordinary proper vertex coloring, a fractional
vertex coloring, or a vector coloring. In effect, we show improved bounds for
-Path and Hamiltonicity in any graph of maximum degree
or with vector chromatic number at most 8
Do Null-Type Mutation Operators Help Prevent Null-Type Faults?
The null-type is a major source of faults in Java programs, and its overuse
has a severe impact on software maintenance. Unfortunately traditional mutation
testing operators do not cover null-type faults by default, hence cannot be
used as a preventive measure. We address this problem by designing four new
mutation operators which model null-type faults explicitly. We show how these
mutation operators are capable of revealing the missing tests, and we
demonstrate that these mutation operators are useful in practice. For the
latter, we analyze the test suites of 15 open-source projects to describe the
trade-offs related to the adoption of these operators to strengthen the test
suite
Accelerated Model Checking of Parametric Markov Chains
Parametric Markov chains occur quite naturally in various applications: they
can be used for a conservative analysis of probabilistic systems (no matter how
the parameter is chosen, the system works to specification); they can be used
to find optimal settings for a parameter; they can be used to visualise the
influence of system parameters; and they can be used to make it easy to adjust
the analysis for the case that parameters change. Unfortunately, these
advancements come at a cost: parametric model checking is---or rather
was---often slow. To make the analysis of parametric Markov models scale, we
need three ingredients: clever algorithms, the right data structure, and good
engineering. Clever algorithms are often the main (or sole) selling point; and
we face the trouble that this paper focuses on -- the latter ingredients to
efficient model checking. Consequently, our easiest claim to fame is in the
speed-up we have often realised when comparing to the state of the art
Reducing the Cost of Grammar-Based Testing Using Pattern Coverage
Part 2: Test Derivation MethodsInternational audienceIn grammar-based testing, context-free grammars may be used to generate relevant test inputs for language processors, or meta programs, such as programming language compilers, refactoring tools, and implementations of software quality metrics. This technique can be used to test these meta programs, but the amount of sentences, and syntax trees thereof, which needs to be generated to obtain reasonable coverage of the input language is exponential.Pattern matching is a programming language feature used often when writing meta programs. Pattern matching helps because it automates the frequently occurring task of detecting shapes in, and extracting information from syntax trees. However, meta programs which contain many patterns are difficult to test using only randomly generated sentences from grammar rules. The reason is that statistically it is uncommon to directly generate sentences which accidentally match the patterns in the code.To solve this problem, in this paper we extract information from the patterns in the code of meta programs to guide the sentence generation process. We introduce a new coverage criterion, called Pattern Coverage, which focuses on providing a test strategy to reduce the amount of test necessary cases, while covering the relevant parts of the meta program. An initial experimental evaluation is presented and the result is compared with traditional grammar-based testing
Practical Verification of Decision-Making in Agent-Based Autonomous Systems
We present a verification methodology for analysing the decision-making component in agent-based hybrid systems. Traditionally hybrid automata have been used to both implement and verify such systems, but hybrid automata based modelling, programming and verification techniques scale poorly as the complexity of discrete decision-making increases making them unattractive in situations where complex log- ical reasoning is required. In the programming of complex systems it has, therefore, become common to separate out logical decision-making into a separate, discrete, component. However, verification techniques have failed to keep pace with this devel- opment. We are exploring agent-based logical components and have developed a model checking technique for such components which can then be composed with a sepa- rate analysis of the continuous part of the hybrid system. Among other things this allows program model checkers to be used to verify the actual implementation of the decision-making in hybrid autonomous systems
Software Testing Techniques Revisited for OWL Ontologies
Ontologies are an essential component of semantic knowledge bases and applications, and nowadays they are used in a plethora of domains. Despite the maturity of ontology languages, support tools and engineering techniques, the testing and validation of ontologies is a field which still lacks consolidated approaches and tools. This paper attempts at partly bridging that gap, taking a first step towards the extension of some traditional software testing techniques to ontologies expressed in a widely-used format. Mutation testing and coverage testing, revisited in the light of the peculiar features of the ontology language and structure, can can assist in designing better test suites to validate them, and overall help in the engineering and refinement of ontologies and software based on them
Testing of aspect-oriented programs: difficulties and lessons learned based on theoretical and practical experience
Gaining certainty about uncertainty: Testing cyber-physical systems in the presence of uncertainties at the application level
A cyber-physical system (CPS) comprises several connected, embedded systems and is additionally equipped with sensors and actuators. Thus, CPSs can communicate with their cyber environment and measure and interact with their physical environment. Due to the complexity of their operational environment, assumptions the manufacturer have made may not hold in operation. During an unforeseen environmental situation, a CPS may expose behavior that negatively impactsits reliability. This may arise due to insufficiently considered environmental conditions during the design of a CPS, or – even worse – it is impossible to anticipate such conditions. In the U-Test project, we are developing a configurable search-based testing framework that exploits information from functional testing and from declarative descriptions of uncertainties. Itaims at revealing unintended behavior in the presence of uncertainties. This framework enables testing for different scenarios of uncertainty and thus, allows to achieve a certain coverage of those, and to find unknown uncertainty scenarios