141,663 research outputs found
DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems
Deep learning (DL) defines a new data-driven programming paradigm that
constructs the internal system logic of a crafted neuron network through a set
of training data. We have seen wide adoption of DL in many safety-critical
scenarios. However, a plethora of studies have shown that the state-of-the-art
DL systems suffer from various vulnerabilities which can lead to severe
consequences when applied to real-world applications. Currently, the testing
adequacy of a DL system is usually measured by the accuracy of test data.
Considering the limitation of accessible high quality test data, good accuracy
performance on test data can hardly provide confidence to the testing adequacy
and generality of DL systems. Unlike traditional software systems that have
clear and controllable logic and functionality, the lack of interpretability in
a DL system makes system analysis and defect detection difficult, which could
potentially hinder its real-world deployment. In this paper, we propose
DeepGauge, a set of multi-granularity testing criteria for DL systems, which
aims at rendering a multi-faceted portrayal of the testbed. The in-depth
evaluation of our proposed testing criteria is demonstrated on two well-known
datasets, five DL systems, and with four state-of-the-art adversarial attack
techniques against DL. The potential usefulness of DeepGauge sheds light on the
construction of more generic and robust DL systems.Comment: The 33rd IEEE/ACM International Conference on Automated Software
Engineering (ASE 2018
Metamodel Instance Generation: A systematic literature review
Modelling and thus metamodelling have become increasingly important in
Software Engineering through the use of Model Driven Engineering. In this paper
we present a systematic literature review of instance generation techniques for
metamodels, i.e. the process of automatically generating models from a given
metamodel. We start by presenting a set of research questions that our review
is intended to answer. We then identify the main topics that are related to
metamodel instance generation techniques, and use these to initiate our
literature search. This search resulted in the identification of 34 key papers
in the area, and each of these is reviewed here and discussed in detail. The
outcome is that we are able to identify a knowledge gap in this field, and we
offer suggestions as to some potential directions for future research.Comment: 25 page
Recommended from our members
A Static Verification Framework for Secure Peer-to-Peer Applications
In this paper we present a static verification framework to support the design and verification of secure peer-to-peer applications. The framework supports the specification, modeling, and analysis of security aspects together with the general characteristics of the system, during early stages of the development life-cycle. The approach avoids security issues to be taken into consideration as a separate layer that is added to the system as an afterthought by the use of security protocols. The main functionality supported by the framework are concerned with the modeling of the system together with its security aspects by using an extension of UML, modeling of abuse cases to represent scenarios of attackers and assist with the identification of properties to be verified, specification of properties to be verified in a graphical template language, verification of the models against the properties, and visualization of the results of the verification process
Specification Patterns for Robotic Missions
Mobile and general-purpose robots increasingly support our everyday life,
requiring dependable robotics control software. Creating such software mainly
amounts to implementing their complex behaviors known as missions. Recognizing
the need, a large number of domain-specific specification languages has been
proposed. These, in addition to traditional logical languages, allow the use of
formally specified missions for synthesis, verification, simulation, or guiding
the implementation. For instance, the logical language LTL is commonly used by
experts to specify missions, as an input for planners, which synthesize the
behavior a robot should have. Unfortunately, domain-specific languages are
usually tied to specific robot models, while logical languages such as LTL are
difficult to use by non-experts. We present a catalog of 22 mission
specification patterns for mobile robots, together with tooling for
instantiating, composing, and compiling the patterns to create mission
specifications. The patterns provide solutions for recurrent specification
problems, each of which detailing the usage intent, known uses, relationships
to other patterns, and---most importantly---a template mission specification in
temporal logic. Our tooling produces specifications expressed in the LTL and
CTL temporal logics to be used by planners, simulators, or model checkers. The
patterns originate from 245 realistic textual mission requirements extracted
from the robotics literature, and they are evaluated upon a total of 441
real-world mission requirements and 1251 mission specifications. Five of these
reflect scenarios we defined with two well-known industrial partners developing
human-size robots. We validated our patterns' correctness with simulators and
two real robots
Artificial table testing dynamically adaptive systems
Dynamically Adaptive Systems (DAS) are systems that modify their behavior and
structure in response to changes in their surrounding environment. Critical
mission systems increasingly incorporate adaptation and response to the
environment; examples include disaster relief and space exploration systems.
These systems can be decomposed in two parts: the adaptation policy that
specifies how the system must react according to the environmental changes and
the set of possible variants to reconfigure the system. A major challenge for
testing these systems is the combinatorial explosions of variants and
envi-ronment conditions to which the system must react. In this paper we focus
on testing the adaption policy and propose a strategy for the selection of
envi-ronmental variations that can reveal faults in the policy. Artificial
Shaking Table Testing (ASTT) is a strategy inspired by shaking table testing
(STT), a technique widely used in civil engineering to evaluate building's
structural re-sistance to seismic events. ASTT makes use of artificial
earthquakes that simu-late violent changes in the environmental conditions and
stresses the system adaptation capability. We model the generation of
artificial earthquakes as a search problem in which the goal is to optimize
different types of envi-ronmental variations
Optimizing compilation with preservation of structural code coverage metrics to support software testing
Code-coverage-based testing is a widely-used testing strategy with the aim of providing a meaningful decision criterion for the adequacy of a test suite. Code-coverage-based testing is also mandated for the development of safety-critical applications; for example, the DO178b document requires the application of the modified condition/decision coverage. One critical issue of code-coverage testing is that structural code coverage criteria are typically applied to source code whereas the generated machine code may result in a different code structure because of code optimizations performed by a compiler. In this work, we present the automatic calculation of coverage profiles describing which structural code-coverage criteria are preserved by which code optimization, independently of the concrete test suite. These coverage profiles allow to easily extend compilers with the feature of preserving any given code-coverage criteria by enabling only those code optimizations that preserve it. Furthermore, we describe the integration of these coverage profile into the compiler GCC. With these coverage profiles, we answer the question of how much code optimization is possible without compromising the error-detection likelihood of a given test suite. Experimental results conclude that the performance cost to achieve preservation of structural code coverage in GCC is rather low.Peer reviewedSubmitted Versio
Model-Based Testing of Safety Critical Real-Time Control Logic Software
The paper presents the experience of the authors in model based testing of
safety critical real-time control logic software. It describes specifics of the
corresponding industrial settings and discusses technical details of usage of
UniTESK model based testing technology in these settings. Finally, we discuss
possible future directions of safety critical software development processes
and a place of model based testing techniques in it.Comment: In Proceedings MBT 2012, arXiv:1202.582
- …