4,448 research outputs found
SMRL: A Metamorphic Security Testing Tool for Web Systems
We present a metamorphic testing tool that alleviates the oracle problem in security testing. The tool enables engineers to specify metamorphic relations that capture security properties of Web systems. It automatically tests Web systems to detect vulnerabilities based on those relations. We provide a domain-specific language accompanied by an Eclipse editor to facilitate the specification of metamorphic relations. The tool automatically collects the input data and transforms the metamorphic relations into executable Java code in order to automatically perform security testing based on the collected data. The tool has been successfully evaluated on a commercial system and a leading open source system (Jenkins). Demo video: https://youtu.be/9kx6u9LsGxs
Automated metamorphic testing on the analyses of feature models
Copyright © 2010 Elsevier B.V. All rights reserved.Context: A feature model (FM) represents the valid combinations of features in a domain. The automated extraction of information from FMs is a complex task that involves numerous analysis operations, techniques and tools. Current testing methods in this context are manual and rely on the ability of the tester to decide whether the output of an analysis is correct. However, this is acknowledged to be time-consuming, error-prone and in most cases infeasible due to the combinatorial complexity of the analyses, this is known as the oracle problem.Objective: In this paper, we propose using metamorphic testing to automate the generation of test data for feature model analysis tools overcoming the oracle problem. An automated test data generator is presented and evaluated to show the feasibility of our approach.Method: We present a set of relations (so-called metamorphic relations) between input FMs and the set of products they represent. Based on these relations and given a FM and its known set of products, a set of neighbouring FMs together with their corresponding set of products are automatically generated and used for testing multiple analyses. Complex FMs representing millions of products can be efficiently created by applying this process iteratively.Results: Our evaluation results using mutation testing and real faults reveal that most faults can be automatically detected within a few seconds. Two defects were found in FaMa and another two in SPLOT, two real tools for the automated analysis of feature models. Also, we show how our generator outperforms a related manual suite for the automated analysis of feature models and how this suite can be used to guide the automated generation of test cases obtaining important gains in efficiency.Conclusion: Our results show that the application of metamorphic testing in the domain of automated analysis of feature models is efficient and effective in detecting most faults in a few seconds without the need for a human oracle.This work has been partially supported by the European Commission(FEDER)and Spanish Government under CICYT project SETI(TIN2009-07366)and the Andalusian Government project ISABEL(TIC-2533)
Automatic Software Repair: a Bibliography
This article presents a survey on automatic software repair. Automatic
software repair consists of automatically finding a solution to software bugs
without human intervention. This article considers all kinds of repairs. First,
it discusses behavioral repair where test suites, contracts, models, and
crashing inputs are taken as oracle. Second, it discusses state repair, also
known as runtime repair or runtime recovery, with techniques such as checkpoint
and restart, reconfiguration, and invariant restoration. The uniqueness of this
article is that it spans the research communities that contribute to this body
of knowledge: software engineering, dependability, operating systems,
programming languages, and security. It provides a novel and structured
overview of the diversity of bug oracles and repair operators used in the
literature
Metamorphic testing for cybersecurity
Metamorphic testing (MT) can enhance security testing by providing an alternative to using a testing oracle, which is often unavailable or impractical. The authors report how MT detected previously unknown bugs in real-world critical applications such as code obfuscators, giving evidence that software testing requires diverse perspectives to achieve greater cybersecurity
Mapping the Structure and Evolution of Software Testing Research Over the Past Three Decades
Background: The field of software testing is growing and rapidly-evolving.
Aims: Based on keywords assigned to publications, we seek to identify
predominant research topics and understand how they are connected and have
evolved.
Method: We apply co-word analysis to map the topology of testing research as
a network where author-assigned keywords are connected by edges indicating
co-occurrence in publications. Keywords are clustered based on edge density and
frequency of connection. We examine the most popular keywords, summarize
clusters into high-level research topics, examine how topics connect, and
examine how the field is changing.
Results: Testing research can be divided into 16 high-level topics and 18
subtopics. Creation guidance, automated test generation, evolution and
maintenance, and test oracles have particularly strong connections to other
topics, highlighting their multidisciplinary nature. Emerging keywords relate
to web and mobile apps, machine learning, energy consumption, automated program
repair and test generation, while emerging connections have formed between web
apps, test oracles, and machine learning with many topics. Random and
requirements-based testing show potential decline.
Conclusions: Our observations, advice, and map data offer a deeper
understanding of the field and inspiration regarding challenges and connections
to explore.Comment: To appear, Journal of Systems and Softwar
Detecting Excessive Data Exposures in Web Server Responses with Metamorphic Fuzzing
APIs often transmit far more data to client applications than they need, and
in the context of web applications, often do so over public channels. This
issue, termed Excessive Data Exposure (EDE), was OWASP's third most significant
API vulnerability of 2019. However, there are few automated tools -- either in
research or industry -- to effectively find and remediate such issues. This is
unsurprising as the problem lacks an explicit test oracle: the vulnerability
does not manifest through explicit abnormal behaviours (e.g., program crashes
or memory access violations).
In this work, we develop a metamorphic relation to tackle that challenge and
build the first fuzzing tool -- that we call EDEFuzz -- to systematically
detect EDEs. EDEFuzz can significantly reduce false negatives that occur during
manual inspection and ad-hoc text-matching techniques, the current most-used
approaches.
We tested EDEFuzz against the sixty-nine applicable targets from the Alexa
Top-200 and found 33,365 potential leaks -- illustrating our tool's broad
applicability and scalability. In a more-tightly controlled experiment of eight
popular websites in Australia, EDEFuzz achieved a high true positive rate of
98.65% with minimal configuration, illustrating our tool's accuracy and
efficiency
- …