5,014 research outputs found
Reasoning and Improving on Software Resilience against Unanticipated Exceptions
In software, there are the errors anticipated at specification and design
time, those encountered at development and testing time, and those that happen
in production mode yet never anticipated. In this paper, we aim at reasoning on
the ability of software to correctly handle unanticipated exceptions. We
propose an algorithm, called short-circuit testing, which injects exceptions
during test suite execution so as to simulate unanticipated errors. This
algorithm collects data that is used as input for verifying two formal
exception contracts that capture two resilience properties. Our evaluation on 9
test suites, with 78% line coverage in average, analyzes 241 executed catch
blocks, shows that 101 of them expose resilience properties and that 84 can be
transformed to be more resilient
Dynamic Analysis can be Improved with Automatic Test Suite Refactoring
Context: Developers design test suites to automatically verify that software
meets its expected behaviors. Many dynamic analysis techniques are performed on
the exploitation of execution traces from test cases. However, in practice,
there is only one trace that results from the execution of one manually-written
test case.
Objective: In this paper, we propose a new technique of test suite
refactoring, called B-Refactoring. The idea behind B-Refactoring is to split a
test case into small test fragments, which cover a simpler part of the control
flow to provide better support for dynamic analysis.
Method: For a given dynamic analysis technique, our test suite refactoring
approach monitors the execution of test cases and identifies small test cases
without loss of the test ability. We apply B-Refactoring to assist two existing
analysis tasks: automatic repair of if-statements bugs and automatic analysis
of exception contracts.
Results: Experimental results show that test suite refactoring can
effectively simplify the execution traces of the test suite. Three real-world
bugs that could previously not be fixed with the original test suite are fixed
after applying B-Refactoring; meanwhile, exception contracts are better
verified via applying B-Refactoring to original test suites.
Conclusions: We conclude that applying B-Refactoring can effectively improve
the purity of test cases. Existing dynamic analysis tasks can be enhanced by
test suite refactoring
DSpot: Test Amplification for Automatic Assessment of Computational Diversity
Context: Computational diversity, i.e., the presence of a set of programs
that all perform compatible services but that exhibit behavioral differences
under certain conditions, is essential for fault tolerance and security.
Objective: We aim at proposing an approach for automatically assessing the
presence of computational diversity. In this work, computationally diverse
variants are defined as (i) sharing the same API, (ii) behaving the same
according to an input-output based specification (a test-suite) and (iii)
exhibiting observable differences when they run outside the specified input
space. Method: Our technique relies on test amplification. We propose source
code transformations on test cases to explore the input domain and
systematically sense the observation domain. We quantify computational
diversity as the dissimilarity between observations on inputs that are outside
the specified domain. Results: We run our experiments on 472 variants of 7
classes from open-source, large and thoroughly tested Java classes. Our test
amplification multiplies by ten the number of input points in the test suite
and is effective at detecting software diversity. Conclusion: The key insights
of this study are: the systematic exploration of the observable output space of
a class provides new insights about its degree of encapsulation; the behavioral
diversity that we observe originates from areas of the code that are
characterized by their flexibility (caching, checking, formatting, etc.).Comment: 12 page
Problems and Solutions in Mobile Application Testing
Mobiilirakenduste testimise alaste teadusartiklite arv on viimastel aastatel visalt suurenenud. Samas testivad vĂ€hesed mobiilirakendustega tegelevad teadlased oma oletusi ja lahendusi firmades. Selle lĂ”putöö eesmĂ€rgiks on pakkuda ĂŒlevaade teaduskirjanduses mainitud mobiilirakenduste testimisega seotud probleemidest ja potentsiaalsetest lahendustest ning kĂ”rvutada seda alal igapĂ€evaselt tegutsevate professionaalide arvamusega. KĂ”igepealt viiakse selle töö kĂ€igus lĂ€bi teaduskirjanduse uuring probleemide ja potentsiaalsete lahenduste vĂ€ljaselgitamiseks, misjĂ€rel intervjueeritakse kuue mobiilirakenduste testimisega tegeleva firma esindajaid, et vĂ€lja selgitada, kas kirjelduses esile toodud probleemid on olulised ka tööstuses. Intervjuude tulemusena selgus, et kuigi firmad hindavad probleemide tĂ€htsust vĂ€ga erinevalt, on siiski olemas mĂ”ned vĂ”tmeprobleemid, mida peetakse oluliseks nii teaduses kui ka tööstuses. Samas on teaduskirjanduses pakutud lahendused tihti liiga teoreetilised, ĂŒldised vĂ”i vananenud, et firmade esindajatele huvi pakkuda.In recent years the amount of scientific papers published on the topic of mobile applications has significantly increased. However, few researchers test their assumptions and solutions in industry. This thesis aims to provide an overview of what current scientific literature considers problems and potential solutions in mobile application testing, and compare it to opinions of industry professionals. A literature review is performed to extract the list of problems and potential solutions, after which representatives of six Estonian companies involved in the field are interviewed to verify whether the problems and solutions proposed in the literature are relevant for industry. The study reveals that while the relevance of each problem is highly variable from one company to another, there are some key problems that are generally considered vital both by research and industry. However, the solution concepts proposed by scientific literature are often too theoretical, general or outdated to be of much interest to industry professionals
Test case quality: an empirical study on belief and evidence
Software testing is a mandatory activity in any serious software development
process, as bugs are a reality in software development. This raises the
question of quality: good tests are effective in finding bugs, but until a test
case actually finds a bug, its effectiveness remains unknown. Therefore,
determining what constitutes a good or bad test is necessary. This is not a
simple task, and there are a number of studies that identify different
characteristics of a good test case. A previous study evaluated 29 hypotheses
regarding what constitutes a good test case, but the findings are based on
developers' beliefs, which are subjective and biased. In this paper we
investigate eight of these hypotheses, through an extensive empirical study
based on open software repositories. Despite our best efforts, we were unable
to find evidence that supports these beliefs. This indicates that, although
these hypotheses represent good software engineering advice, they do not
necessarily mean that they are enough to provide the desired outcome of good
testing code.Comment: 12 pages, 1 figure, 3 table
PerfBlower: Quickly Detecting Memory-Related Performance Problems via Amplification
Performance problems in managed languages are extremely difficult to find. Despite many efforts to find those problems, most existing work focuses on how to debug a user-provided test execution in which performance problems already manifest. It remains largely unknown how to effectively find performance bugs before software release. As a result, performance bugs often escape to production runs, hurting software reliability and user experience. This paper describes PerfBlower, a general performance testing framework that allows developers to quickly test Java programs to find memory-related performance problems. PerfBlower provides (1) a novel specification language ISL to describe a general class of performance problems that have observable symptoms; (2) an automated test oracle via emph{virtual amplification}; and (3) precise reference-path-based diagnostic information via object mirroring. Using this framework, we have amplified three different types of problems. Our experimental results demonstrate that (1) ISL is expressive enough to describe various memory-related performance problems; (2) PerfBlower successfully distinguishes executions with and without problems; 8 unknown problems are quickly discovered under small workloads; and (3) PerfBlower outperforms existing detectors and does not miss any bugs studied before in the literature
On Oracles for Automated Diagnosis and Repair of Software Bugs
This HDR focuses on my work on automatic diagnosis and repair done over the past years. Among my past publications, it highlights three contributions on this topic, respectively published in ACM Transactions on Software Engineering and Methodology (TOSEM), IEEE Transactions on Software Engineering (TSE) and Elsevier Information & Software Technology (IST). My goal is to show that those three contributions share something deep, that they are founded on a unifying concept, which is the one of oracle. The first contribution is about statistical oracles. In the context of object-oriented software, we have defined a notion of context and normality that is specific to a fault class: missing method calls. Those inferred regularities act as oracle and their violations are considered as bugs. The second contribution is about test case based oracles for automatic repair. We describe an automatic repair system that fixes failing test cases by generating a patch. It is founded on the idea of refining the knowledge given by the violation of the oracle of the failing test case into finer-grain information, which we call a âmicro-oracleâ. By considering micro-oracles, we are capable of obtaining at the same time a precise fault localization diagnostic and a well-formed input-output specification to be used for program synthesis in order to repair a bug. The third contribution discusses a novel generic oracle in the context of exception handling. A generic oracle states properties that hold for many domains. Our technique verifies the compliance to this new oracle using test suite execution and exception injection. This document concludes with a research agenda about the future of engineering ultra-dependable and antifragile software systems
Efficiently Manifesting Asynchronous Programming Errors in Android Apps
Android, the #1 mobile app framework, enforces the single-GUI-thread model,
in which a single UI thread manages GUI rendering and event dispatching. Due to
this model, it is vital to avoid blocking the UI thread for responsiveness. One
common practice is to offload long-running tasks into async threads. To achieve
this, Android provides various async programming constructs, and leaves
developers themselves to obey the rules implied by the model. However, as our
study reveals, more than 25% apps violate these rules and introduce
hard-to-detect, fail-stop errors, which we term as aysnc programming errors
(APEs). To this end, this paper introduces APEChecker, a technique to
automatically and efficiently manifest APEs. The key idea is to characterize
APEs as specific fault patterns, and synergistically combine static analysis
and dynamic UI exploration to detect and verify such errors. Among the 40
real-world Android apps, APEChecker unveils and processes 61 APEs, of which 51
are confirmed (83.6% hit rate). Specifically, APEChecker detects 3X more APEs
than the state-of-art testing tools (Monkey, Sapienz and Stoat), and reduces
testing time from half an hour to a few minutes. On a specific type of APEs,
APEChecker confirms 5X more errors than the data race detection tool,
EventRacer, with very few false alarms
Effective recognition of facial micro-expressions with video motion magnification
Facial expression recognition has been intensively studied for decades, notably by the psychology community and more recently the pattern recognition community. What is more challenging, and the subject of more recent research, is the problem of recognizing subtle emotions exhibited by so-called micro-expressions. Recognizing a micro-expression is substantially more challenging than conventional expression recognition because these micro-expressions are only temporally exhibited in a fraction of a second and involve minute spatial changes. Until now, work in this field is at a nascent stage, with only a few existing
micro-expression databases and methods. In this article, we propose a new micro-expression recognition approach based on the Eulerian motion magnification technique, which could reveal the hidden information and accentuate the subtle changes in micro-expression motion. Validation of our proposal was done on the recently proposed CASME II dataset in comparison with baseline and state-of-the-art methods. We achieve a good recognition accuracy of up to 75.30% by using leave-one-out cross validation evaluation protocol. Extensive experiments on various factors at play further demonstrate the effectiveness of our proposed approach
- âŠ