28,281 research outputs found
Scripted GUI Testing of Android Apps: A Study on Diffusion, Evolution and Fragility
Background. Evidence suggests that mobile applications are not thoroughly
tested as their desktop counterparts. In particular GUI testing is generally
limited. Like web-based applications, mobile apps suffer from GUI test
fragility, i.e. GUI test classes failing due to minor modifications in the GUI,
without the application functionalities being altered.
Aims. The objective of our study is to examine the diffusion of GUI testing
on Android, and the amount of changes required to keep test classes up to date,
and in particular the changes due to GUI test fragility. We define metrics to
characterize the modifications and evolution of test classes and test methods,
and proxies to estimate fragility-induced changes.
Method. To perform our experiments, we selected six widely used open-source
tools for scripted GUI testing of mobile applications previously described in
the literature. We have mined the repositories on GitHub that used those tools,
and computed our set of metrics.
Results. We found that none of the considered GUI testing frameworks achieved
a major diffusion among the open-source Android projects available on GitHub.
For projects with GUI tests, we found that test suites have to be modified
often, specifically 5\%-10\% of developers' modified LOCs belong to tests, and
that a relevant portion (60\% on average) of such modifications are induced by
fragility.
Conclusions. Fragility of GUI test classes constitute a relevant concern,
possibly being an obstacle for developers to adopt automated scripted GUI
tests. This first evaluation and measure of fragility of Android scripted GUI
testing can constitute a benchmark for developers, and the basis for the
definition of a taxonomy of fragility causes, and actionable guidelines to
mitigate the issue.Comment: PROMISE'17 Conference, Best Paper Awar
Maintenance of Automated Test Suites in Industry: An Empirical study on Visual GUI Testing
Context: Verification and validation (V&V) activities make up 20 to 50
percent of the total development costs of a software system in practice. Test
automation is proposed to lower these V&V costs but available research only
provides limited empirical data from industrial practice about the maintenance
costs of automated tests and what factors affect these costs. In particular,
these costs and factors are unknown for automated GUI-based testing.
Objective: This paper addresses this lack of knowledge through analysis of
the costs and factors associated with the maintenance of automated GUI-based
tests in industrial practice.
Method: An empirical study at two companies, Siemens and Saab, is reported
where interviews about, and empirical work with, Visual GUI Testing is
performed to acquire data about the technique's maintenance costs and
feasibility.
Results: 13 factors are observed that affect maintenance, e.g. tester
knowledge/experience and test case complexity. Further, statistical analysis
shows that developing new test scripts is costlier than maintenance but also
that frequent maintenance is less costly than infrequent, big bang maintenance.
In addition a cost model, based on previous work, is presented that estimates
the time to positive return on investment (ROI) of test automation compared to
manual testing.
Conclusions: It is concluded that test automation can lower overall software
development costs of a project whilst also having positive effects on software
quality. However, maintenance costs can still be considerable and the less time
a company currently spends on manual testing, the more time is required before
positive, economic, ROI is reached after automation
LittleDarwin: a Feature-Rich and Extensible Mutation Testing Framework for Large and Complex Java Systems
Mutation testing is a well-studied method for increasing the quality of a
test suite. We designed LittleDarwin as a mutation testing framework able to
cope with large and complex Java software systems, while still being easily
extensible with new experimental components. LittleDarwin addresses two
existing problems in the domain of mutation testing: having a tool able to work
within an industrial setting, and yet, be open to extension for cutting edge
techniques provided by academia. LittleDarwin already offers higher-order
mutation, null type mutants, mutant sampling, manual mutation, and mutant
subsumption analysis. There is no tool today available with all these features
that is able to work with typical industrial software systems.Comment: Pre-proceedings of the 7th IPM International Conference on
Fundamentals of Software Engineerin
Tailored Source Code Transformations to Synthesize Computationally Diverse Program Variants
The predictability of program execution provides attackers a rich source of
knowledge who can exploit it to spy or remotely control the program. Moving
target defense addresses this issue by constantly switching between many
diverse variants of a program, which reduces the certainty that an attacker can
have about the program execution. The effectiveness of this approach relies on
the availability of a large number of software variants that exhibit different
executions. However, current approaches rely on the natural diversity provided
by off-the-shelf components, which is very limited. In this paper, we explore
the automatic synthesis of large sets of program variants, called sosies.
Sosies provide the same expected functionality as the original program, while
exhibiting different executions. They are said to be computationally diverse.
This work addresses two objectives: comparing different transformations for
increasing the likelihood of sosie synthesis (densifying the search space for
sosies); demonstrating computation diversity in synthesized sosies. We
synthesized 30184 sosies in total, for 9 large, real-world, open source
applications. For all these programs we identified one type of program analysis
that systematically increases the density of sosies; we measured computation
diversity for sosies of 3 programs and found diversity in method calls or data
in more than 40% of sosies. This is a step towards controlled massive
unpredictability of software
Recommended from our members
The location of executive suites and business centers in the United States: an exploratory analysis
Increasingly, corporate occupiers seek more flexible ways of meeting their accommodation needs. One consequence of this process has been the growth of the executive suite, serviced office or business centre market. This paper, the final report of a research project funded by the Real Estate Research Institute, focuses upon the geographical distribution of business centers offering executive suites within the US. After a brief review of the development of the market, the paper examines the availability of data, provides basic descriptive statistics of the distribution of executive suites by state and by metropolitan statistical area and then attempts to model the distribution using demographic and socio-economic data at MSA level. The distribution reflects employment in key growth sectors and the position of the MSA in the urban hierarchy. An appendix presents a preliminary view of the global distribution of suites
- …