44 research outputs found
Evaluating Maintainability Prejudices with a Large-Scale Study of Open-Source Projects
Exaggeration or context changes can render maintainability experience into
prejudice. For example, JavaScript is often seen as least elegant language and
hence of lowest maintainability. Such prejudice should not guide decisions
without prior empirical validation. We formulated 10 hypotheses about
maintainability based on prejudices and test them in a large set of open-source
projects (6,897 GitHub repositories, 402 million lines, 5 programming
languages). We operationalize maintainability with five static analysis
metrics. We found that JavaScript code is not worse than other code, Java code
shows higher maintainability than C# code and C code has longer methods than
other code. The quality of interface documentation is better in Java code than
in other code. Code developed by teams is not of higher and large code bases
not of lower maintainability. Projects with high maintainability are not more
popular or more often forked. Overall, most hypotheses are not supported by
open-source data.Comment: 20 page
A methodology for validating cloud models using metamorphic testing
Cloud computing is a paradigm that provides access to a flexible, elastic and on-demand computing infrastructure, allowing users to dynamically request virtual resources. However, researchers typically cannot experiment with critical parts of cloud systems such as the underlying cloud architecture, resource-provisioning policies and the configuration of resource virtualisation. This problem can be partially addressed through using simulations of cloud systems. Unfortunately, the problem of testing cloud systems is still challenging due to the many parameters that such systems typically have and the difficulty in determining whether an observed behaviour is correct. In order to alleviate these issues, we propose a methodology to semi-automatically test and validate cloud models by integrating simulation techniques and metamorphic testing.This research was partially supported by the Spanish MEC projects TESIS (TIN2009-14312-C02-01) and ESTuDIo (TIN2012-36812-C02-01)
Refinement type contracts for verification of scientific investigative software
Our scientific knowledge is increasingly built on software output. User code
which defines data analysis pipelines and computational models is essential for
research in the natural and social sciences, but little is known about how to
ensure its correctness. The structure of this code and the development process
used to build it limit the utility of traditional testing methodology. Formal
methods for software verification have seen great success in ensuring code
correctness but generally require more specialized training, development time,
and funding than is available in the natural and social sciences. Here, we
present a Python library which uses lightweight formal methods to provide
correctness guarantees without the need for specialized knowledge or
substantial time investment. Our package provides runtime verification of
function entry and exit condition contracts using refinement types. It allows
checking hyperproperties within contracts and offers automated test case
generation to supplement online checking. We co-developed our tool with a
medium-sized (3000 LOC) software package which simulates
decision-making in cognitive neuroscience. In addition to helping us locate
trivial bugs earlier on in the development cycle, our tool was able to locate
four bugs which may have been difficult to find using traditional testing
methods. It was also able to find bugs in user code which did not contain
contracts or refinement type annotations. This demonstrates how formal methods
can be used to verify the correctness of scientific software which is difficult
to test with mainstream approaches
A partial oracle for uniformity statistics
This paper investigates the problem of testing implementations of uniformity statistics. In this paper we used Metamorphic Testing to address the oracle problem, of checking the output of one or more test executions, for uniformity statistics. We defined a partial oracle that uses regression analysis (a Regression Model based Metamorphic Relation).
We investigated the effectiveness of our partial oracle. We found that the technique can achieve mutation scores ranging from 77.78% to 100%, and tends towards higher mutation scores in this range. These results are promising, and suggest that the Regression Model based Metamorphic Relation approach is a viable method of alleviating the oracle problem in implementations of uniformity statistics, and potentially other classes of statistics e.g. correlation statistics
Data mining for software engineering and humans in the loop
The field of data mining for software engineering has been growing over the last decade. This field is concerned with the use of data mining to provide useful insights into how to improve software engineering processes and software itself, supporting decision-making. For that, data produced by software engineering processes and products during and after software development are used. Despite promising results, there is frequently a lack of discussion on the role of software engineering practitioners amidst the data mining approaches. This makes adoption of data mining by software engineering practitioners difficult. Moreover, the fact that expertsâ knowledge is frequently ignored by data mining approaches, together with the lack of transparency of such approaches, can hinder the acceptability of data mining by software engineering practitioners. To overcome these problems, this position paper provides a discussion of the role of software engineering experts when adopting data mining approaches. It also argues that this role can be extended to increase expertsâ involvement in the process of building data mining models. We believe that such extended involvement is not only likely to increase software engineersâ acceptability of the resulting models, but also improve the models themselves. We also provide some recommendations aimed at increasing the success of experts involvement and model acceptability
Syntactic Complexity Metrics and the Readability of Programs in a Functional Computer Language
This article reports on the defintion and the measutement of the software complexity metrics of Halstead and McCabe for programs written in the functional programming language Miranda. An automated measurement of these metrics is described. In a case study, the correlation is established between the complexity metrics and the expert assessment of the readability of programs in Miranda, and compared with those for programs in Pascal