Redefining and Evaluating Coverage Criteria Based on the Testing Scope

Test coverage information can help testers in deciding when to stop testing and in augmenting their test suites when the measured coverage is not deemed sufficient. Since the notion of a test criterion was introduced in the 70’s, research on coverage testing has been very active with much effort dedicated to the definition of new, more cost-effective, coverage criteria or to the adaptation of existing ones to a different domain. All these studies share the premise that after defining the entity to be covered (e.g., branches), one cannot consider a program to be adequately tested if some of its entities have never been exercised by any input data. However, it is not the case that all entities are of interest in every context. This is particularly true for several paradigms that emerged in the last decade (e.g., component-based development, service-oriented architecture). In such cases, traditional coverage metrics might not always provide meaningful information. In this thesis we address such situation and we redefine coverage criteria so to focus on the program parts that are relevant to the testing scope. We instantiate this general notion of scope-based coverage by introducing three coverage criteria and we demonstrate how they could be applied to different testing contexts. When applied to the context of software reuse, our approach proved to be useful for supporting test case prioritization, selection and minimization. Our studies showed that for prioritization we can improve the average rate of faults detected. For test case selection and minimization, we can considerably reduce the test suite size with small to no extra impact on fault detection effectiveness. When the source code is not available, such as in the service-oriented architecture paradigm, we propose an approach that customizes coverage, measured on invocations at service interface, based on data from similar users. We applied this approach to a real world application and, in our study, we were able to predict the entities that would be of interest for a given user with high precision. Finally, we introduce the first of its kind coverage criterion for operational profile based testing that exploits program spectra obtained from usage traces. Our study showed that it is better correlated than traditional coverage with the probability that the next test input will fail, which implies that our approach can provide a better stopping rule. Promising results were also observed for test case selection. Our redefinition of coverage criteria approaches the topic of coverage testing from a completely different angle. Such a novel perspective paves the way for new avenues of research towards improving the cost-effectiveness of testing, yet all to be explored

How does the introduction of ‘choice’ affect the pooling of risks in European welfare states? The case of long-term care

Ongoing reforms of European welfare states that aim at increasing ‘choice’ for patients, clients, and beneficiaries provide a unique opportunity to explore what exactly drives these reforms and how they reconstitute communities of economic risks. Traditionally, the solidaristic bargain underpinning European welfare states revolved around the twin objectives of a redistribution of resources and a pooling of risks (Baldwin, 1990). Much of the retrenchment literature to date has focused on the income distributive effects of dwindling resources to explain changes in European welfare states. We postulate that more profound changes in welfare arrangements are being driven by the introduction of ‘consumer choice’ which is compatible with welfare expansion of new welfare state pillars. Our case study on long term care explores in particular, what choices users get and whether this allow us to infer the thrust behind ‘choice’ reforms as well as the effects on the pooling of risks in European welfare states. We find that welfare state expansion in long-term care has responded to growing demand and that there is a great variety in the cost-sharing arrangements which cannot all be subsumed under the imperative of cost containment

Automatic instantiation of abstract tests on specific configurations for large critical control systems

Computer-based control systems have grown in size, complexity, distribution and criticality. In this paper a methodology is presented to perform an abstract testing of such large control systems in an efficient way: an abstract test is specified directly from system functional requirements and has to be instantiated in more test runs to cover a specific configuration, comprising any number of control entities (sensors, actuators and logic processes). Such a process is usually performed by hand for each installation of the control system, requiring a considerable time effort and being an error prone verification activity. To automate a safe passage from abstract tests, related to the so called generic software application, to any specific installation, an algorithm is provided, starting from a reference architecture and a state-based behavioural model of the control software. The presented approach has been applied to a railway interlocking system, demonstrating its feasibility and effectiveness in several years of testing experience

Introducing a Romanian Frequency List and the Romanian Vocabulary Levels Test

Vocabulary is considered essential to language learning, thus English word lists and tests based on frequency information have become the centre of attention for researchers, teachers and learners alike. As a result, it is argued hereby that frequency based word lists and tests should be adapted and regarded as key elements for teaching and learning Romanian as an additional language as well. Since there are currently no reliable frequency lists and lexical tests in Romanian, this paper aims to bridge this gap by introducing the first Romanian Word List and the Romanian Vocabulary Levels Test. The list contains the 10,000 most frequent Romanian words and is based on the Romanian Balanced Annotated Corpus (ROMBAC, Ion, Irimia, Ștefănescu, Tufiș 2012). The primary objective of the paper is to elaborate on the compilation criteria, the challenges involved and the benefits of such a list in the case of teaching, learning and curriculum design for Romanian as an additional language. The secondary objective is to present a practical application of the word list by introducing an exemplary Romanian lexical test, the Romanian Vocabulary Levels Test and examine its reliability and validity

Model-Based Security Testing

Security testing aims at validating software system requirements related to security properties like confidentiality, integrity, authentication, authorization, availability, and non-repudiation. Although security testing techniques are available for many years, there has been little approaches that allow for specification of test cases at a higher level of abstraction, for enabling guidance on test identification and specification as well as for automated test generation. Model-based security testing (MBST) is a relatively new field and especially dedicated to the systematic and efficient specification and documentation of security test objectives, security test cases and test suites, as well as to their automated or semi-automated generation. In particular, the combination of security modelling and test generation approaches is still a challenge in research and of high interest for industrial applications. MBST includes e.g. security functional testing, model-based fuzzing, risk- and threat-oriented testing, and the usage of security test patterns. This paper provides a survey on MBST techniques and the related models as well as samples of new methods and tools that are under development in the European ITEA2-project DIAMONDS.Comment: In Proceedings MBT 2012, arXiv:1202.582

Consumer Cooperatives: An Alternative Institutional Model for Delivery of Urban Water Supply and Sanitation Services?

This paper describes the essential characteristics of consumer cooperatives engaged in the provision of basic services and discusses their applicability as a model for water supply and sanitation service provision in urban areas. A cooperative is an autonomous association of persons united voluntarily to meet their common economic, social, and cultural needs and aspirations through a jointly-owned and democratically-controlled enterprise. The paper focuses on system-wide urban water supply cooperatives and it is thus not concerned with urban or peri-urban cooperatives that depend on either boreholes or bulk purchases of water from a utility for distribution, nor does it refer to rural water supply cooperatives that are generally small. After a general overview of cooperatives and a discussion of the main characteristics of utility cooperatives, the paper reviews the case of SAGUAPAC, a successful urban WSS cooperative in Bolivia, from which it draws some conclusions in the form of a preliminary assessment of cooperatives as a model for delivery of urban water supply and sanitation (WSS) services

Q&A Platforms Evaluated Using Butler University Q&A Intelligence Index

A new study using the Butler University Q&A Intelligence Index measures how various mobile Q&A platforms deliver quality, accurate answers in a timely manner to a broad variety of questions. Based on the results of our analysis, ChaCha led all Q&A platforms on mobile devices. Results of the study are based upon review of a large set of responses from each of the major Q&A platforms, coupled with a comparison of disparate Q&A platforms that serve answers in different ways. Our methodology included the creation of a new metric, termed the Butler University Q&A Intelligence Index, which measures the likelihood that a user can expect to receive a correct answer in a timely manner to any random question asked using natural language. We asked questions via mobile services and randomized the questions to cover both popular and long-tail knowledge requests