Redefining and Evaluating Coverage Criteria Based on the Testing Scope

Test coverage information can help testers in deciding when to stop testing and in augmenting their test suites when the measured coverage is not deemed sufficient. Since the notion of a test criterion was introduced in the 70’s, research on coverage testing has been very active with much effort dedicated to the definition of new, more cost-effective, coverage criteria or to the adaptation of existing ones to a different domain. All these studies share the premise that after defining the entity to be covered (e.g., branches), one cannot consider a program to be adequately tested if some of its entities have never been exercised by any input data. However, it is not the case that all entities are of interest in every context. This is particularly true for several paradigms that emerged in the last decade (e.g., component-based development, service-oriented architecture). In such cases, traditional coverage metrics might not always provide meaningful information. In this thesis we address such situation and we redefine coverage criteria so to focus on the program parts that are relevant to the testing scope. We instantiate this general notion of scope-based coverage by introducing three coverage criteria and we demonstrate how they could be applied to different testing contexts. When applied to the context of software reuse, our approach proved to be useful for supporting test case prioritization, selection and minimization. Our studies showed that for prioritization we can improve the average rate of faults detected. For test case selection and minimization, we can considerably reduce the test suite size with small to no extra impact on fault detection effectiveness. When the source code is not available, such as in the service-oriented architecture paradigm, we propose an approach that customizes coverage, measured on invocations at service interface, based on data from similar users. We applied this approach to a real world application and, in our study, we were able to predict the entities that would be of interest for a given user with high precision. Finally, we introduce the first of its kind coverage criterion for operational profile based testing that exploits program spectra obtained from usage traces. Our study showed that it is better correlated than traditional coverage with the probability that the next test input will fail, which implies that our approach can provide a better stopping rule. Promising results were also observed for test case selection. Our redefinition of coverage criteria approaches the topic of coverage testing from a completely different angle. Such a novel perspective paves the way for new avenues of research towards improving the cost-effectiveness of testing, yet all to be explored

Risk-based supervision of pension funds in Australia

This paper examines the development of risk-based supervision of pension funds in Australia. The large number of pension funds has meant that since the inception of pension fund supervision in the early 1990's the regulator has sought to identify high risk funds and focus its attention on these funds. However, the regulator developed a more sophisticated risk-rating model, known as PAIRS/SOARS, in 1992 in order to apply a more disciplined and consistent ratings methodology. Four reasons are given for the move towards more sophisticated risk-based supervision: 1) creation of an integrated supervisor which allowed the use of techniques used in banking and insurance to be adopted for pension fund; 2) the need to better use available supervisory resources; 3) several pension fund failures; and 4) concerns about industry weaknesses. Supervisory techniques used particularly in the banking industry, such as universal licensing,'fit and proper'assessment, and risk management requirements were adopted for the pension sector between 2004 and 2006. The paper provides an outline of the PAIRS/SOARS risk-rating model which was also adopted. It observes that the approach provides an analytical discipline to risk assessment, strengthens the link between risk assessment and supervisory response, and allows better targeting of supervisory resources.Debt Markets,,Non Bank Financial Institutions,Labor Policies,Emerging Markets

Towards quality programming in the automated testing of distributed applications

PhD ThesisSoftware testing is a very time-consuming and tedious activity and accounts for over 25% of the cost of software development. In addition to its high cost, manual testing is unpopular and often inconsistently executed. Software Testing Environments (STEs) overcome the deficiencies of manual testing through automating the test process and integrating testing tools to support a wide range of test capabilities. Most prior work on testing is in single-thread applications. This thesis is a contribution to testing of distributed applications, which has not been well explored. To address two crucial issues in testing, when to stop testing and how good the software is after testing, a statistics-based integrated test environment which is an extension of the testing concept in Quality Programming for distributed applications is presented. It provides automatic support for test execution by the Test Driver, test development by the SMAD Tree Editor and the Test Data Generator, test failure analysis by the Test Results Validator and the Test Paths Tracer, test measurement by the Quality Analyst, test management by the Test Manager and test planning by the Modeller. These tools are integrated around a public, shared data model describing the data entities and relationships which are manipulable by these tools. It enables early entry of the test process into the life cycle due to the definition of the quality planning and message-flow routings in the modelling. After well-prepared modelling and requirements specification are undertaken, the test process and the software design and implementation can proceed concurrently. A simple banking application written using Java Remote Method Invocation (RMI) and Java DataBase Connectivity (JDBC) shows the testing process of fitting it into the integrated test environment. The concept of the automated test execution through mobile agents across multiple platforms is also illustrated on this 3-tier client/server application.The National Science Council, Taiwan: The Ministry of National Defense, Taiwan

Survey of users of earth resources remote sensing data

A user survey was conducted to determine current earth resources survey (ERS) data use/user status and recommendations for strengthening use. Only high-altitude aircraft and satellite (primarily LANDSAT) data were included. Emphasis was placed on the private sector/industrial user. Objectives of the survey included: who is using ERS data, how they are using the data, the relative value of current data use as well as obtaining user views as to possible ways of strengthening future ERS data use. The survey results are documented and should provide relevant decision making information for developing future programs of maximum benefit to all end users of satellite ERS data

Studies On The Potential Impacts Of The New Basel Capital Accord

In April 2003, the Basel Committee on Banking Supervision published the third consultative paper (CP3) of the new Basel Capital Accord relating to the prudential regulation of banks, which was followed in July 2003 by the EU Commission’s draft directive with the same contents, but slightly different detailed rules (Capital Adequacy Directive, CAD3). During the consultative process both organisations expect comments from the players affected by the new capital regulation, thus from the central banks of each country as well. The significance of the new capital regulation is underlined by the fact that the Basel recommendation will soon be followed by the European Union’s directive (presumably in 2004), the implementation of which will be one of the largest regulative challenges for Hungary. Accordingly, the Magyar Nemzeti Bank pays special attention to preparing the implementation of the Basel II/CAD3 capital accords, laying the groundwork for the adaptation and carrying out the necessary background analyses. Our main objective in the first phase of this rather complex and far-reaching project was – through participation in the legislative process – to analyse the issues important and relevant for the MNB, as well as to assess the potential consequences of implementation in Hungary. During such analyses we focused on the macro-prudential consequences. Accordingly, we carried out a detailed assessment of five topics.Basel Capital Accord, Pro-cyclicality, Credit risk, Market Risk, Regulation, Corporate governance.

A Review of Software Reliability Testing Techniques

In the era of intelligent systems, the safety and reliability of software have received more attention. Software reliability testing is a significant method to ensure reliability, safety and quality of software. The intelligent software technology has not only offered new opportunities but also posed challenges to software reliability technology. The focus of this paper is to explore the software reliability testing technology under the impact of intelligent software technology. In this study, the basic theories of traditional software and intelligent software reliability testing were investigated via related previous works, and a general software reliability testing framework was established. Then, the technologies of software reliability testing were analyzed, including reliability modeling, test case generation, reliability evaluation, testing criteria and testing methods. Finally, the challenges and opportunities of software reliability testing technology were discussed at the end of this paper

Best Practices for Evaluating Flight Deck Interfaces for Transport Category Aircraft with Particular Relevance to Issues of Attention, Awareness, and Understanding CAST SE-210 Output 2 Report 6 of 6

Attention, awareness, and understanding of the flight crew are a critical contributor to safety and the flight deck plays a critical role in supporting these cognitive functions. Changes to the flight deck need to be evaluated for whether the changed device provides adequate support for these functions. This report describes a set of diverse evaluation methods. The report recommends designing the interface-evaluation to span the phases of the device development, from early to late, and it provides methods appropriate at each phase. It describes the various ways in which an interface or interface component can fail to support awareness as potential issues to be assessed in evaluation. It summarizes appropriate methods to evaluate different issues concerning inadequate support for these functions, throughout the phases of development

Enhancing coverage adequacy of service compositions after runtime adaptation

Laufzeitüberwachung (engl. runtime monitoring) ist eine wichtige Qualitätssicherungs-Technik für selbstadaptive Service-Komposition. Laufzeitüberwachung überwacht den Betrieb der Service-Komposition. Zur Bestimmung der Genauigkeit von Software-Tests werden häufig Überdeckungskriterien verwendet. Überdeckungskriterien definieren Anforderungen die Software-Tests erfüllen muss. Wegen ihrer wichtigen Rolle im Software-Testen haben Forscher Überdeckungskriterien an die Laufzeitüberwachung von Service-Komposition angepasst. Die passive Art der Laufzeitüberwachung und die adaptive Art der Service-Komposition können die Genauigkeit von Software-Tests zur Laufzeit negativ beeinflussen. Dies kann jedoch die Zuversicht in der Qualität der Service-Komposition begrenzen. Um die Überdeckung selbstadaptiver Service-Komposition zur Laufzeit zu verbessern, untersucht diese Arbeit, wie die Laufzeitüberwachung und Online-Testen kombiniert werden können. Online-Testen bedeutet dass Testen parallel zu der Verwendung einer Service-Komposition erfolgt. Zunächst stellen wir einen Ansatz vor, um gültige Execution-Traces für Service-Komposition zur Laufzeit zu bestimmen. Der Ansatz berücksichtigt die Execution-Traces von Laufzeitüberwachung und (Online)-Testen. Er berücksichtigt Änderungen im Workflow und Software-Services eines Service-Komposition. Zweitens, definieren wir Überdeckungskriterien für Service-Komposition. Die Überdeckungskriterien berücksichtigen Ausführungspläne einer Service-Komposition und berücksichtigen die Überdeckung für Software-Services und die Service-Komposition. Drittens stellen wir Online-Testfälle Priorisierungs Techniken, um die Abdeckungniveau einer Service-Komposition schneller zu erreichen. Die Techniken berücksichtigen die Überdeckung einer Service-Komposition durch beide Laufzeitüberwachung und Online-Tests. Zusätzlich, berücksichtigen sie die Ausführungszeit von Testfällen und das Nutzungsmodell der Service-Komposition. Viertens stellen wir einen Rahmen für die Laufzeitüberwachung und Online-Testen von Software-Services und Service-Komposition, genannt PROSA, vor. PROSA bietet technische Unterstützung für die oben genannten Beiträge. Wir evaluieren die Beiträge anhand einer beispielhaften Service-Komposition, die häufig in dem Forschungsgebiet Service-oriented Computing eingesetzt wird.Runtime monitoring (or monitoring for short) is a key quality assurance technique for self-adaptive service compositions. Monitoring passively observes the runtime behaviour of service compositions. Coverage criteria are extensively used for assessing the adequacy (or thoroughness) of software testing. Coverage criteria specify certain requirements on software testing. The importance of coverage criteria in software testing has motivated researchers to adapt them to the monitoring of service composition. However, the passive nature of monitoring and the adaptive nature of service composition could negatively influence the adequacy of monitoring, thereby limiting the confidence in the quality of the service composition. To enhance coverage adequacy of self-adaptive service compositions at runtime, this thesis investigates how to combine runtime monitoring and online testing. Online testing means testing a service composition in parallel to its actual usage and operation. First, we introduce an approach for determining valid execution traces for service compositions at runtime. The approach considers execution traces of both monitoring and (online) testing. It considers modifications in both workflow and constituent services of a service composition. Second, we define coverage criteria for service compositions. The criteria consider execution plans of a service composition for coverage assessment and consider the coverage of an abstract service and the overall service composition. Third, we introduce online-test-case prioritization techniques to achieve a faster coverage of a service composition. The techniques employ coverage of a service composition from both monitoring and online testing, execution time of test cases, and the usage model of the service composition. Fourth, we introduce a framework for monitoring and online testing of services and service compositions called PROSA. PROSA provides technical support for the aforementioned contributions. We evaluate the contributions of this thesis using service compositions frequently used in service-oriented computing research