A Testability Analysis Framework for Non-Functional Properties

This paper presents background, the basic steps and an example for a testability analysis framework for non-functional properties

Introducing Energy Efficiency into SQALE

Energy Efficiency is becoming a key factor in software development, given the sharp growth of IT systems and their impact on worldwide energy consumption. We do believe that a quality process infrastructure should be able to consider the Energy Efficiency of a system since its early development: for this reason we propose to introduce Energy Efficiency into the existing quality models. We selected the SQALE model and we tailored it inserting Energy Efficiency as a sub-characteristic of efficiency. We also propose a set of six source code specific requirements for the Java language starting from guidelines currently suggested in the literature. We experienced two major challenges: the identification of measurable, automatically detectable requirements, and the lack of empirical validation on the guidelines currently present in the literature and in the industrial state of the practice as well. We describe an experiment plan to validate the six requirements and evaluate the impact of their violation on Energy Efficiency, which has been partially proved by preliminary results on C code. Having Energy Efficiency in a quality model and well verified code requirements to measure it, will enable a quality process that precisely assesses and monitors the impact of software on energy consumptio

A survey on software testability

Context: Software testability is the degree to which a software system or a unit under test supports its own testing. To predict and improve software testability, a large number of techniques and metrics have been proposed by both practitioners and researchers in the last several decades. Reviewing and getting an overview of the entire state-of-the-art and state-of-the-practice in this area is often challenging for a practitioner or a new researcher. Objective: Our objective is to summarize the body of knowledge in this area and to benefit the readers (both practitioners and researchers) in preparing, measuring and improving software testability. Method: To address the above need, the authors conducted a survey in the form of a systematic literature mapping (classification) to find out what we as a community know about this topic. After compiling an initial pool of 303 papers, and applying a set of inclusion/exclusion criteria, our final pool included 208 papers. Results: The area of software testability has been comprehensively studied by researchers and practitioners. Approaches for measurement of testability and improvement of testability are the most-frequently addressed in the papers. The two most often mentioned factors affecting testability are observability and controllability. Common ways to improve testability are testability transformation, improving observability, adding assertions, and improving controllability. Conclusion: This paper serves for both researchers and practitioners as an "index" to the vast body of knowledge in the area of testability. The results could help practitioners measure and improve software testability in their projects

Understanding Class-level Testability Through Dynamic Analysis

It is generally acknowledged that software testing is both challenging and time-consuming. Understanding the factors that may positively or negatively affect testing effort will point to possibilities for reducing this effort. Consequently there is a significant body of research that has investigated relationships between static code properties and testability. The work reported in this paper complements this body of research by providing an empirical evaluation of the degree of association between runtime properties and class-level testability in object-oriented (OO) systems. The motivation for the use of dynamic code properties comes from the success of such metrics in providing a more complete insight into the multiple dimensions of software quality. In particular, we investigate the potential relationships between the runtime characteristics of production code, represented by Dynamic Coupling and Key Classes, and internal class-level testability. Testability of a class is consider ed here at the level of unit tests and two different measures are used to characterise those unit tests. The selected measures relate to test scope and structure: one is intended to measure the unit test size, represented by test lines of code, and the other is designed to reflect the intended design, represented by the number of test cases. In this research we found that Dynamic Coupling and Key Classes have significant correlations with class-level testability measures. We therefore suggest that these properties could be used as indicators of class-level testability. These results enhance our current knowledge and should help researchers in the area to build on previous results regarding factors believed to be related to testability and testing. Our results should also benefit practitioners in future class testability planning and maintenance activities

IEEE Standard 1500 Compliance Verification for Embedded Cores

Core-based design and reuse are the two key elements for an efficient system-on-chip (SoC) development. Unfortunately, they also introduce new challenges in SoC testing, such as core test reuse and the need of a common test infrastructure working with cores originating from different vendors. The IEEE 1500 Standard for Embedded Core Testing addresses these issues by proposing a flexible hardware test wrapper architecture for embedded cores, together with a core test language (CTL) used to describe the implemented wrapper functionalities. Several intellectual property providers have already announced IEEE Standard 1500 compliance in both existing and future design blocks. In this paper, we address the problem of guaranteeing the compliance of a wrapper architecture and its CTL description to the IEEE Standard 1500. This step is mandatory to fully trust the wrapper functionalities in applying the test sequences to the core. We present a systematic methodology to build a verification framework for IEEE Standard 1500 compliant cores, allowing core providers and/or integrators to verify the compliance of their products (sold or purchased) to the standar

Real world evaluation of aspect-oriented software development : a thesis submitted in partial fulfilment of the requirements for the degree of Master of Science in Computer Science at Massey University, Palmerston North, New Zealand

Software development has improved over the past decade with the rise in the popularity of the Object-Oriented (OO) development approach. However, software projects continue to grow in complexity and continue to have alarmingly low rates of success. Aspect-Oriented Programming (AOP) is touted to be one solution to this software development problem. It shows promise of reducing programming complexity, making software more flexible and more amenable to change. The central concept introduced by AOP is the aspect. An aspect is used to modularise crosscutting concerns in a similar fashion to the way classes modularise business concerns. A crosscutting concern cannot be modularised in approaches such as OO because the code to realise the concern must be spread throughout the module (e.g. a tracing concent is implemented by adding code to every method in a system). AOP also introduces join points, pointcuts, and advice which are used with aspects to capture crosscutting concerns so they can be localised in a modular unit. OO took approximately 20 years to become a mainstream development approach. AOP was only invented in 1997. This project considers whether AOP is ready for commercial adoption. This requires analysis of the AOP implementations available, tool support, design processes, testing tools, standards, and support infrastructure. Only when AOP is evaluated across all these criteria can it be established whether it is ready to be used in commercial projects. Moreover, if companies are to invest time and money into adopting AOP, they must be aware of the benefits and risks associated with its adoption. This project attempts to quantify the potential benefits in adopting AOP, as well as identifying areas of risk. SolNet Solutions Ltd, an Information Technology (IT) company in Wellington, New Zealand, is used in this study as a target environment for integration of aspects into a commercial development process. SolNet is in the business of delivering large scale enterprise Java applications. To assist in this process they have developed a Common Services Architecture (CSA) containing components that can be reused to reduce risk and cost to clients. However, the CSA is complicated and SolNet have identified aspects as a potential solution to decrease the complexity. Aspects were found to bring substantial improvement to the Service Layer of SolNet. applications, including substantial reductions in complexity and size. This reduces the cost and time of development, as well as the risk associated with the projects. Moreover, the CSA was used in a more consistent fashion making the system easier to understand and maintain, and several crosscutting concerns were modularised as part of a reusable aspect library which could eventually form part of their CSA. It was found that AOP is approaching commercial readiness. However, more work is needed on defining standards for aspect languages and modelling of design elements. The current solutions in this area are commercially viable, but would greatly benefit from a standardised approach. Aspect systems can be difficult to test and the effect of the weaving process on Java serialisation requires further investigation

Two ways to Grid: the contribution of Open Grid Services Architecture (OGSA) mechanisms to service-centric and resource-centric lifecycles

Service Oriented Architectures (SOAs) support service lifecycle tasks, including Development, Deployment, Discovery and Use. We observe that there are two disparate ways to use Grid SOAs such as the Open Grid Services Architecture (OGSA) as exemplified in the Globus Toolkit (GT3/4). One is a traditional enterprise SOA use where end-user services are developed, deployed and resourced behind firewalls, for use by external consumers: a service-centric (or ‘first-order’) approach. The other supports end-user development, deployment, and resourcing of applications across organizations via the use of execution and resource management services: A Resource-centric (or ‘second-order’) approach. We analyze and compare the two approaches using a combination of empirical experiments and an architectural evaluation methodology (scenario, mechanism, and quality attributes) to reveal common and distinct strengths and weaknesses. The impact of potential improvements (which are likely to be manifested by GT4) is estimated, and opportunities for alternative architectures and technologies explored. We conclude by investigating if the two approaches can be converged or combined, and if they are compatible on shared resources