A Study on Software Testability and the Quality of Testing in Object-Oriented Systems

Software testing is known to be important to the delivery of high-quality systems, but it is also challenging, expensive and time-consuming. This has motivated academic and industrial researchers to seek ways to improve the testability of software. Software testability is the ease with which a software artefact can be effectively tested. The first step towards building testable software components is to understand the factors – of software processes, products and people – that are related to and can influence software testability. In particular, the goal of this thesis is to provide researchers and practitioners with a comprehensive understanding of design and source code factors that can affect the testability of a class in object oriented systems. This thesis considers three different views on software testability that address three related aspects: 1) the distribution of unit tests in relation to the dynamic coupling and centrality of software production classes, 2) the relationship between dynamic (i.e., runtime) software properties and class testability, and 3) the relationship between code smells, test smells and the factors related to smells distribution. The thesis utilises a combination of source code analysis techniques (both static and dynamic), software metrics, software visualisation techniques and graph-based metrics (from complex networks theory) to address its goals and objectives. A systematic mapping study was first conducted to thoroughly investigate the body of research on dynamic software metrics and to identify issues associated with their selection, design and implementation. This mapping study identified, evaluated and classified 62 research works based on a pre-tested protocol and a set of classification criteria. Based on the findings of this study, a number of dynamic metrics were selected and used in the experiments that were then conducted. The thesis demonstrates that by using a combination of visualisation, dynamic analysis, static analysis and graph-based metrics it is feasible to identify central classes and to diagrammatically depict testing coverage information. Experimental results show that, even in projects with high test coverage, some classes appear to be left without any direct unit testing, even though they play a central role during a typical execution profile. It is contended that the proposed visualisation techniques could be particularly helpful when developers need to maintain and reengineer existing test suites. Another important finding of this thesis is that frequently executed and tightly coupled classes are correlated with the testability of the class – such classes require larger unit tests and more test cases. This information could inform estimates of the effort required to test classes when developing new unit tests or when maintaining and refactoring existing tests. An additional key finding of this thesis is that test and code smells, in general, can have a negative impact on class testability. Increasing levels of size and complexity in code are associated with the increased presence of test smells. In addition, production classes that contain smells generally require larger unit tests, and are also likely to be associated with test smells in their associated unit tests. There are some particular smells that are more significantly associated with class testability than other smells. Furthermore, some particular code smells can be seen as a sign for the presence of test smells, as some test and code smells are found to co-occur in the test and production code. These results suggest that code smells, and specifically certain types of smells, as well as measures of size and complexity, can be used to provide a more comprehensive indication of smells likely to emerge in test code produced subsequently (or vice versa in a test-first context). Such findings should contribute positively to the work of testers and maintainers when writing unit tests and when refactoring and maintaining existing tests

Uso de patrones en el proceso de construcción de escenarios

En este trabajo se ha propuesto una heurística que incluye el uso de patrones en el proceso de construcción de escenarios. La formulación y aplicación de esta heurística permitió arribar a las siguientes conclusiones: Las regularidades existentes en los escenarios son independientes del problema, ya que se basan en la relación de los componentes del escenario y no en particularidades del dominio. Por otra parte, las regularidades son suficientemente nítidas como para ser utilizadas en la creación de patrones. El uso de un enfoque basado en patrones fabricó un punto de vista suficientemente preciso como para hacer visibles ambigüedades e imprecisiones en escenarios de muy buena calidad que habían sido objeto de procesos de certificación previos. La aplicación de la nueva heurística a casos de estudio preexistentes permitió obtener escenarios de mejor calidad que los obtenidos con anterioridad, especialmente en lo que se refiere al contexto temporal y a los episodios. Se observó que un porcentaje importante de escenarios de los casos de estudio adolecían del defecto de una mala definición del contexto temporal con sus consecuencias sobre los episodios. Una brecha temporal no es una cuestión menor, ya que significa la existencia de escenarios no descubiertos o parcialmente empotrados en otros, lo que naturalmente provoca una solución de software diferente de la que realmente se necesita Puede afirmarse que los patrones obtenidos son verosímiles ya que aproximadamente el 93% de los escenarios de los casos de estudio analizados pudieron ser reflejados en los mismos. Los patrones de escenarios permiten reusar información con un alto grado de abstracción, ya que constituyen una guía en un proceso bien determinado y claro de reuso. El uso de patrones en el proceso de construcción de escenarios enriquece las heurísticas existentes, ya que brinda orientación en el proceso de construcción y además ofrece una fuente de confirmación del escenario en desarrollo. De este modo, mejora la calidad de los escenarios producidos y se reduce el tiempo utilizado en escribir los escenarios. La selección del patrón no es un proceso intuitivo, sino que cuenta con un árbol de decisión que permite elegir el patrón, o la familia de patrones, con un alto grado de certeza. El segundo árbol de decisión permite seleccionar más específicamente el patrón dentro de la familia de patrones determinada por medio del primer árbol, y confirmar el escenario en desarrollo, contrastándolo con la estructura del patrón seleccionado.Facultad de Informátic