Automating Requirements Traceability: Two Decades of Learning from KDD

This paper summarizes our experience with using Knowledge Discovery in Data (KDD) methodology for automated requirements tracing, and discusses our insights.Comment: The work of the second author has been supported in part by NSF grants CCF-1511117 and CICI 1642134; 4 pages; in Proceedings of IEEE Requirements Engineering 201

What have we learnt from the challenges of (semi-) automated requirements traceability? A discussion on blockchain applicability.

Over the last 3 decades, researchers have attempted to shed light into the requirements traceability problem by introducing tracing tools, techniques, and methods with the vision of achieving ubiquitous traceability. Despite the technological advances, requirements traceability remains problematic for researchers and practitioners. This study aims to identify and investigate the main challenges in implementing (semi-)automated requirements traceability, as reported in the recent literature. A systematic literature review was carried out based on the guidelines for systematic literature reviews in software engineering, proposed by Kitchenham. We retrieved 4530 studies by searching five major bibliographic databases and selected 70 primary studies. These studies were analysed and classified according to the challenges they present and/or address. Twenty-one challenges were identified and were classified into five categories. Findings reveal that the most frequent challenges are technological challenges, in particular, low accuracy of traceability recovery methods. Findings also suggest that future research efforts should be devoted to the human facet of tracing, to explore traceability practices in organisational settings, and to develop traceability approaches that support agile and DevOps practices. Finally, it is recommended that researchers leverage blockchain technology as a suitable technical solution to ensure the trustworthiness of traceability information in interorganisational software projects.publishedVersio

Process improvement for traceability: A study of human fallibility

Abstract—Human analysts working with results from automated traceability tools often make incorrect decisions that lead to lower quality final trace matrices. As the human must vet the results of trace tools for mission- and safety-critical systems, the hopes of developing expedient and accurate tracing procedures lies in understanding how analysts work with trace matrices. This paper describes a study to understand when and why humans make correct and incorrect decisions during tracing tasks through logs of analyst actions. In addition to the traditional measures of recall and precision to describe the accuracy of the results, we introduce and study new measures that focus on analyst work quality: potential recall, sensitivity, and effort distribution. We use these measures to visualize analyst progress towards the final trace matrix, identifying factors that may influence their performance and determining how actual tracing strategies, derived from analyst logs, affect results

IMPROVING TRACEABILITY RECOVERY TECHNIQUES THROUGH THE STUDY OF TRACING METHODS AND ANALYST BEHAVIOR

Developing complex software systems often involves multiple stakeholder interactions, coupled with frequent requirements changes while operating under time constraints and budget pressures. Such conditions can lead to hidden problems, manifesting when software modifications lead to unexpected software component interactions that can cause catastrophic or fatal situations. A critical step in ensuring the success of software systems is to verify that all requirements can be traced to the design, source code, test cases, and any other software artifacts generated during the software development process. The focus of this research is to improve on the trace matrix generation process and study how human analysts create the final trace matrix using traceability information generated from automated methods. This dissertation presents new results in the automated generation of traceability matrices and in the analysis of analyst actions during a tracing task. The key contributions of this dissertation are as follows: (1) Development of a Proximity-based Vector Space Model for automated generation of TMs. (2) Use of Mean Average Precision (a ranked retrieval-based measure) and 21-point interpolated precision-recall graph (a set-based measure) for statistical evaluation of automated methods. (3) Logging and visualization of analyst actions during a tracing task. (4) Study of human analyst tracing behavior with consideration of decisions made during the tracing task and analyst tracing strategies. (5) Use of potential recall, sensitivity, and effort distribution as analyst performance measures. Results show that using both a ranked retrieval-based and a set-based measure with statistical rigor provides a framework for evaluating automated methods. Studying the human analyst provides insight into how analysts use traceability information to create the final trace matrix and identifies areas for improvement in the traceability process. Analyst performance measures can be used to identify analysts that perform the tracing task well and use effective tracing strategies to generate a high quality final trace matrix

Toward an Effective Automated Tracing Process

Traceability is defined as the ability to establish, record, and maintain dependency relations among various software artifacts in a software system, in both a forwards and backwards direction, throughout the multiple phases of the project’s life cycle. The availability of traceability information has been proven vital to several software engineering activities such as program comprehension, impact analysis, feature location, software reuse, and verification and validation (V&V). The research on automated software traceability has noticeably advanced in the past few years. Various methodologies and tools have been proposed in the literature to provide automatic support for establishing and maintaining traceability information in software systems. This movement is motivated by the increasing attention traceability has been receiving as a critical element of any rigorous software development process. However, despite these major advances, traceability implementation and use is still not pervasive in industry. In particular, traceability tools are still far from achieving performance levels that are adequate for practical applications. Such low levels of accuracy require software engineers working with traceability tools to spend a considerable amount of their time verifying the generated traceability information, a process that is often described as tedious, exhaustive, and error-prone. Motivated by these observations, and building upon a growing body of work in this area, in this dissertation we explore several research directions related to enhancing the performance of automated tracing tools and techniques. In particular, our work addresses several issues related to the various aspects of the IR-based automated tracing process, including trace link retrieval, performance enhancement, and the role of the human in the process. Our main objective is to achieve performance levels, in terms of accuracy, efficiency, and usability, that are adequate for practical applications, and ultimately to accomplish a successful technology transfer from research to industry

The safety of industrially-based controllers incorporating software

This thesis is concerned with the safety of industrial controllers which incorporate software. Software safety is compared with software reliability as a means of discussing the special concerns of safety. Definitions are given for the terms hazard, risk, danger and safe. A relationship between these terms has been attempted and the philosophy of safety is discussed. A formal definition of software safety is given. The factors influencing the development of software are examined. The subjectivity of safety is discussed in the context of safety measurement being a conjoint measurement. Methods of assessing the risk resulting from the use of software are described along with a discussion on the impracticability of using state transition diagrams to isolate catastrophic failure conditions. Categories of danger are discussed and three categories are advanced. The structuring of the software for safety is discussed and the principle of using safety modules and integrity locks is proposed. In discussing the reasons for errors remaining present in the software after testing two methods of measurement are suggested; Plexus and Fallibility Index. The need to declare variables is discussed. An experiment involving 119 volunteers was conducted to examine the influence of the length of variable names'on the correct usage. It was found that variables with a character length of 7 have a better probability of correct interpretation than others. The methods of assessing safety are discussed and the measurements proposed were applied to a commercially available product in the form of a Software Safety Audit. It is concluded that some aspects of the safety of controllers incorporating software can be quantified and that further research is needed

Experiences of countertransference in beginning psychotherapists.

Given the rise of intersubjective theory (Marzi, Hautman & Maestro, 2006) and contemporary formulations of countertransference as an integral aspect of the psychotherapeutic process (Cassorla, 2005; Marchon, 2006), understandings of countertransference are still being developed and explored in the psychoanalytic literature. This study explores beginning psychotherapists’ experiences and understandings of countertransference and countertransference-related phenomena. In depth data was obtained from five volunteer psychotherapists in their first three years of practice who were interviewed using a semi-structured interview schedule. These interviews were analysed using a psychoanalytically informed narrative methodology. The need for an experiential model of countertransference that remains true to the often alive, elusive and indescribable nature of countertransference experiences was a central finding of the project. A secondary focus of the research was the therapists’ development of their sense of professional identity. The paucity of research exploring the interaction of countertransference and professional identity development rendered this an important area of investigation. The effect of this process upon the way in which the beginning psychotherapists made sense of their countertransference experiences, was a finding of further significance

Beyond the Witness: Bringing a Process Perspective

For centuries, the foundation of the Anglo-American trial has been the witness.\u27 Witnesses report on their personal observations, provide opinions of character, offer scientific explanations, and in the case of parties, narrate their own story. Indeed, even for documentary and other physical evidence, witnesses often provide the conduit through which such evidence reaches the factfinder. Documentary or physical evidence rarely stands on its own. The law of evidence has thus unsurprisingly focused on-or perhaps obsessed over-witnesses. The hearsay rule and the Confrontation Clause demand that declarants be available witnesses at trial so that they may be subject to cross-examination.\u27 Expert evidence rules emphasize an expert witness\u27s qualifications, bases, and methods.\u27 Even the framework for admitting photographs-evidence that is often self-explanatory-is witnesscentric. Trial practice commonly treats photographs as demonstrative evidence, reducing them to a mere illustration of the vouching witness\u27s testimony. Our contention is that this witness-centered perspective is antiquated and counterproductive. It is a deeply limited and ultimately distortive lens through which the legal system views the evidence available in the modem world