Research Findings on Empirical Evaluation of Requirements Specifications Approaches

Numerous software requirements specification (SRS) approaches have been proposed in software engineering. However, there has been little empirical evaluation of the use of these approaches in specific contexts. This paper describes the results of a mapping study, a key instrument of the evidence-based paradigm, in an effort to understand what aspects of SRS are evaluated, in which context, and by using which research method. On the basis of 46 identified and categorized primary studies, we found that understandability is the most commonly evaluated aspect of SRS, experiments are the most commonly used research method, and the academic environment is where most empirical evaluation takes place

Improving Requirements Completeness: Automated Assistance through Large Language Models

Natural language (NL) is arguably the most prevalent medium for expressing systems and software requirements. Detecting incompleteness in NL requirements is a major challenge. One approach to identify incompleteness is to compare requirements with external sources. Given the rise of large language models (LLMs), an interesting question arises: Are LLMs useful external sources of knowledge for detecting potential incompleteness in NL requirements? This article explores this question by utilizing BERT. Specifically, we employ BERT's masked language model (MLM) to generate contextualized predictions for filling masked slots in requirements. To simulate incompleteness, we withhold content from the requirements and assess BERT's ability to predict terminology that is present in the withheld content but absent in the disclosed content. BERT can produce multiple predictions per mask. Our first contribution is determining the optimal number of predictions per mask, striking a balance between effectively identifying omissions in requirements and mitigating noise present in the predictions. Our second contribution involves designing a machine learning-based filter to post-process BERT's predictions and further reduce noise. We conduct an empirical evaluation using 40 requirements specifications from the PURE dataset. Our findings indicate that: (1) BERT's predictions effectively highlight terminology that is missing from requirements, (2) BERT outperforms simpler baselines in identifying relevant yet missing terminology, and (3) our filter significantly reduces noise in the predictions, enhancing BERT's effectiveness as a tool for completeness checking of requirements.Comment: Submitted to Requirements Engineering Journal (REJ) - REFSQ'23 Special Issue. arXiv admin note: substantial text overlap with arXiv:2302.0479

Grand Challenges of Traceability: The Next Ten Years

In 2007, the software and systems traceability community met at the first Natural Bridge symposium on the Grand Challenges of Traceability to establish and address research goals for achieving effective, trustworthy, and ubiquitous traceability. Ten years later, in 2017, the community came together to evaluate a decade of progress towards achieving these goals. These proceedings document some of that progress. They include a series of short position papers, representing current work in the community organized across four process axes of traceability practice. The sessions covered topics from Trace Strategizing, Trace Link Creation and Evolution, Trace Link Usage, real-world applications of Traceability, and Traceability Datasets and benchmarks. Two breakout groups focused on the importance of creating and sharing traceability datasets within the research community, and discussed challenges related to the adoption of tracing techniques in industrial practice. Members of the research community are engaged in many active, ongoing, and impactful research projects. Our hope is that ten years from now we will be able to look back at a productive decade of research and claim that we have achieved the overarching Grand Challenge of Traceability, which seeks for traceability to be always present, built into the engineering process, and for it to have "effectively disappeared without a trace". We hope that others will see the potential that traceability has for empowering software and systems engineers to develop higher-quality products at increasing levels of complexity and scale, and that they will join the active community of Software and Systems traceability researchers as we move forward into the next decade of research

Grand Challenges of Traceability: The Next Ten Years

In 2007, the software and systems traceability community met at the first Natural Bridge symposium on the Grand Challenges of Traceability to establish and address research goals for achieving effective, trustworthy, and ubiquitous traceability. Ten years later, in 2017, the community came together to evaluate a decade of progress towards achieving these goals. These proceedings document some of that progress. They include a series of short position papers, representing current work in the community organized across four process axes of traceability practice. The sessions covered topics from Trace Strategizing, Trace Link Creation and Evolution, Trace Link Usage, real-world applications of Traceability, and Traceability Datasets and benchmarks. Two breakout groups focused on the importance of creating and sharing traceability datasets within the research community, and discussed challenges related to the adoption of tracing techniques in industrial practice. Members of the research community are engaged in many active, ongoing, and impactful research projects. Our hope is that ten years from now we will be able to look back at a productive decade of research and claim that we have achieved the overarching Grand Challenge of Traceability, which seeks for traceability to be always present, built into the engineering process, and for it to have "effectively disappeared without a trace". We hope that others will see the potential that traceability has for empowering software and systems engineers to develop higher-quality products at increasing levels of complexity and scale, and that they will join the active community of Software and Systems traceability researchers as we move forward into the next decade of research

OntoEng: A design method for ontology engineering in information systems

This paper addresses the design problem relating to ontology engineering in the discipline of information systems. Ontology engineering is a realm that covers issues related to ontology development and use throughout its life span. Nowadays, ontology as a new innovation promises to improve the design, semantic integration, and utilization of information systems. Ontologies are the backbone of knowledge-based systems. In addition, they establish sharable and reusable common understanding of specific domains amongst people, information systems, and software agents. Notwithstanding, the ontology engineering literature does not provide adequate guidance on how to build, evaluate, and maintain ontologies. On the basis of the gathered experience during the development of V4 Telecoms Business Model Ontology as well as the conducted integration of the related literature from the design science paradigm, this paper introduces OntoEng and its application as a novel systematic design method for ontology engineering

An Experimental Validation of Public Cloud Mobile Banking

Currently, financial institutions incur huge expenditure to implement and maintain mobile banking (m- Banking) solutions and this cost is bound to rise significantly, as more customers subscribe to m-Banking services. Cloud computing has potential to facilitate reduced cost, high scalability and a variable cost structure that could guarantee cheaper, reliable and sustainable m-Banking in the long term. While the adoption of organizational private clouds seems natural for banks because of the sensitive nature of banking transactions, some have argued for the adoption of public clouds as a better alternative, despite concerns on issues such as trust, security and privacy. However, there is lack of sufficient empirical evidence in the literature on the suitability of public clouds for m- Banking. Hence, this study presents an investigation of the use of public cloud for m-Banking. A prototype cloud- based m-Banking application was developed using a public platform-as-a-service (Paas) cloud model, which was evaluated for usability and robustness in a controlled experiment. The evaluation result shows that m-Banking on public cloud is viable, if the cloud-based application is sufficiently robust and usable. The result also indicates that m-Banking services on public cloud are suitable for adoption by the banking industry