Towards a Model-Centric Software Testing Life Cycle for Early and Consistent Testing Activities

The constant improvement of the available computing power nowadays enables the accomplishment of more and more complex tasks. The resulting implicit increase in the complexity of hardware and software solutions for realizing the desired functionality requires a constant improvement of the development methods used. On the one hand over the last decades the percentage of agile development practices, as well as testdriven development increases. On the other hand, this trend results in the need to reduce the complexity with suitable methods. At this point, the concept of abstraction comes into play, which manifests itself in model-based approaches such as MDSD or MBT. The thesis is motivated by the fact that the earliest possible detection and elimination of faults has a significant influence on product costs. Therefore, a holistic approach is developed in the context of model-driven development, which allows applying testing already in early phases and especially on the model artifacts, i.e. it provides a shift left of the testing activities. To comprehensively address the complexity problem, a modelcentric software testing life cycle is developed that maps the process steps and artifacts of classical testing to the model-level. Therefore, the conceptual basis is first created by putting the available model artifacts of all domains into context. In particular, structural mappings are specified across the included domain-specific model artifacts to establish a sufficient basis for all the process steps of the life cycle. Besides, a flexible metamodel including operational semantics is developed, which enables experts to carry out an abstract test execution on the modellevel. Based on this, approaches for test case management, automated test case generation, evaluation of test cases, and quality verification of test cases are developed. In the context of test case management, a mechanism is realized that enables the selection, prioritization, and reduction of Test Model artifacts usable for test case generation. I.e. a targeted set of test cases is generated satisfying quality criteria like coverage at the model-level. These quality requirements are accomplished by using a mutation-based analysis of the identified test cases, which builds on the model basis. As the last step of the model-centered software testing life cycle two approaches are presented, allowing an abstract execution of the test cases in the model context through structural analysis and a form of model interpretation concerning data flow information. All the approaches for accomplishing the problem are placed in the context of related work, as well as examined for their feasibility by of a prototypical implementation within the Architecture And Analysis Framework. Subsequently, the described approaches and their concepts are evaluated by qualitative as well as quantitative evaluation. Moreover, case studies show the practical applicability of the approach

Implementation Science Meets Software Development to Create eHealth Components for an Integrated Care Model for Allogeneic Stem Cell Transplantation Facilitated by eHealth: The SMILe Study as an Example

To describe a process of creating eHealth components for an integrated care model using an agile software development approach, user-centered design and, via the Behavior Change Wheel, behavior theory-guided content development. Following the principles of implementation science and using the SMILe project (integrated care model for allogeneic stem cell transplantation facilitated by eHealth) as an example, this study demonstrates how to narrow the research-to-practice gap often encountered in eHealth projects.; We followed a four-step process: (a) formation of an interdisciplinary team; (b) a contextual analysis to drive the development process via behavioral theory; (c) transfer of content to software following agile software development principles; and (d) frequent stakeholder and end user involvement following user-centered design principles.; Our newly developed comprehensive development approach allowed us to create a running eHealth component and embed it in an integrated care model. An interdisciplinary team's collaboration at specified interaction points supported clear, timely communication and interactions between the specialists. Because behavioral theory drove the content development process, we formulated user stories to define the software features, which were prioritized and iteratively developed using agile software development principles. A prototype intervention module has now been developed and received high ratings on the System Usability Scale after two rounds of usability testing.; Following an agile software development process, structured collaboration between nursing scientists and software specialists allowed our interdisciplinary team to develop meaningful, theory-based eHealth components adapted to context-specific needs.; The creation of high-quality, accurately fitting eHealth components specifically to be embedded in integrated care models should increase the chances of uptake, adoption, and sustainable implementation in clinical practice

Towards Debugging and Improving Adversarial Robustness Evaluations ​

Despite exhibiting unprecedented success in many application domains, machine‐learning models have been shown to be vulnerable to adversarial examples, i.e., maliciously perturbed inputs that are able to subvert their predictions at test time. Rigorous testing against such perturbations requires enumerating all possible outputs for all possible inputs, and despite impressive results in this field, these methods remain still difficult to scale to modern deep learning systems. For these reasons, empirical methods are often used. These adversarial perturbations are optimized via gradient descent, minimizing a loss function that aims to increase the probability of misleading the model’s predictions. To understand the sensitivity of the model to such attacks, and to counter the effects, machine-learning model designers craft worst-case adversarial perturbations and test them against the model they are evaluating. However, many of the proposed defenses have been shown to provide a false sense of security due to failures of the attacks, rather than actual improvements in the machine‐learning models’ robustness. They have been broken indeed under more rigorous evaluations. Although guidelines and best practices have been suggested to improve current adversarial robustness evaluations, the lack of automatic testing and debugging tools makes it difficult to apply these recommendations in a systematic and automated manner. To this end, we tackle three different challenges: (1) we investigate how adversarial robustness evaluations can be performed efficiently, by proposing a novel attack that can be used to find minimum-norm adversarial perturbations; (2) we propose a framework for debugging adversarial robustness evaluations, by defining metrics that reveal faulty evaluations as well as mitigations to patch the detected problems; and (3) we show how to employ a surrogate model for improving the success of transfer-based attacks, that are useful when gradient-based attacks are failing due to problems in the gradient information. To improve the quality of robustness evaluations, we propose a novel attack, referred to as Fast Minimum‐Norm (FMN) attack, which competes with state‐of‐the‐art attacks in terms of quality of the solution while outperforming them in terms of computational complexity and robustness to sub‐optimal configurations of the attack hyperparameters. These are all desirable characteristics of attacks used in robustness evaluations, as the aforementioned problems often arise from the use of sub‐optimal attack hyperparameters, including, e.g., the number of attack iterations, the step size, and the use of an inappropriate loss function. The correct refinement of these variables is often neglected, hence we designed a novel framework that helps debug the optimization process of adversarial examples, by means of quantitative indicators that unveil common problems and failures during the attack optimization process, e.g., in the configuration of the hyperparameters. Commonly accepted best practices suggest further validating the target model with alternative strategies, among which is the usage of a surrogate model to craft the adversarial examples to transfer to the model being evaluated is useful to check for gradient obfuscation. However, how to effectively create transferable adversarial examples is not an easy process, as many factors influence the success of this strategy. In the context of this research, we utilize a first-order model to show what are the main underlying phenomena that affect transferability and suggest best practices to create adversarial examples that transfer well to the target models.

Powering the Academic Web

Context: Locating resources on the Web has become increasingly difficult for users and poses a number of issues. The sheer size of the Web means that despite what appears to be an increase in the amount of quality material available, the effort involved in locating that material is also increasing; in effect, the higher quality material is being diluted by the lesser quality. One such group affected by this problem is post-graduate students. Having only a finite amount of time to devote to research, this reduces their overall quality study time. Aim: This research investigates how post-graduate students use the Web as a learning resource and identifies a number of areas of concern with its use. It considers the potential for improvement in this matter by using a number of concepts such as: collaboration; peer reviewing and document classification and comparison techniques. This research also investigates whether by combining several of the identified technologies and concepts, student research on the Web can be improved. Method: Using some of the identified concepts as components, this research proposes a model to address the highlighted areas of concern. The proposed model, named the Durham Browsing Assistant (DurBA) is defined, and a number of key concepts which show potential within it are uncovered. One of the key concepts is chosen, that of document comparison. Given a source document, can a computer system reliably identify other documents which most closely match it from other on the Web? A software tool was created which allowed the testing of document comparison techniques, this was called the Durham Textual Comparison system (DurTeC) and it had two key concepts. The first was that it would allow various algorithms to be applied to the comparison process. The second concept was that it could simulate collaboration by allowing data to be altered, added and removed as if by multiple users. A set of experiments were created to test these algorithms and identify those which gave the best results. Results: The results from the experiments identified a number of the most promising relationships between comparison and collaboration processes. It also highlighted those which had a negative effect on the process, and those which produced variable results. Amongst the results, it was found that: 1. By providing DurTeC with additional source documents to the original, as if through a recommendation process, it was able to increase its accuracy substantially. 2. By allowing DurTeC to use synonym lists to expand its vocabulary, in many cases, it was found to have reduced its accuracy. 3. By restricting those words which DurTeC considered in its comparison process, based upon their value in the source document, accuracy could be increased. This could be considered as a form of collaborative keyword selection. Conclusion: This research shows that improvements can be made in the accuracy of identifying similar resources by using a combination of comparison and collaboration processes. The proposed model, DurBA would be an ideal host for such a system

Determinants of user continuance intention towards mobile money services : the case of M-pesa in Kenya

Includes bibliographical referencesThe turn of the millennium witnessed the uptake and proliferation of mobile technology in developing regions. This occurrence has provided a medium for mobile telecommunication vendors within the region to create and offer services that are now accessible across socio-economic classes. A notable case of a widely adopted mobile technology-enabled service in the developing world is a mobile money service in Kenya called M-pesa. Since its inception, M-pesa has witnessed a mass adoption which has generally been attributed to prior lack of access by majority of individuals' in the country to affordable regulated financial services. M-pesa's presence has now been anticipated to afford a larger population the initial opportunity to harness economic benefits such as: increase money circulation, increase employment opportunities, facilitate social capital accumulation, facilitate savings, and promote financial autonomy, amongst others. Also, M-pesa based transactions in Kenya are reported to exceed those of western union globally. Whilst M-pesa presently vaunts large user adoption numbers, it is the first of its kind in the region to amass such achievement. Further, historically: products and services of similar nature to M-pesa have been unsustainable. A case of M-pesa's demise would have dire implication for the Kenyan economy and 30% of the households in the country that rely on it for remittances. To understand this phenomenon, extant studies have examined the drivers of adoption of this service but have slacked in subsequent investigations to understand user continuance with the service. As such, the information systems literature cautions that initial adoption of technology, although crucial, does not guarantee sustained use. Therefore it is imperative to investigate drivers of continuance. In general, extant research has not focused on investigations of user continuance intention in Africa. In response, this thesis presents an African based study on the determinants of user continuance intention towards M-pesa. Specifically, the purpose of this study was to i) identify and discuss factors from the literature that are most likely to influence user continuance intention towards M-pesa, (ii) develop a research model that is grounded in theory, (iii) test the model within the sample context to identify the antecedents and determinants of user continuance intention towards M-pesa in Kenya. A broad, critical review of the relevant literature provided basis for hypothesized relationships between the identified factors. A formal survey of users of M-pesa in Kenya comprised the phase of data collection and resulted in a usable data set of (n=434). The data collected from the respondents within Kenya was relied upon to test the hypotheses. The survey instrument used to measure the study's constructs was developed via a process of literature review, expert pre-testing, pilot testing, and statistical validation. Partial Least Square and Artificial Neural Network analyses were used to examine the study's measurement and structural model comprising variables of : behavioural beliefs (post-usage usefulness, confirmation, satisfaction), control-beliefs (utilization and flow), object-based beliefs (perceived task-technology fit, system quality, information quality, and service quality), and attitudinal belief (trust). Collectively, the afore-listed ten independent variables and one dependent variable (continuance intention) comprised the study's model. Four of the independent variables (utilization, satisfaction, flow, and trust) were hypothesized to directly determine continuance intention. Of these four, all emerged as determinants of continuance intention. However, trust emerged as the strongest determinant, subsequently, utilization, flow, and satisfaction respectively. The result was unexpected, as satisfaction (a behavioural belief) has been presented in the extant literature as the dominant determinant of continuance intention but does not hold a consistent predictive strength in a developing world. Its predictive power was diluted by trust, utilization, and flow amongst the Kenyan sample. The study's model revealed an R² of 0.334. The analyses demonstrated that user continuance intention is determined by factors across object, control, attitudinal, and behavioural beliefs. The unexpected finding of the rankings of predictive strength of the factors turns a new leaf and introduces areas of further inquiry in future studies. The study concludes with realized contributions to theory and important guidelines for current and future technology-enabled service vendors in developing regions

Bayesian inference and failure analysis for risk assessment in quality engineering

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University LondonFailure is the state of not achieving a desired or intended goal. Failure analysis planning in the context of risk assessment is an approach that helps to reduce total cost, increase production capacity, and produce higher-quality products. One of the most common issues that businesses confront are defective products. This issue not only results in monetary loss, but also in a loss of status. Companies must improve their production quality and reduce the quantity of faulty products in order to continue operating in a healthy and profitable manner in today’s very competitive environment. On the other hand, there is the ongoing COVID-19 pandemic, which has thrown the world’s natural order into disarray, and has been designated a Public Health Emergency of International Concern by the World Health Organization. The demand for quality control is rapidly increasing. Failure analysis is thus an useful tool for identifying common failures, their likely causes, and their impact on the health system, as well as plotting strategies to limit COVID-19 transmission. It is now more vital than ever to enhance failure analysis methods. The traditional FMEA (Failure mode and effects analysis) is one of the most widely used approaches for identifying and classifying failure modes (FMs) and failure causes (FCs). It is a risk analysis tool for coping with possible failures and is widely used in the reliability engineering, safety engineering and quality engineering. To prioritize risks of different failure modes, FMEA uses the risk priority number (RPN), which is the product of three risk measures: severity (S), occurrence (O) and detection (D). Traditional FMEA, on the other hand, has drawbacks, such as the inability to cope with uncertain failure data, such as expert subjective evaluations, the failure events’ conditionality, RPN has a high degree of subjectivity, comparing various RPNs is challenging, potential errors may be ignored in the conventional FMEA process, etc. To overcome these limitations, I present an integrated Bayesian approach to FMEA in this thesis. In this proposed approach, I worked with experts in quality engineering and used Bayesian inference to estimate the FMEA risk parameters: S, O and D. The proposed approach is intended to become more practical and less subjective as more data is added. Bayesian statistics is a statistical theory that is based on the Bayesian interpretation of probability, which states that probability expresses a degree of belief or information (knowledge) about an event. Bayesian statistics addresses the issues with uncertainties found in frequentist statistics, such as the distribution of contributing factors, the implications of using specific distributions and specifies that there is some prior probability. A prior can be derived from previous information, such as previous experiments, but it can also be derived from a trained subject-matter expert’s purely subjective assessment. Frequentist statistics (also known as classical statistics) has several limitations, including a lack of uncertainty information in predictions, no built-in regularisation, and no consideration of prior knowledge. Due to the availability of powerful computers and new algorithms, Bayesian methods have seen increased use within statistics in the twenty-first century, and this thesis highlights the effective use of Bayesian analyses to address the shortcomings of the current FMEA with the revamped Bayesian FMEA. As a demonstration of the approach, three case studies are presented. The first case study is a Bayesian risk assessment approach of the modified SEIR (susceptible-exposed-infectious-recovered) model for the transmission dynamics of COVID-19 with an exponentially distributed. The effective reproduction number is estimated based on laboratory-confirmed cases and death data using Bayesian inference and analyse the impact of the community spread of COVID-19 across the United Kingdom. The value of effective reproduction number models the average number of infections caused by a case of an infectious disease in a population that includes not only susceptible people. The FMEA is then applied to evaluate the effectiveness of the action measures taken to manage the COVID-19 pandemic. In the FMEA, the focus was on COVID-19 infections and therefore the failure mode is taken as positive cases. The model is applied to COVID-19 data showing the effectiveness of interventions adopted to control the epidemic by reducing the effective reproduction number of COVID-19. The risk measures were estimated from the case fatality rate (S), the posterior median of the effective reproduction number (O) and the current corrective measures used by government policies (D). The second case study is a Bayesian risk assessment of a coordinate measuring machine (CMM) process using failure mode, effects and criticality analysis (FMECA) and an augmented form error model. The form error is defined as the deviation of a manufactured part from its design or ideal shape, and it is a key characteristic to evaluate in quality engineering and manufacturing. The form error is presented as a probabilistic model using symmetric unimodal distributions. Bayesian inference is then used to identify influence factors associated with the measurement process due to form error, environmental, human and random effects. A risk assessment is then performed by combining Bayesian inference, FMECA and conformity testing, to quantify and minimise the risk of wrong decisions. In the FMECA, the focus was on CMM measurement process and I identified four major FMs that can occur: probe, mechanical, environmental and measurement performance failure. Eleven FCs were also observed, each of which was linked to one of the four FMs. The risk measures were estimated from the posterior probability of failure causes associated with the CMM measurement process (O), the severity of a specific consumer’s risk (S) and the detectability of failures from the posterior standard deviation of the form error model (D). The third case study is a Bayesian risk assessment of a CMM measurement process using an autoregressive (AR) form error model and a combined Fault tree analysis (FTA) and FMEA approach to predict significant failure modes and causes. The main idea is to estimate and predict the form error based on CMM data using Gibbs sampling and analyse the impact of the CMM measurement process on product conformity testing. The FTA is used to compare the actual and predicted form error data from the Bayesian AR plot to determine the likelihood of the CMM measurement process failing using binary data. The acquired binary data is then classified into four states (true positive, true negative, false positive, and false negative) using a confusion matrix, which is subsequently utilized to calculate key classification measures (i.e., error rate, prediction rate, prevalence rate, sensitivity rate, etc). The classification measures were then used to assess the FMEA risk measures S, O, and D, which were critical for determining the RPN and making decisions. Analytical and numerical methods are used in all case studies to highlight the practical implications of our findings and are meant to be practical without complex computing. The proposed methodologies can find applications in numerous disciplines and wide quality engineering

Testing in the incremental design and development of complex products

Testing is an important aspect of design and development which consumes significant time and resource in many companies. However, it has received less research attention than many other activities in product development, and especially, very few publications report empirical studies of engineering testing. Such studies are needed to establish the importance of testing and inform the development of pragmatic support methods. This paper combines insights from literature study with findings from three empirical studies of testing. The case studies concern incrementally developed complex products in the automotive domain. A description of testing practice as observed in these studies is provided, confirming that testing activities are used for multiple purposes depending on the context, and are intertwined with design from start to finish of the development process, not done after it as many models depict. Descriptive process models are developed to indicate some of the key insights, and opportunities for further research are suggested

A Quality Model for Actionable Analytics in Rapid Software Development

Background: Accessing relevant data on the product, process, and usage perspectives of software as well as integrating and analyzing such data is crucial for getting reliable and timely actionable insights aimed at continuously managing software quality in Rapid Software Development (RSD). In this context, several software analytics tools have been developed in recent years. However, there is a lack of explainable software analytics that software practitioners trust. Aims: We aimed at creating a quality model (called Q-Rapids quality model) for actionable analytics in RSD, implementing it, and evaluating its understandability and relevance. Method: We performed workshops at four companies in order to determine relevant metrics as well as product and process factors. We also elicited how these metrics and factors are used and interpreted by practitioners when making decisions in RSD. We specified the Q-Rapids quality model by comparing and integrating the results of the four workshops. Then we implemented the Q-Rapids tool to support the usage of the Q-Rapids quality model as well as the gathering, integration, and analysis of the required data. Afterwards we installed the Q-Rapids tool in the four companies and performed semi-structured interviews with eight product owners to evaluate the understandability and relevance of the Q-Rapids quality model. Results: The participants of the evaluation perceived the metrics as well as the product and process factors of the Q-Rapids quality model as understandable. Also, they considered the Q-Rapids quality model relevant for identifying product and process deficiencies (e.g., blocking code situations). Conclusions: By means of heterogeneous data sources, the Q-Rapids quality model enables detecting problems that take more time to find manually and adds transparency among the perspectives of system, process, and usage.Comment: This is an Author's Accepted Manuscript of a paper to be published by IEEE in the 44th Euromicro Conference on Software Engineering and Advanced Applications (SEAA) 2018. The final authenticated version will be available onlin