Effects of variability in models: a family of experiments

The ever-growing need for customization creates a need to maintain software systems in many different variants. To avoid having to maintain different copies of the same model, developers of modeling languages and tools have recently started to provide implementation techniques for such variant-rich systems, notably variability mechanisms, which support implementing the differences between model variants. Available mechanisms either follow the annotative or the compositional paradigm, each of which have dedicated benefits and drawbacks. Currently, language and tool designers select the used variability mechanism often solely based on intuition. A better empirical understanding of the comprehension of variability mechanisms would help them in improving support for effective modeling. In this article, we present an empirical assessment of annotative and compositional variability mechanisms for three popular types of models. We report and discuss findings from a family of three experiments with 164 participants in total, in which we studied the impact of different variability mechanisms during model comprehension tasks. We experimented with three model types commonly found in modeling languages: class diagrams, state machine diagrams, and activity diagrams. We find that, in two out of three experiments, annotative technique lead to better developer performance. Use of the compositional mechanism correlated with impaired performance. For all three considered tasks, the annotative mechanism was preferred over the compositional one in all experiments. We present actionable recommendations concerning support of flexible, tasks-specific solutions, and the transfer of established best practices from the code domain to models

A Graphical Approach to Security Risk Analysis

"The CORAS language is a graphical modeling language used to support the security analysis process with its customized diagrams. The language has been developed within the research project "SECURIS" (SINTEF ICT/University of Oslo), where it has been applied and evaluated in seven major industrial field trials. Experiences from the field trials show that the CORAS language has contributed to a more actively involvement of the participants, and it has eased the communication within the analysis group. The language has been found easy to understand and suitable for presentation purposes. With time we have become more and more dependent on various kinds of computerized systems. When the complexity of the systems increases, the number of security risks is likely to increase. Security analyses are often considered complicated and time consuming. A well developed security analysis method should support the analysis process by simplifying communication, interaction and understanding between the participants in the analysis. This thesis describes the development of the CORAS language that is particularly suited for security analyses where "structured brainstorming" is part of the process. Important design decisions are based on empirical investigations. The thesis has resulted in the following artifacts: - A modeling guideline that explains how to draw the different kind of diagrams for each step of the analysis. - Rules for translation which enables consistent translation from graphical diagrams to text. - Concept definitions that contributes to a consistent use of security analysis terms. - An evaluation framework to evaluate and compare the quality of security analysis modeling languages.

Distribution pattern-driven development of service architectures

Distributed systems are being constructed by composing a number of discrete components. This practice is particularly prevalent within the Web service domain in the form of service process orchestration and choreography. Often, enterprise systems are built from many existing discrete applications such as legacy applications exposed using Web service interfaces. There are a number of architectural configurations or distribution patterns, which express how a composed system is to be deployed in a distributed environment. However, the amount of code required to realise these distribution patterns is considerable. In this paper, we propose a distribution pattern-driven approach to service composition and architecting. We develop, based on a catalog of patterns, a UML-compliant framework, which takes existing Web service interfaces as its input and generates executable Web service compositions based on a distribution pattern chosen by the software architect

An experimental evaluation of the understanding of safety compliance needs with models

Proceedings of: 36th International Conference on Conceptual Modeling, ER 2017, Valencia, Spain, November 6–9, 2017Context: Most safety-critical systems have to fulfil compliance needs specified in safety standards. These needs can be difficult to understand from the text of the standards, and the use of conceptual models has been proposed as a solution. Goal: We aim to evaluate the understanding of safety compliance needs with models. Method: We have conducted an experiment to study the effectiveness, efficiency, and perceived benefits in understanding these needs, with text of safety standards and with UML object diagrams. Results: Sixteen Bachelor students participated in the experiment. Their average effectiveness in understanding compliance needs and their average efficiency were higher with models (17% and 15%, respectively). However, the difference is not statistically significant. The students found benefits in using models, but on average they are undecided about their ease of understanding. Conclusions: Although the results are not conclusive enough, they suggest that the use of models could improve the understanding of safety compliance needs.The research leading to this paper has received funding from the AMASS project (H2020-ECSEL grant agreement no 692474; Spain’s MINECO ref. PCIN-2015-262) and the AMoDDI project (Ref. 11130583). We also thank the subjects that participated in the experiment

Using empirical studies to mitigate symbol overload in iStar extensions

UID/CEC/04516/2019Modelling languages are frequently extended to include new constructs to be used together with the original syntax. New constructs may be proposed by adding textual information, such as UML stereotypes, or by creating new graphical representations. Thus, these new symbols need to be expressive and proposed in a careful way to increase the extension’s adoption. A method to create symbols for the original constructs of a modelling language was proposed and has been used to create the symbols when a new modelling language is designed. We argue this method can be used to recommend new symbols for the extension’s constructs. However, it is necessary to make some adjustments since the new symbols will be used with the existing constructs of the modelling language original syntax. In this paper, we analyse the usage of this adapted method to propose symbols to mitigate the occurrence of overloaded symbols in the existing iStar extensions. We analysed the existing iStar extensions in an SLR and identified the occurrence of symbol overload among the existing constructs. We identified a set of fifteen overloaded symbols in existing iStar extensions. We used these concepts with symbol overload in a multi-stage experiment that involved users in the visual notation design process. The study involved 262 participants, and its results revealed that most of the new graphical representations were better than those proposed by the extensions, with regard to semantic transparency. Thus, the new representations can be used to mitigate this kind of conflict in iStar extensions. Our results suggest that next extension efforts should consider user-generated notation design techniques in order to increase the semantic transparency.authorsversionpublishe

A Framework for Seamless Variant Management and Incremental Migration to a Software Product-Line

Context: Software systems often need to exist in many variants in order to satisfy varying customer requirements and operate under varying software and hardware environments. These variant-rich systems are most commonly realized using cloning, a convenient approach to create new variants by reusing existing ones. Cloning is readily available, however, the non-systematic reuse leads to difficult maintenance. An alternative strategy is adopting platform-oriented development approaches, such as Software Product-Line Engineering (SPLE). SPLE offers systematic reuse, and provides centralized control, and thus, easier maintenance. However, adopting SPLE is a risky and expensive endeavor, often relying on significant developer intervention. Researchers have attempted to devise strategies to synchronize variants (change propagation) and migrate from clone&own to an SPL, however, they are limited in accuracy and applicability. Additionally, the process models for SPLE in literature, as we will discuss, are obsolete, and only partially reflect how adoption is approached in industry. Despite many agile practices prescribing feature-oriented software development, features are still rarely documented and incorporated during actual development, making SPL-migration risky and error-prone.Objective: The overarching goal of this PhD is to bridge the gap between clone&own and software product-line engineering in a risk-free, smooth, and accurate manner. Consequently, in the first part of the PhD, we focus on the conceptualization, formalization, and implementation of a framework for migrating from a lean architecture to a platform-based one.Method: Our objectives are met by means of (i) understanding the literature relevant to variant-management and product-line migration and determining the research gaps (ii) surveying the dominant process models for SPLE and comparing them against the contemporary industrial practices, (iii) devising a framework for incremental SPL adoption, and (iv) investigating the benefit of using features beyond PL migration; facilitating model comprehension.Results: Four main results emerge from this thesis. First, we present a qualitative analysis of the state-of-the-art frameworks for change propagation and product-line migration. Second, we compare the contemporary industrial practices with the ones prescribed in the process models for SPL adoption, and provide an updated process model that unifies the two to accurately reflect the real practices and guide future practitioners. Third, we devise a framework for incremental migration of variants into a fully integrated platform by exploiting explicitly recorded metadata pertaining to clone and feature-to-asset traceability. Last, we investigate the impact of using different variability mechanisms on the comprehensibility of various model-related tasks.Future work: As ongoing and future work, we aim to integrate our framework with existing IDEs and conduct a developer study to determine the efficiency and effectiveness of using our framework. We also aim to incorporate safe-evolution in our operators

Evaluating practitioner cyber-security attack graph configuration preferences

Attack graphs and attack trees are a popular method of mathematically and visually rep- resenting the sequence of events that lead to a successful cyber-attack. Despite their popularity, there is no standardised attack graph or attack tree visual syntax configuration, and more than seventy self-nominated attack graph and twenty attack tree configurations have been described in the literature - each of which presents attributes such as preconditions and exploits in a different way. This research proposes a practitioner-preferred attack graph visual syntax configuration which can be used to effectively present cyber-attacks. Comprehensive data on participant ( n=212 ) preferences was obtained through a choice based conjoint design in which participants scored attack graph configuration based on their visual syntax preferences. Data was obtained from multiple participant groups which included lecturers, students and industry practitioners with cyber-security specific or general computer science backgrounds. The overall analysis recommends a winning representation with the following attributes. The flow of events is represented top-down as in a flow diagram - as opposed to a fault tree or attack tree where it is presented bottom-up, preconditions - the conditions required for a successful exploit, are represented as ellipses and exploits are represented as rectangles. These results were consistent across the multiple groups and across scenarios which differed according to their attack complexity. The research tested a number of bottom-up approaches - similar to that used in attack trees. The bottom-up designs received the lowest practitioner preference score indicating that attack trees - which also utilise the bottom-up method, are not a preferred design amongst practitioners - when presented with an alternative top-down design. Practitioner preferences are important for any method or framework to become accepted, and this is the first time that an attack modelling technique has been developed and tested for practitioner preferences

Broadening the Scope of Security Usability from the Individual to the Organizational : Participation and Interaction for Effective, Efficient, and Agile Authorization

Restrictions and permissions in information systems -- Authorization -- can cause problems for those interacting with the systems. Often, the problems materialize as an interference with the primary tasks, for example, when restrictions prevent the efficient completing of work and cause frustration. Conversely, the effectiveness can also be impacted when staff is forced to circumvent the measure to complete work -- typically sharing passwords among each other. This is the perspective of functional staff and the organization. There are further perspectives involved in the administration and development of the authorization measure. For instance, functional staff need to interact with policy makers who decide on the granting of additional permissions, and policy makers, in turn, interact with policy authors who actually implement changes. This thesis analyzes the diverse contexts in which authorization occurs, and systematically examines the problems that surround the different perspectives on authorization in organizational settings. Based on prior research and original research in secure agile development, eight principles to address the authorization problems are identified and explored through practical artifacts

A novel knowledge discovery based approach for supplier risk scoring with application in the HVAC industry

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University LondonThis research has led to a novel methodology for assessment and quantification of supply risks in the supply chain. The research has built on advanced Knowledge Discovery techniques and has resulted to a software implementation to be able to do so. The methodology developed and presented here resembles the well-known consumer credit scoring methods as it leads to a similar metric, or score, for assessing a supplier’s reliability and risk of conducting business with that supplier. However, the focus is on a wide range of operational metrics rather than just financial, which credit scoring techniques typically focus on. The core of the methodology comprises the application of Knowledge Discovery techniques to extract the likelihood of possible risks from within a range of available datasets. In combination with cross-impact analysis, those datasets are examined for establish the inter-relationships and mutual connections among several factors that are likely contribute to risks associated with particular suppliers. This approach is called conjugation analysis. The resulting parameters become the inputs into a logistic regression which leads to a risk scoring model the outcome of the process is the standardized risk score which is analogous to the well-known consumer risk scoring model, better known as FICO score. The proposed methodology has been applied to an Air Conditioning manufacturing company. Two models have been developed. The first identifies the supply risks based on the data about purchase orders and selected risk factors. With this model the likelihoods of delivery failures, quality failures and cost failures are obtained. The second model built on the first one but also used the actual data about the performance of supplier to identify risks of conducting business with particular suppliers. Its target was to provide quantitative measures of an individual supplier’s risk level. The supplier risk scoring model is tested on the data acquired from the company for its performance analysis. The supplier risk scoring model achieved 86.2% accuracy, while the area under curve (AUC) was 0.863. The AUC curve is much higher than required model’s validity threshold value of 0.5. It represents developed model’s validity and reliability for future data. The numerical studies conducted with real-life datasets have demonstrated the effectiveness of the proposed methodology and system as well as its future potential for industrial adoption

Development of Security Risk Measurement Model within Misuse Cases and BPMN

Iga organisatsiooni kõige tähtsam ülesanne on oma vara kaitsta. Kuna mitte ühtegi süsteemi ei ole võimalik täielikult turvaliseks teha, seega rakendavad ettevõtted erinevaid kontrolle, et oma vara erinevate ohtude eest kaitsta. Riskianalüüs on üks oluline samm infosüsteemide (IS) turvalisuse tagamises ja tänaseks on välja töötatud erinevaid IS-de riskianalüüsi meetodeid, kuid need osutavad peamiselt üldisi suunised riskide hindamiseks. See dokument, aga käsitleb probleemi kuidas mõõta riski illustreerituna modelleerimiskeelte abist. Selleks on valitud kaks modelleerimise keelt: väärkasutamise juhtumid (Misuse Case) ja äriprotsesside modelleerimiskeel (BPMN). Praktilisest kogemustest on näha, et samad turvaaukudega seotud sündmused toimuvad perioodiliselt ning nende järel turvalisusega seotud riske ei maandata. Seda sellepärast, et ei ole näha turvaaukude korduvat kasutamist või riskide erinevaid tasemeid ja kaotused ei ole mõõdetud, mistõttu arvestatakse, et turvaaukudega kaasnevad probleemid on vähem tähtsad. Teadmata, kui palju kahju üks turvalisusega seotud sündmus teeb, ei saa juhtorgan otsustada, kas tegeleda riski maandamisega või mitte. Kui riskid oleksid mõõdetud ja nende väärtused oleksid nähtavad, oleks lihtsam teha õigeid otsuseid riskide maandamiseks. Selle töö eesmärk on aidata organisatsiooni juhtidel aru saada kui tõsised on turvalisusega seotud riskid, selleks visualiseerides meetrikaid ja tuues välja riskide kalkulatsioone. Et seda teha ka modelleerimiskeeltes, tuleb selleks visualiseerida riskidega seotud juhtumeid. Alles seejärel on võimalik mõõta turvalisusega seotud juhtumite tõsisust. Selle töö kirjutamise hetkel ei eksisteeri ühtegi mudelit mis suudaks visualiseerida mõõtmist koos juhtumi endaga. Selle töö tulemusena arendatakse mõõtmisemudel väärkasutamise juhtumite ja äriprotsesside modelleerimiskeele diagrammide piirides. Need mudelid hõlbustavad üldise riski hindamist jagades riski alam-osadeks ja mõõdavad eraldi vara väärtust, ohu potentsiaalsust ja haavatavust. Samuti annavad need teavet riskide kulukuse kohta ja toovad välja vastumeetmete rakendamise kasulikkuse. See tähendab, et riski meetrika ja tõsisus on koheselt nähtav. See aitab turvalisuse spetsialistil teha otsuseid, kas mõne konkreetse turvariski maandamiseks investeerimine on mõistlik või mitte. See peaks andma ka selge pildi ettevõtte kahjumist, kui riske kasutatakse ära ja aitab mõista, kas see on märkimisväärne kaotus või mitte. Kahe mudeli välja töötatamiseks kasutades nii teoreetilisi kui ka empiirilisi andmeid, seega turvalisusega seotud riskide mõõtmise mudelid annavad lahenduse probleemile, kuidas arvutada riske mis on võetud pärismaailmast, kasutades selleks väärkasutamise juhtumeid ja äriprotsesside modelleerimiskeelt. Lisaks uuritakse olemasolevaid hindamise meetoditeid ja standardeid koos erinevate modelleerimiskeeltega, ning töös kasutakse näiteid ühest töötavast organisatsioonist. Pärast mudelite välja töötamist need ka rakendatakse, et uurida väljapakutud meetrikate nähtavust. Valideerimise ajal võrreldakse kahte mudelit selgitamaks välja milline nendest annab parema ülevaate juurutatud meetrikatest.One of the most important tasks of any organization is to secure its assets. Since no system could be made completely secure, in order to prevent security flaws, companies apply controls to safeguard their assets from different threats. Therefore, risk analysis is an important step for the management of information systems security (ISS). Today various ISS risk analysis methods have been developed, but they mainly provide general guidelines to estimate the risk. The problem defined in the thesis is how to measure the risk illustrated with the help of a modeling languages. For that two modeling languages were chosen: misuse cases and BPMN. This is a problem, because we can see from a practical experience that the same security events are happening periodically, but the security risks are not treated. This may occur either because people do not see the repeated exploitation of vulnerabilities, the risk level and losses are not measured, considering the problems of a less importance. Without knowing exactly how much damage the security event makes, the management is not able to decide whether the risk should be fixed or not. If a risk is measured and values are visible, it is easier to do a proper decision about the risk mitigation. Our goal is to help understand the severity of the security risks by visualizing the metrics and calculations of a risk. For that in modeling languages a visualization of thread cases is needed. Then security cases need to be measured. Today there is no existing model that can visualize the measurement together with the case itself. The contribution of this thesis will be the development of measurement model within misuse case and BPMN diagrams. These models will facilitate the evaluation of an overall risk, by dividing the risk into sub-components and individually measuring the asset value, potentiality of thread, level of vulnerability. It will also give information about cost and benefit of implementation of countermeasures. This means that the metrics and the severity of a risk will be visible straight away. This will help the security specialist to make a decision whether the investment into a particular security flaw is reasonable or not. It should give a clear picture of the company's losses from exploitation of risk and will make it easier to understand whether it is a substantial loss or not. Two models will be developed using both theoretical and empirical data. Existing assessment approaches and standards together with different modeling languages will be studied. At the same moment the cases from the working organization will be taken. Two models will be developed and applied to investigate the visibility of metrics proposed. The developed security risk measurement models will give a solution how to calculate the risks taken from a real world example using misuse cases and BPMN. During validation we have tested our two models, which of them gives better visibility of the metrics introduced