Architectural decisions in AI-based systems: an ontological view

Architecting AI-based systems entails making some decisions that are particular to this type of systems. Therefore, it becomes necessary to gather all necessary knowledge to inform such decisions, and to articulate this knowledge in a form that facilitates knowledge transfer among different AI projects. In this exploratory paper, we first present the main results of a literature survey in the area, and then we propose a preliminary ontology for architectural decision making, which we exemplify using a subset of the papers selected in the literature review. In the discussion, we remark on the variety of decision types and system contexts, highlighting the need to further investigate the current state of research and practice in this area. Besides, we summarize our plans to move along this research area by widening the literature review and incorporating more AI-related concepts to this first version of the ontology.This paper has been funded by the Spanish Ministerio de Ciencia e Innovación under project / funding scheme PID2020-117191RB-I00 / AEI/10.13039/501100011033.Peer ReviewedPostprint (author's final draft

Evaluating Services Computing Systems Engineering Framework using An Acceptance Model

The gap between business services and IT services becomes a major concern in services computing. As an approach for service-based IT solution, services computing systems are promised to be able to bridge the gap between these services. The implementation will require an engineering framework as a guide to building the systems. The framework needs to be evaluated to provide important feedback to the framework development. This paper outlines the evaluation of SCSE framework through an acceptance model. The study develops an acceptance model based on the experiences of a group of engineers after using the framework to build smart campus services systems. A survey involving 54 systems engineers with various engineering backgrounds was conducted to assess the experiences of the engineers in using the framework. The results of the acceptance model show that both perceived ease of use, represented by the level of agreement (υ1) and perceived usefulness, represented by the level of importance (υ2) deliver good results almost for the entire stages of the proposed framework. In addition, the user experiences of using the proposed framework are in the acceptable levels. The contribution of this paper is an enrichment of the engineering methodologies for the service-oriented system from the perspective of services computing

A Perception of the Practice of Software Security and Performance Verification

Security and performance are critical nonfunctional requirements for software systems. Thus, it is crucial to include verification activities during software development to identify defects related to such requirements, avoiding their occurrence after release. Software verification, including testing and reviews, encompasses a set of activities that have a purpose of analyzing the software searching for defects. Security and performance verification are activities that look at defects related to these specific quality attributes. Few empirical studies have been focused on how is the state of the practice in security and performance verification. This paper presents the results of a case study performed in the context of Brazilian organizations aiming to characterize security and performance verification practices. Additionally, it provides a set of conjectures indicating recommendations to improve security and performance verification activities.acceptedVersio

A survey on quality attributes in servicebased systems

Service-based systems have become popular in the software industry. In software engineering, it is widely acknowledged that requirements on quality attributes (e.g., performance, security, reliability) significantly impact the design of software systems. This study explores the role of quality attributes during the design of service-based systems. We investigate the significance of quality attributes when designing service-based systems and how quality attributes are addressed through design decisions, across application domains, and related to other aspects of software development, e.g., architecture documentation. We conducted a descriptive survey. The survey was done as an online questionnaire targeting practitioners. Furthermore, we included researchers with practical design experience. We obtained 56 valid responses. Most survey participants consider quality attributes and functionality as equally important and treat quality attributes explicitly rather than implicitly. Furthermore, dependability is the most relevant quality attribute in service-based systems; we do not find quality attributes that are particularly important in specific application domains. Most quality attributes are addressed by ad hoc decisions, rather than established architecture or design patterns or technologies. Only few decision alternatives are considered when making architectural decisions to address quality attributes. Our results partially confirm anecdotal evidence from current literature, but also strengthen previous claims by providing empirical evidence. Our results point to future research directions (e.g., exploring the impact of decision types on how well quality attributes can be achieved) and implications for practitioners (e.g., training makes a difference to how quality attributes are treated).Peer ReviewedPostprint (author’s final draft

A Model for Software Quality Evaluation Using the User's Point of Views

Contexte: Dans le marché des logiciels en constante évolution, les acheteurs de logiciels sont confrontés à un défi majeur: parmi ces différents produits, lequel répond le mieux aux exigences et au budget des utilisateurs? Bien que la plupart des acheteurs de logiciels soient conscients de leurs besoins fonctionnels et budgétaires, les facteurs de qualité tels que la « disponibilité » ou la «fiabilité » ne sont généralement pas pris en compte. Les fournisseurs de logiciels parlent aussi rarement des aspects de qualité de leurs produits. Le défi principal est « comment susciter les attentes de qualité des utilisateurs ? », puis « comment déterminer les caractéristiques de qualité d'un produit logiciel ?». La comparaison de ces deux facteurs de qualité peut aider les acheteurs de logiciels à sélectionner le produit le mieux adapté et à ne pas gaspiller de budget supplémentaire pour des facteurs de qualité inutiles et ne pas acheter un produit qui ne couvre pas leurs exigences de qualité. Existe-t-il une méthode systématique pour rendre cette comparaison possible? Quels sont les facteurs qui influent sur la perception de l'utilisateur de la qualité du produit logiciel? Objectif: Dans cette thèse, nous visons à aborder la qualité du produit logiciel du point de vue des utilisateurs. L'objectif est de créer le profil de qualité attendu et observé du produit logiciel afin de démontrer les différences entre les qualités attendues par les utilisateurs et quelles qualités sont observées chez le produit logiciel. Méthode: Nous avons utilisé une stratégie empirique en utilisant une méthode basée sur un sondage pour créer le profil des caractéristiques de qualité attendues et observées. Après avoir développé un modèle de qualité standard, nous avons créé quatre types de questionnaires pour les utilisateurs finaux et les utilisateurs expérimentés, qui visent à susciter les facteurs de qualité. À titre d'étude de cas, nous avons mené trois enquêtes en deux phases dans l'industrie. Dans la phase I, nous avons demandé aux utilisateurs potentiels d'un produit logiciel en cours de développement, de répondre au questionnaire «qualité attendue». Dans la phase II, les utilisateurs d'un produit logiciel existant ont été invités à répondre au questionnaire «qualité observée». Résultats: Les résultats de la première phase montrent qu'il n'y a pas de différence significative entre les attentes de qualité de groupe d'utilisateurs final (end user) et d'utilisateurs expérimentés (power user). Dans la phase II, les résultats révèlent que les utilisateurs du département de développement, connus comme utilisateurs techniquement compétents, trouvent le logiciel plus performant que les utilisateurs d'autres départements de l'entreprise. Conclusion: Avec des profils de plan de qualité en main, il est possible d'effectuer un contrôle croisé utile entre les attentes de qualité spécifiques des utilisateurs et d'autres pilotes (exigences fonctionnelles et architecture / conception), avant ou pendant le processus de développement de logiciels. Le contrôle croisé devrait viser à garantir qu'il existe suffisamment d'activités et de sous-activités dans le processus de développement de logiciels pour répondre aux attentes de qualité des utilisateurs. Sur la base des enquêtes menées, nous concluons que la qualité du logiciel du point de vue des utilisateurs dépend de la connaissance des utilisateurs sur les développements et la qualité de la technologie, en général, et sur le produit logiciel à l'étude, en particulier.----------ABSTRACT : Context: In the ever-evolving software market, software buyers face a central challenge: Among these various products, which one best meets the users’ requirements and budget? While most software buyers are aware of their functional and budgetary requirements, quality factors such as ‘availability’ or ‘reliability’ are not usually taken into account. Software vendors also rarely talk about the quality aspects of their products. The primary challenge is “how to elicit the users’ quality expectations”, and then “how to determine the quality characteristics of a software product”. Comparing these two can assist the software buyers to select the best-fit product; not to waste extra budget for unnecessary quality factors, and not buy a product that does not cover their quality requirements. Is there any systematic method to make this comparison possible? What are the influencing factors that affect the user’s perception of the software product quality? Objective: In this thesis, we aim to address the quality of the software product from the users’ point of view. The goal is to create the expected and observed quality profile of the software product to demonstrate the differences between what qualities were expected from the users’ side, and what qualities are observed in the software product. Method: We employed an empirical strategy using a surveybased method to create the profile of expected and observed quality characteristics. After developing a standard-based quality model, we created four types of questionnaires for end users and power users, which aim to elicit the quality factors. As a case study, we conducted three surveys in two phases in the industry. In phase I, we asked the potential users of a software product which was under development, to answer the ‘expected-quality’ questionnaire. In phase II, the users of an existing software product were asked to answer the ‘observed-quality’ questionnaire. Results: The results of the first phase show that there is no significant difference between the quality expectations of the end and power user groups. In phase II, the results reveal that the users in development department who are known as technically knowledgeable users find the software as higher quality than the users in other departments of the company. Conclusion: With quality plan profiles in hand, it is possible to perform a useful crosscheck between users’ specific quality expectations and other drivers (functional and architecture/design requirements), before or during the software development process. The crosscheck should be aimed to guarantee that there are enough activities and sub-activities in the software development process to support the users’ quality expectations. Based on the conducted surveys, we conclude that the software quality from the users’ point of view depends on the knowledge of the users about the software developments and quality, in general, and on the software product under study, specifically

Handling non-functional requirements in Model-Driven Development: an ongoing industrial survey

Model-Driven Development (MDD) is no longer a novel development paradigm. It has become mature from a research perspective and recent studies show its adoption in industry. Still, some issues remain a challenge. Among them, we are interested in the treatment of non-functional requirements (NFRs) in MDD processes. Very few MDD approaches have been reported to deal with NFRs (and they do it in a limited way). However, it is clear that NFRs need to be considered somehow in the final product of the MDD process. To better understand how NFRs are integrated into the existing MDD approaches, we have initiated the NFR4MDD project, a multi-national empirical study, based on interviews with companies working on MDD projects. Our project aims at surveying the state of the practice for this topic. In this paper, we summarize our research protocol and present the current status of our study. The discussion will focus on the peculiarities of our study’s context and organization involving about 20 researchers from 8 European countriesPeer ReviewedPostprint (author's final draft

Using Ethical Guidelines for Defining Critical Quality Requirements of AI Solutions

Artificial Intelligence (AI) has become a fast-growing trend and an increasing number of companies are interested in developing AI solutions. Many companies have realised that black box nature of AI technologies can raise ethical questions. And they are defining ethical guidelines which can be utilised when developing AI solutions. The goal of this thesis was to explore how companies can solve potential ethical issues of AI solutions during requirements engineering activities. The main research method used in this thesis was a multiple-case study approach. Semi-structured interviews and document analysis were carried out to discover the AI ethical guidelines in three Finnish companies. The results are based on the open coding of the interview data and the ethical guidelines of the case companies. The ethical guidelines developed by the case companies focus on solving potential ethical issues like accountability, fairness, privacy, safety, security, transparency and trust. All three case companies devised ethical guideline relating to fairness, transparency and trust. To capture different views on critical ethical issues, two of the case companies recommend using multi-disciplinary development teams consisting of experts like data scientists, designers, and sociologists. The results of this thesis suggest that companies can utilise the ethical guidelines of AI solutions to identify and prioritise the essential quality requirements of AI solutions