A reference architecture for big data systems

Over dozens of years, applying new IT technologies into organizations has always been a big concern for business. Big data certainly is a new concept exciting business. To be able to access more data and empower to analysis big data requires new big data platforms. However, there still remains limited reference architecture for big data systems. In this paper, based on existing reference architecture of big data systems, we propose new high level abstract reference architecture and related reference architecture notations, that better express the overall architecture. The new reference architecture is verified using one existing case and an additional new use case

A mapping study on documentation in Continuous Software Development

Context: With an increase in Agile, Lean, and DevOps software methodologies over the last years (collectively referred to as Continuous Software Development (CSD)), we have observed that documentation is often poor. Objective: This work aims at collecting studies on documentation challenges, documentation practices, and tools that can support documentation in CSD. Method: A systematic mapping study was conducted to identify and analyze research on documentation in CSD, covering publications between 2001 and 2019. Results: A total of 63 studies were selected. We found 40 studies related to documentation practices and challenges, and 23 studies related to tools used in CSD. The challenges include: informal documentation is hard to understand, documentation is considered as waste, productivity is measured by working software only, documentation is out-of-sync with the software and there is a short-term focus. The practices include: non-written and informal communication, the usage of development artifacts for documentation, and the use of architecture frameworks. We also made an inventory of numerous tools that can be used for documentation purposes in CSD. Overall, we recommend the usage of executable documentation, modern tools and technologies to retrieve information and transform it into documentation, and the practice of minimal documentation upfront combined with detailed design for knowledge transfer afterwards. Conclusion: It is of paramount importance to increase the quantity and quality of documentation in CSD. While this remains challenging, practitioners will benefit from applying the identified practices and tools in order to mitigate the stated challenges

Representing Variability in Software Architecture

Software Architecture is a high level description of a software intensive system that enables architects to have a better intellectual control over the complete system. It is also used as a communication vehicle among the various system stakeholders. Variability in software-intensive systems is the ability of a software artefact (e.g., a system, subsystem, or component) to be extended, customised, or configured for deployment in a specific context. Although variability in software architecture is recognised as a challenge in multiple domains, there has been no formal consensus on how variability should be captured or represented. In this research, we addressed the problem of representing variability in software architecture through a three phase approach. First, we examined existing literature using the Systematic Literature Review (SLR) methodology, which helped us identify the gaps and challenges within the current body of knowledge. Equipped with the findings from the SLR, a set of design principles have been formulated that are used to introduce variability management capabilities to an existing Architecture Description Language (ADL). The chosen ADL was developed within our research group (ALI) and to which we have had complete access. Finally, we evaluated the new version of the ADL produced using two distinct case studies: one from the Information Systems domain, an Asset Management System (AMS); and another from the embedded systems domain, a Wheel Brake System (WBS). This thesis presents the main findings from the three phases of the research work, including a comprehensive study of the state-of-the-art; the complete specification of an ADL that is focused on managing variability; and the lessons learnt from the evaluation work of two distinct real-life case studies

Understanding, Analysis, and Handling of Software Architecture Erosion

Architecture erosion occurs when a software system's implemented architecture diverges from the intended architecture over time. Studies show erosion impacts development, maintenance, and evolution since it accumulates imperceptibly. Identifying early symptoms like architectural smells enables managing erosion through refactoring. However, research lacks comprehensive understanding of erosion, unclear which symptoms are most common, and lacks detection methods. This thesis establishes an erosion landscape, investigates symptoms, and proposes identification approaches. A mapping study covers erosion definitions, symptoms, causes, and consequences. Key findings: 1) "Architecture erosion" is the most used term, with four perspectives on definitions and respective symptom types. 2) Technical and non-technical reasons contribute to erosion, negatively impacting quality attributes. Practitioners can advocate addressing erosion to prevent failures. 3) Detection and correction approaches are categorized, with consistency and evolution-based approaches commonly mentioned.An empirical study explores practitioner perspectives through communities, surveys, and interviews. Findings reveal associated practices like code review and tools identify symptoms, while collected measures address erosion during implementation. Studying code review comments analyzes erosion in practice. One study reveals architectural violations, duplicate functionality, and cyclic dependencies are most frequent. Symptoms decreased over time, indicating increased stability. Most were addressed after review. A second study explores violation symptoms in four projects, identifying 10 categories. Refactoring and removing code address most violations, while some are disregarded.Machine learning classifiers using pre-trained word embeddings identify violation symptoms from code reviews. Key findings: 1) SVM with word2vec achieved highest performance. 2) fastText embeddings worked well. 3) 200-dimensional embeddings outperformed 100/300-dimensional. 4) Ensemble classifier improved performance. 5) Practitioners found results valuable, confirming potential.An automated recommendation system identifies qualified reviewers for violations using similarity detection on file paths and comments. Experiments show common methods perform well, outperforming a baseline approach. Sampling techniques impact recommendation performance

Techniques and Practices for Software Architecture Work in Agile Software Development

Since the publishing of Agile Manifesto in 2001, the agile software development has taken the world by storm. Agile software development does not emphasize the importance of software architecture, even though software architecture is often seen as a key factor for achieving the quality goals set for the software system. It has been even said that agile software development and software architecture are a clash of two cultures.In many software projects there is no need to consider software architecture anymore. For example, when designing a mobile application, the ecosystem forces the developer to use certain architecture style provided by the platform. In web development ready-made frameworks and project templates are available offering complete software architecture designs for the application developer.There are still domains and systems where careful architecture design is needed. When developing complex systems or systems with a long lifecycle, diligent software architecture design is a key to avoid massive rework during the development. It cannot be a coincidence that companies developing these kinds of systems struggle with agile software development the most.On the one hand, the goal of this thesis was to study software architecture practices used in agile software development in the industry. On the other hand, the goal was to develop new methods and techniques to support incremental software architecture working practices which can be aligned with agile methods such as Scrum. The study applied case study, interviews and design science as the main research methods.The results show that there are four main ways to carry out software architecture work while using agile methods. Basing on this result, models for aligning software architec-ture knowledge management were developed. These models can be used as guidelines for selecting the appropriate software architecture practices in an organization.As a part of the research work, an architecture knowledge repository was developed for sharing the knowledge in agile projects and for automatic software architecture document generation. Additionally, the results of this study show that by taking a decision-centric approach to software architecture evaluation, the evaluation method can be lightweight enough to make incremental evaluation a viable option. Similarly, existing software architecture evaluation methods can be boosted to fit agile software development by utilizing domain knowledge

Conception architecturale des systèmes robotiques orientée services

Robotics has experienced an increasing evolution and interest from the society in recent years. Robots are no longer produced exclusively to perform repetitive tasks in factories, they have been designed to collaborate with humans in several important application domains. Robotic systems that control these robots are therefore becoming larger, more complex, and difficult to develop. In this scenario, Service-Oriented Architecture (SOA) has been investigated as a promising architectural style for the design of robotic systems in a flexible, reusable, and productive manner. Despite the existence of a considerable amount of Service-Oriented Robotic Systems (SORS), most of them have been developed in an ad hoc manner. The little attention and limited support devoted to the design of SORS software architectures may not only hamper the benefits of SOA adoption, but also reduce the overall quality of robotic systems, which are often used in safety-critical contexts. This thesis aims at improving the understanding and systematization of SORS architectural design.La Robotique a connu une évolution remarquable au cours des dernières années, couplée à un intérêt croissant de la société pour ce domaine. Les robots ne sont plus fabriqués exclusivement pour effectuer des tâches répétitives dans les usines, mais ils sont aussi créés pour collaborer avec les humains dans plusieurs domaines d'application d'importance. Les systèmes robotiques qui contrôlent ces robots sont donc de plus en plus larges, complexes et difficiles à développer. Dans ce contexte, l'Architecture Orientée Services (SOA) a été identifiée comme un style d'architecture logicielle prometteur pour concevoir des systèmes robotiques de manière flexible, réutilisable et productive. Cependant, malgré le nombre considérable de Systèmes Robotiques Orientées Services (SORS) existants aujourd'hui, la plupart d'entre eux ont été développés de manière ad hoc. Le peu d'attention et le soutien limité portés à la conception d'architectures logicielles SORS peuvent non seulement masquer les avantages de l'adoption de la SOA, mais aussi réduire la qualité globale des systèmes robotiques, qui sont souvent utilisés dans des contextes de sécurité critiques. Cette thèse vise à améliorer la compréhension et la systématisation de la conception architecturale SORS. Elle décrit une taxonomie des services pour le domaine de la robotique, puis propose un processus ainsi qu'une architecture de référence afin de systématiser la conception d'architectures logicielles SORS. Les résultats obtenus dans les études d'évaluation montrent qu'à la fois le processus et l'architecture de référence peuvent avoir un impact positif sur la qualité des architectures logicielles SORS et, par conséquent, contribuent à l'amélioration des systèmes robotique

Supporting the grow-and-prune model for evolving software product lines

207 p.Software Product Lines (SPLs) aim at supporting the development of a whole family of software products through a systematic reuse of shared assets. To this end, SPL development is separated into two interrelated processes: (1) domain engineering (DE), where the scope and variability of the system is defined and reusable core-assets are developed; and (2) application engineering (AE), where products are derived by selecting core assets and resolving variability. Evolution in SPLs is considered to be more challenging than in traditional systems, as both core-assets and products need to co-evolve. The so-called grow-and-prune model has proven great flexibility to incrementally evolve an SPL by letting the products grow, and later prune the product functionalities deemed useful by refactoring and merging them back to the reusable SPL core-asset base. This Thesis aims at supporting the grow-and-prune model as for initiating and enacting the pruning. Initiating the pruning requires SPL engineers to conduct customization analysis, i.e. analyzing how products have changed the core-assets. Customization analysis aims at identifying interesting product customizations to be ported to the core-asset base. However, existing tools do not fulfill engineers needs to conduct this practice. To address this issue, this Thesis elaborates on the SPL engineers' needs when conducting customization analysis, and proposes a data-warehouse approach to help SPL engineers on the analysis. Once the interesting customizations have been identified, the pruning needs to be enacted. This means that product code needs to be ported to the core-asset realm, while products are upgraded with newer functionalities and bug-fixes available in newer core-asset releases. Herein, synchronizing both parties through sync paths is required. However, the state of-the-art tools are not tailored to SPL sync paths, and this hinders synchronizing core-assets and products. To address this issue, this Thesis proposes to leverage existing Version Control Systems (i.e. git/Github) to provide sync operations as first-class construct

Automated Reliability Prediction and Analysis from Software Architectures

Tese de doutoramento em Ciências e Tecnologias da Informação, apresentada ao Departamento de Engenharia Informática da Faculdade de Ciências e Tecnologias da Universidade de CoimbraThe quality of a software is determined by how it meets non-functional requirements such as performance, reliability, availability, maintainability and other ‘-ilities’. Depending on the application context, certain qualities are more critical to attain than others. As an example, a web-server processing large amounts of data should present qualities regarding to performance, while a software applied to a medical context must assure that no human life is at risk and as such, should comply to safety as a quality requirement. In a software engineering perspective, quality requirements should be assessed throughout the software development life-cycle. In an early stage, quality assessment supports design decisions and promotes analysis of possible alternatives. During the implementation or testing stages, project managers may confirm that the design meets the developed product and assure that it will conform to the stakeholder’s requirements. Regarding evolutionary stages, architects can also compare different designs and decide for the most suitable solution ta- king into account the desired quality attributes. During the development of a software system, neglecting the assessment of the quality requirements may lead, sooner or later, to the developed product failing to achieve one or more non-functional attributes desired by the stakeholder. Consequently, the development process returns to a previous phase for re-designing, re-implement and re-test a new solution to solve the problem. In short, it will involve more time, effort and money, causing more costs to the whole software project. Software architecture plays an important role in the achievement of non-functional attributes. Designers use architectures to codify non- functional properties and employ good design practices. In addition, it allows to maintain the traceability of the project through its lifespan and also serve as a form of communication between stakeholders, developers and maintainers. Software architectures can be considered one of the first documents in the project to structure the system, since allows to describe the development plans and specify rules, properties and architectural styles to attain specific quality attributes. For these reasons, the existent techniques to assess quality attributes use software architectures to obtain information about the system and provide accurate quantitative results. The problem addressed by this thesis resides in the fact that in today’s world most of the methods to assess quality attributes from a software architecture are still manually performed. To quantitatively assess an architecture’s quality attribute, designers have to build mathematical models through manual tasks and rebuild them for every change per- formed in the architecture. As any other manual task, building such models is error-prone, time consuming and can be almost unfeasible for large and complex systems. With this in mind, this thesis proposes to fill a gap in research by investigating towards a method that automatically assesses the reli- ability as a quality attribute from a software architecture. In particular, we exploit the formalisms of Architecture Description Languages (ADLs) to automatically generate mathematical models ex- pressing the reliability behavior of a system. Then, we extended the notion of ‘automated assessment’ to perform a thorough analysis to identify architectural weak points that are affecting the system. This analysis aims to provide information for architects about reliability improvements and suggest alternatives. With the goal of providing an assisting tool to aid architects in the design process, we implemented a plugin integrated in a ADL design tool. This plugin aims to make our automated approach available for architects to test and analyze their designs regarding reliability. In addition, we showed the different application contexts of our approach by including it in the reasoning process of self-adaptive systems. The results showed an improvement in the overall system quality when comparing to the traditional planning approaches. To conclude, we validated our method through a set of experiments that put into comparison our method with others that used manual approaches to assess reliability. In this work we pursue the motivation of contributing with a set of methods to give support for practitioners and researchers to avoid, prevent and detect undesired or unfeasible architectural designs. Moreover, we intend to promote the development of software with better quality and assure that it meets the desired quality requirements during the development process.FCT - SFRH/BD/89702/201

Technical Debt Decision-Making Framework

Software development companies strive to produce high-quality software. In commercial software development environments, due to resource and time constraints, software is often developed hastily which gives rise to technical debt. Technical debt refers to the consequences of taking shortcuts when developing software. These consequences include making the system difficult to maintain and defect prone. Technical debt can have financial consequences and impede feature enhancements. Identifying technical debt and deciding which debt to address is challenging given resource constraints. Project managers must decide which debt has the highest priority and is most critical to the project. This decision-making process is not standardized and sometimes differs from project to project. My research goal is to develop a framework that project managers can use in their decision-making process to prioritize technical debt based on its potential impact. To achieve this goal, we survey software practitioners, conduct literature reviews, and mine software repositories for historical data to build a framework to model the technical debt decision-making process and inform practitioners of the most critical debt items