Open source software ecosystems quality analysis from data sources

Background: Open source software (OSS) and software ecosystems (SECOs) are two consolidated research areas in software engineering. The adoption of OSS by firms, governments, researchers and practitioners has been increasing rapidly in the last decades, and in consequence, they find themselves in a new kind of ecosystem composed by software communities,foundations, developers and partners, namely Open Source Software Ecosystem (OSSECO). In order to perform a systematic quality evaluation of a SECO, it is necessary to define certain types of concrete elements. This means that measures and evaluations should be described (e.g., through thresholds or expert judgment). The quality evaluation of an OSSECO may serve several purposes, for example: adopters of the products of the OSSECO may want to know about the liveliness of the OSSECO (e.g., recent updates); software developers may want to know about the activeness (e.g., how many collaborators are involved and how active they are); and the OSSECO community itself to know about the OSSECO health (e.g., evolving in the right direction). However, the current approaches for evaluating software quality (even those specific for open source software) do not cover all the aspects relevant in an OSSECO from an ecosystem perspective. Goal: The main goal of this PhD thesis is to support the OSSECO quality evaluation by designing a framework that supports the quality evaluation of OSSECOs. Methods: To accomplish this goal, we have used and approach based on design science methodology by Wieringa [1] and the characterization of software engineering proposed by M. Shaw [2], in order to produce a set of artefacts to contribute in thequality evaluation of OSSECOs and to learn about the effects of using these artefacts in practice. Results: We have conducted a systematic mapping to characterize OSSECOs and designed the QuESo framework (a framework to evaluate the OSSECO quality) composed by three artifacts: (i) QuESo-model, a quality model for OSSECOs; (ii) QuESoprocess, a process for conducting OSSECO quality evaluations using the QuESo-model; and (iii) QuESo-tool, a software component to support semi-automatic quality evaluation of OSSECOs. Furthermore, this framework has been validated with a case study on Eclipse. Conclusions: This thesis has contributed to increase the knowledge and understanding of OSSECOs, and to support the qualityevaluation of OSSECOs. [ntecedentes: el software de código abierto (OSS, por sus siglas en inglés) y los ecosistemas de software (SECOs, por sus siglas en inglés) son dos áreas de investigación consolidadas en ingeniería de software. La adopción de OSS por parte de empresas, gobiernos, investigadores y profesionales se ha incrementado rápidamente en las últimas décadas, y, en consecuencia, todos ellos hacen parte de un nuevo tipo de ecosistema formado por comunidades de software, fundaciones, desarrolladores y socios denominado ecosistema de software de código abierto. (OSSECO, por sus siglas en inglés)). Para realizar una evaluación sistemática de la calidad de un SECO, es necesario definir ciertos tipos de elementos concretos. Esto significa que tanto las métricas como las evaluaciones deben ser descritos (por ejemplo, a través de datos históricos o el conocimiento de expertos). La evaluación de la calidad de un OSSECO puede ser de utilidad desde diferentes perspectivas, por ejemplo: los que adoptan los productos del OSSECO pueden querer conocer la vitalidad del OSSECO (por ejemplo, el número de actualizaciones recientes); los desarrolladores de software pueden querer saber sobre la actividad del OSSECO (por ejemplo, cuántos colaboradores están involucrados y qué tan activos son); incluso la propia comunidad del OSSECO para conocer el estado de salud del OSSECO (por ejemplo, si está evolucionando en la dirección correcta). Sin embargo, los enfoques actuales para evaluar la calidad del software (incluso aquellos específicos para el software de código abierto) no cubren todos los aspectos relevantes en un OSSECO desde una perspectiva ecosistémica. Objetivo: El objetivo principal de esta tesis doctoral es apoyar la evaluación de la calidad de OSSECO mediante el diseño de un marco de trabajo que ayude a la evaluación de la calidad de un OSSECO. Métodos: Para lograr este objetivo, hemos utilizado un enfoque basado en la metodología design science propuesta por Wieringa [1]. Adicionalmente, nos hemos basado en la caracterización de la ingeniería de software propuesta por M. Shaw [2], con el fin de construir un conjunto de artefactos que contribuyan en la evaluación de la calidad de un OSSECO y para conocer los efectos del uso de estos artefactos en la práctica. Resultados: Hemos realizado un mapeo sistemático para caracterizar los OSSECOs y hemos diseñado el marco de trabajo denominado QuESo (es un marco de trabajo para evaluar la calidad de los OSSECOs). QuESo a su vez está compuesto por tres artefactos: (i) QuESo-model, un modelo de calidad para OSSECOs; (ii) QuESo-process, un proceso para llevar a cabo las evaluaciones de calidad de OSSECOs utilizando el modelo QuESo; y (iii) QuESo-tool, un conjunto de componentes de software que apoyan la evaluación de calidad de los OSSECOs de manera semiautomática. QuESo ha sido validado con un estudio de caso sobre Eclipse. Conclusiones: esta tesis ha contribuido a aumentar el conocimiento y la comprensión de los OSSECOs, y tambien ha apoyado la evaluación de la calidad de los OSSECOsPostprint (published version

Open source software ecosystems quality analysis from data sources

Background: Open source software (OSS) and software ecosystems (SECOs) are two consolidated research areas in software engineering. The adoption of OSS by firms, governments, researchers and practitioners has been increasing rapidly in the last decades, and in consequence, they find themselves in a new kind of ecosystem composed by software communities,foundations, developers and partners, namely Open Source Software Ecosystem (OSSECO). In order to perform a systematic quality evaluation of a SECO, it is necessary to define certain types of concrete elements. This means that measures and evaluations should be described (e.g., through thresholds or expert judgment). The quality evaluation of an OSSECO may serve several purposes, for example: adopters of the products of the OSSECO may want to know about the liveliness of the OSSECO (e.g., recent updates); software developers may want to know about the activeness (e.g., how many collaborators are involved and how active they are); and the OSSECO community itself to know about the OSSECO health (e.g., evolving in the right direction). However, the current approaches for evaluating software quality (even those specific for open source software) do not cover all the aspects relevant in an OSSECO from an ecosystem perspective. Goal: The main goal of this PhD thesis is to support the OSSECO quality evaluation by designing a framework that supports the quality evaluation of OSSECOs. Methods: To accomplish this goal, we have used and approach based on design science methodology by Wieringa [1] and the characterization of software engineering proposed by M. Shaw [2], in order to produce a set of artefacts to contribute in thequality evaluation of OSSECOs and to learn about the effects of using these artefacts in practice. Results: We have conducted a systematic mapping to characterize OSSECOs and designed the QuESo framework (a framework to evaluate the OSSECO quality) composed by three artifacts: (i) QuESo-model, a quality model for OSSECOs; (ii) QuESoprocess, a process for conducting OSSECO quality evaluations using the QuESo-model; and (iii) QuESo-tool, a software component to support semi-automatic quality evaluation of OSSECOs. Furthermore, this framework has been validated with a case study on Eclipse. Conclusions: This thesis has contributed to increase the knowledge and understanding of OSSECOs, and to support the qualityevaluation of OSSECOs. [ntecedentes: el software de código abierto (OSS, por sus siglas en inglés) y los ecosistemas de software (SECOs, por sus siglas en inglés) son dos áreas de investigación consolidadas en ingeniería de software. La adopción de OSS por parte de empresas, gobiernos, investigadores y profesionales se ha incrementado rápidamente en las últimas décadas, y, en consecuencia, todos ellos hacen parte de un nuevo tipo de ecosistema formado por comunidades de software, fundaciones, desarrolladores y socios denominado ecosistema de software de código abierto. (OSSECO, por sus siglas en inglés)). Para realizar una evaluación sistemática de la calidad de un SECO, es necesario definir ciertos tipos de elementos concretos. Esto significa que tanto las métricas como las evaluaciones deben ser descritos (por ejemplo, a través de datos históricos o el conocimiento de expertos). La evaluación de la calidad de un OSSECO puede ser de utilidad desde diferentes perspectivas, por ejemplo: los que adoptan los productos del OSSECO pueden querer conocer la vitalidad del OSSECO (por ejemplo, el número de actualizaciones recientes); los desarrolladores de software pueden querer saber sobre la actividad del OSSECO (por ejemplo, cuántos colaboradores están involucrados y qué tan activos son); incluso la propia comunidad del OSSECO para conocer el estado de salud del OSSECO (por ejemplo, si está evolucionando en la dirección correcta). Sin embargo, los enfoques actuales para evaluar la calidad del software (incluso aquellos específicos para el software de código abierto) no cubren todos los aspectos relevantes en un OSSECO desde una perspectiva ecosistémica. Objetivo: El objetivo principal de esta tesis doctoral es apoyar la evaluación de la calidad de OSSECO mediante el diseño de un marco de trabajo que ayude a la evaluación de la calidad de un OSSECO. Métodos: Para lograr este objetivo, hemos utilizado un enfoque basado en la metodología design science propuesta por Wieringa [1]. Adicionalmente, nos hemos basado en la caracterización de la ingeniería de software propuesta por M. Shaw [2], con el fin de construir un conjunto de artefactos que contribuyan en la evaluación de la calidad de un OSSECO y para conocer los efectos del uso de estos artefactos en la práctica. Resultados: Hemos realizado un mapeo sistemático para caracterizar los OSSECOs y hemos diseñado el marco de trabajo denominado QuESo (es un marco de trabajo para evaluar la calidad de los OSSECOs). QuESo a su vez está compuesto por tres artefactos: (i) QuESo-model, un modelo de calidad para OSSECOs; (ii) QuESo-process, un proceso para llevar a cabo las evaluaciones de calidad de OSSECOs utilizando el modelo QuESo; y (iii) QuESo-tool, un conjunto de componentes de software que apoyan la evaluación de calidad de los OSSECOs de manera semiautomática. QuESo ha sido validado con un estudio de caso sobre Eclipse. Conclusiones: esta tesis ha contribuido a aumentar el conocimiento y la comprensión de los OSSECOs, y tambien ha apoyado la evaluación de la calidad de los OSSECO

Interpersonal Conflicts During Code Review

Code review consists of manual inspection, discussion, and judgment of source code by developers other than the code's author. Due to discussions around competing ideas and group decision-making processes, interpersonal conflicts during code reviews are expected. This study systematically investigates how developers perceive code review conflicts and addresses interpersonal conflicts during code reviews as a theoretical construct. Through the thematic analysis of interviews conducted with 22 developers, we confirm that conflicts during code reviews are commonplace, anticipated and seen as normal by developers. Even though conflicts do happen and carry a negative impact for the review, conflicts-if resolved constructively-can also create value and bring improvement. Moreover, the analysis provided insights on how strongly conflicts during code review and its context (i.e., code, developer, team, organization) are intertwined. Finally, there are aspects specific to code review conflicts that call for the research and application of customized conflict resolution and management techniques, some of which are discussed in this paper. Data and material: https://doi.org/10.5281/zenodo.584879

Towards an Intelligent System for Software Traceability Datasets Generation

Software datasets and artifacts play a crucial role in advancing automated software traceability research. They can be used by researchers in different ways to develop or validate new automated approaches. Software artifacts, other than source code and issue tracking entities, can also provide a great deal of insight into a software system and facilitate knowledge sharing and information reuse. The diversity and quality of the datasets and artifacts within a research community have a significant impact on the accuracy, generalizability, and reproducibility of the results and consequently on the usefulness and practicality of the techniques under study. Collecting and assessing the quality of such datasets are not trivial tasks and have been reported as an obstacle by many researchers in the domain of software engineering. In this dissertation, we report our empirical work that aims to automatically generate and assess the quality of such datasets. Our goal is to introduce an intelligent system that can help researchers in the domain of software traceability in obtaining high-quality “training sets”, “testing sets” or appropriate “case studies” from open source repositories based on their needs. In the first project, we present a first-of-its-kind study to review and assess the datasets that have been used in software traceability research over the last fifteen years. It presents and articulates the current status of these datasets, their characteristics, and their threats to validity. Second, this dissertation introduces a Traceability-Dataset Quality Assessment (T-DQA) framework to categorize software traceability datasets and assist researchers to select appropriate datasets for their research based on different characteristics of the datasets and the context in which those datasets will be used. Third, we present the results of an empirical study with limited scope to generate datasets using three baseline approaches for the creation of training data. These approaches are (i) Expert-Based, (ii) Automated Web-Mining, which generates training sets by automatically mining tactic\u27s APIs from technical programming websites, and lastly, (iii) Automated Big-Data Analysis, which mines ultra-large-scale code repositories to generate training sets. We compare the trace-link creation accuracy achieved using each of these three baseline approaches and discuss the costs and benefits associated with them. Additionally, in a separate study, we investigate the impact of training set size on the accuracy of recovering trace links. Finally, we conduct a large-scale study to identify which types of software artifacts are produced by a wide variety of open-source projects at different levels of granularity. Then we propose an automated approach based on Machine Learning techniques to identify various types of software artifacts. Through a set of experiments, we report and compare the performance of these algorithms when applied to software artifacts. Finally, we conducted a study to understand how software traceability experts and practitioners evaluate the quality of their datasets. In addition, we aim at gathering experts’ opinions on all quality attributes and metrics proposed by T-DQA

IDENTIFICATION AND QUANTIFICATION OF VARIABILITY MEASURES AFFECTING CODE REUSABILITY IN OPEN SOURCE ENVIRONMENT

Open source software (OSS) is one of the emerging areas in software engineering, and is gaining the interest of the software development community. OSS was started as a movement, and for many years software developers contributed to it as their hobby (non commercial purpose). Now, OSS components are being reused in CBSD (commercial purpose). However, recently, the use of OSS in SPL is envisioned recently by software engineering researchers, thus bringing it into a new arena. Being an emerging research area, it demands exploratory study to explore the dimensions of this phenomenon. Furthermore, there is a need to assess the reusability of OSS which is the focal point of these disciplines (CBSE, SPL, and OSS). In this research, a mixed method based approach is employed which is specifically 'partially mixed sequential dominant study'. It involves both qualitative (interviews) and quantitative phases (survey and experiment). During the qualitative phase seven respondents were involved, sample size of survey was 396, and three experiments were conducted. The main contribution of this study is results of exploration of the phenomenon 'reuse of OSS in reuse intensive software development'. The findings include 7 categories and 39 dimensions. One of the dimension factors affecting reusability was carried to the quantitative phase (survey and experiment). On basis of the findings, proposal for reusability attribute model was presented at class and package level. Variability is one of the newly identified attribute of reusability. A comprehensive theoretical analysis of variability implementation mechanisms is conducted to propose metrics for its assessment. The reusability attribute model is validated by statistical analysis of I 03 classes and 77 packages. An evolutionary reusability analysis of two open source software was conducted, where different versions of software are analyzed for their reusability. The results show a positive correlation between variability and reusability at package level and validate the other identified attributes. The results would be helpful to conduct further studies in this area

QuESo V2.0 a quality model for open source software ecosystems: List of measures

This technical report is part of QuESo-V2.0 a quality model for open source software ecosystems (OSSECOs). Our prior efforts have focused on providing a detailed list of the quality measures found during the execution of a systematic mapping on OSSECOs. In this new version of the model, we addressing some of the issues that were highlighted in the QuESo V1.0 such as: the unbalanced distribution of measures and the ambiguity of some measures names. The measures listed in this report are not intended to be an exhaustive and complete set. However, this list provides a representative collection of OSSECOs measures. It is a small step in the direction of developing a platform for support the analysis of OSSECO.Postprint (published version

A systematic literature review of open source software quality assessment models

Background: Many open source software (OSS) quality assessment models are proposed and available in the litera- ture. However, there is little or no adoption of these models in practice. In order to guide the formulation of newer models so they can be acceptable by practitioners, there is need for clear discrimination of the existing models based on their speci c properties. Based on this, the aim of this study is to perform a systematic literature review to inves- tigate the properties of the existing OSS quality assessment models by classifying them with respect to their quality characteristics, the methodology they use for assessment, and their domain of application so as to guide the formula- tion and development of newer models. Searches in IEEE Xplore, ACM, Science Direct, Springer and Google Search is performed so as to retrieve all relevant primary studies in this regard. Journal and conference papers between the year 2003 and 2015 were considered since the rst known OSS quality model emerged in 2003. Results: A total of 19 OSS quality assessment model papers were selected. To select these models we have devel- oped assessment criteria to evaluate the quality of the existing studies. Quality assessment models are classi ed into ve categories based on the quality characteristics they possess namely: single-attribute, rounded category, community-only attribute, non-community attribute as well as the non-quality in use models. Our study re ects that software selection based on hierarchical structures is found to be the most popular selection method in the existing OSS quality assessment models. Furthermore, we found that majority (47%) of the existing models do not specify any domain of application. Conclusions: In conclusion, our study will be a valuable contribution to the community and helps the quality assess- ment model developers in formulating newer models and also to the practitioners (software evaluators) in selecting suitable OSS in the midst of alternatives