Multilingual investigation of theory-based intervention for program comprehension

This thesis is the continuation of an experiment called “Eye-movement Modeling Examples in Source Code Comprehension: A Classroom Study”. This first experiment studies how effective is showing novice programmers how experts read code with a video with the expert’s gaze guided by a verbal explanation. Therefore, this thesis studies, using a similar experiment, whether only verbal explanation and visual stimuli without the expert’s gaze could be also helpful for the programming novices.Grado en Ingeniería Informática de Servicios y Aplicacione

Exploiting Parts-of-Speech for Effective Automated Requirements Traceability

Context: Requirement traceability (RT) is defined as the ability to describe and follow the life of a requirement. RT helps developers ensure that relevant requirements are implemented and that the source code is consistent with its requirement with respect to a set of traceability links called trace links. Previous work leverages Parts Of Speech (POS) tagging of software artifacts to recover trace links among them. These studies work on the premise that discarding one or more POS tags results in an improved accuracy of Information Retrieval (IR) techniques. Objective: First, we show empirically that excluding one or more POS tags could negatively impact the accuracy of existing IR-based traceability approaches, namely the Vector Space Model (VSM) and the Jensen Shannon Model (JSM). Second, we propose a method that improves the accuracy of IR-based traceability approaches. Method: We developed an approach, called ConPOS, to recover trace links using constraint-based pruning. ConPOS uses major POS categories and applies constraints to the recovered trace links for pruning as a filtering process to significantly improve the effectiveness of IR-based techniques. We conducted an experiment to provide evidence that removing POSs does not improve the accuracy of IR techniques. Furthermore, we conducted two empirical studies to evaluate the effectiveness of ConPOS in recovering trace links compared to existing peer RT approaches. Results: The results of the first empirical study show that removing one or more POS negatively impacts the accuracy of VSM and JSM. Furthermore, the results from the other empirical studies show that ConPOS provides 11%-107%, 8%-64%, and 15%-170% higher precision, recall, and mean average precision (MAP) than VSM and JSM. Conclusion: We showed that ConPosout performs existing IR-based RT approaches that discard some POS tags from the input documents

Collaborative traceability management: a multiple case study from the perspectives of organization, process, and culture

Traceability is crucial for many activities in software and systems engineering including monitoring the development progress, and proving compliance with standards. In practice, the use and maintenance of trace links are challenging as artifacts undergo constant change, and development takes place in distributed scenarios with multiple collaborating stakeholders. Although traceability management in general has been addressed in previous studies, there is a need for empirical insights into the collaborative aspects of traceability management and how it is situated in existing development contexts. The study reported in this paper aims to close this gap by investigating the relation of collaboration and traceability management, based on an understanding of characteristics of the development effort. In our multiple exploratory case study, we conducted semi-structured interviews with 24 individuals from 15 industrial projects. We explored which challenges arise, how traceability management can support collaboration, how collaboration relates to traceability management approaches, and what characteristics of the development effort influence traceability management and collaboration. We found that practitioners struggle with the following challenges: (1) collaboration across team and tool boundaries, (2) conveying the benefits of traceability, and (3) traceability maintenance. If these challenges are addressed, we found that traceability can facilitate communication and knowledge management in distributed contexts. Moreover, there exist multiple approaches to traceability management with diverse collaboration approaches, i.e., requirements-centered, developer-driven, and mixed approaches. While traceability can be leveraged in software development with both agile and plan-driven paradigms, a certain level of rigor is needed to realize its benefits and overcome challenges. To support practitioners, we provide principles of collaborative traceability management. The main contribution of this paper is empirical evidence of how culture, processes, and organization impact traceability management and collaboration, and principles to support practitioners with collaborative traceability management. We show that collaboration and traceability management have the potential to be mutually beneficial—when investing in one, also the other one is positively affected

Collaborative traceability management: a multiple case study from the perspectives of organization, process, and culture

Traceability is crucial for many activities in software and systems engineering including monitoring the development progress, and proving compliance with standards. In practice, the use and maintenance of trace links are challenging as artifacts undergo constant change, and development takes place in distributed scenarios with multiple collaborating stakeholders. Although traceability management in general has been addressed in previous studies, there is a need for empirical insights into the collaborative aspects of traceability management and how it is situated in existing development contexts. The study reported in this paper aims to close this gap by investigating the relation of collaboration and traceability management, based on an understanding of characteristics of the development effort. In our multiple exploratory case study, we conducted semi-structured interviews with 24 individuals from 15 industrial projects. We explored which challenges arise, how traceability management can support collaboration, how collaboration relates to traceability management approaches, and what characteristics of the development effort influence traceability management and collaboration. We found that practitioners struggle with the following challenges: (1) collaboration across team and tool boundaries, (2) conveying the benefits of traceability, and (3) traceability maintenance. If these challenges are addressed, we found that traceability can facilitate communication and knowledge management in distributed contexts. Moreover, there exist multiple approaches to traceability management with diverse collaboration approaches, i.e., requirements-centered, developer-driven, and mixed approaches. While traceability can be leveraged in software development with both agile and plan-driven paradigms, a certain level of rigor is needed to realize its benefits and overcome challenges. To support practitioners, we provide principles of collaborative traceability management. The main contribution of this paper is empirical evidence of how culture, processes, and organization impact traceability management and collaboration, and principles to support practitioners with collaborative traceability management. We show that collaboration and traceability management have the potential to be mutually beneficial—when investing in one, also the other one is positively affected

On the Influence of Representation Type and Gender on Recognition Tasks of Program Comprehension

RÉSUMÉ L’objectif de la maintenance logicielle est d’améliorer les logiciels existants en préservant leur intégrité. La maintenance peut représenter jusqu’à 60% du budget d’un logiciel. Ainsi, améliorer la maintenabilité des logiciels est bénéfique aussi bien pour les fournisseurs que les utilisateurs de logiciels. Les développeurs de logiciels consacrent un effort considérable à la compréhension des programmes, qui est une étape primordiale à la maintenance logicielle. Nous faisons l’hypothèque que le genre des développeurs de logiciels et le type de représentation peut affecter leur effort et leur efficacité. Ces facteurs doivent être considérés et minutieusement analysés dans la mesure où ils peuvent cacher certains effets significatifs pouvant être identifiés en analysant le processus de compréhension. Dans cette thèse, nous nous inspirons de l’utilisation de l’occulomètre pour l’étude du processus cognitif lors de la résolution des problèmes. Nous avons effectué une étude fonctionnelle pour évaluer tous les travaux de recherche faisant usage de l’occulomètre en génie logiciel. Les résultats obtenus nous ont motivé à utiliser l’occulomètre pour effectuer un ensemble d’études afin analyser l’effet de deux facteurs importants sur la compréhension des programmes : le type de représentation (textuelle ou graphique) et le genre du développeur. Afin de comprendre comment les différents types de représentations et le genre influencent les stratégies de visualisation, nous avons étudié la différence de stratégie entre développeurs. Les résultats obtenus montrent que, comparé à une représentation graphique, la représentation sous forme de texte structuré aide mieux le développeur dans son processus cognitif lors de la compréhension des programmes de petite taille. Ainsi, la représentation textuelle requiert moins de temps et d’effort aux participants. Par contre, la représentation graphique est celle préférée par les développeurs. Nos résultats montrent que la structure topologique de la représentation graphique aide les développeurs à mémoriser l’emplacement des éléments et à retrouver plus rapidement les éléments pertinents comparé à la représentation textuelle. En plus, la structure hiérarchique de la représentation graphique guide les développeurs à suivre une stratégie de visualisation spécifique. Nous avons observé que les femmes et les hommes ont des stratégies de visualisation différentes lors de la lecture du code ou de la mémorisation des noms des identificateurs. Les femmes ont tendance à inspecter minutieusement toutes les options afin de procéder à l’élimination de la mauvaise réponse. Au contraire, les hommes ont tendance à inspecter brièvement certaines réponses. Pendant que les femmes consacrent plus de temps à analyser chaque type d’entité l’un après l’autre, les hommes alternent leur attention entre différents type d’entité.----------ABSTRACT The purpose of software maintenance is to correct and enhance an existing software system while preserving its integrity. Software maintenance can cost more than 60% of the budget of a software system, thus improving the maintainability of software is important for both the software industry and its customers. Program comprehension is the initial step of software maintenance that requires the major amount of maintenance’s time and effort. We conjuncture that developers’ gender and the type of representations that developers utilize to perform program comprehension impact their efficiency and effectiveness. These factors must be considered and carefully studied, because they may hide some significant effects to be found by analyzing the comprehension process. In this dissertation, inspired by the literature on the usefulness of eye-trackers to study the cognitive process involved in problem solving activities, we perform a mapping study and evaluate all research relevant to the use of eye-tracking technique in software engineering. The results motivate us to perform a set of eye-tracking studies to analyze the impact of two important factors on program comprehension: representation type (textual vs. graphical) and developers’ gender. Moreover, we investigate and compare viewing strategies variability amongst developers to understand how the representation type and gender differences influence viewing strategies. Overall, our results indicate that structured text provides more cognitive support for developers while performing program comprehension with small systems compared to a graphical representation. Developers spend less time and effort working with textual representations. However, developers mostly preferred to use graphical representations and our results confirm that the topological structure of graphical representations helps developers to memorize the location of the elements and to find the relevant ones faster in comparison with textual representation. Moreover, the hierarchical structure of the representation guides developers to follow specific viewing strategies while working with representations. Regarding the impact of gender, our results emphasize that male and female developers exploit different viewing strategies while reading source code or recalling the names of identifiers. Female developers seem to carefully weigh all options and rule out wrong answers, while male developers seem to quickly set their minds on some answers and move forward. Moreover, female developers spend more time on each source code entity and analyze it before going to the next one. In contrast, male developers utilize several attention switching strategies between different source code entities

Analysing Source Code Structure and Mining Software Repositories to Create Requirements Traceability Links

Resume La traçabilité est le seul moyen de s'assurer que le code source d'un système est conforme aux exigences et que toutes ces exigences et uniquement celles-ci ont été implantées par les développeurs. Lors de la maintenance et de l'évolution, les développeurs ajoutent, suppriment ou modifient des fonctionnalités (y compris les fautes) dans le code source. Les liens de traçabilité deviennent alors obsolètes car les développeurs n'ont pas ou ne peuvent pas consacrer les efforts nécessaires pour les mettre à jour. Pourtant, la récupération de liens de traçabilité est une tâche ardue et coûteuse pour les développeurs. Par conséquent, nous trouvons dans la littérature des méthodes, des techniques et des outils pour récupérer ces liens de traçabilité automatiquement ou semi-automatiquement. Parmi les techniques proposées, la littérature montre que les techniques de recherche d'information (RI) peuvent récupérer automatiquement des liens de traçabilité entre les exigences écrites sous forme textuelle et le code source. Toutefois, la précision et le rappel des techniques RI sont affectés par certains facteurs qui influencent les entrées du processus de traçabilité des exigences. En raison de la faible précision et du faible rappel de ces techniques, la confiance des développeurs en l'efficacité des techniques de récupération des liens de traçabilité est affectée négativement. Dans cette thèse, notre proposition est que l'ajout de nouvelles sources d'information et l'intégration des connaissances des développeurs à l'aide d'un modèle de confiance pourrait atténuer l'impact de ces facteurs et améliorer la précision et le rappel des techniques RI. Notre hypothèse est que la précision et le rappel des techniques RI pourraient être améliorés si deux (ou plus) sources d'information confirment un lien de traçabilité. Nous utilisons les données des référentiels logiciels, les relations entre classes, le partitionnement du code source, et les connaissances des développeurs comme sources d'informations supplémentaires pour confirmer un lien. Nous proposons quatre nouvelles approches de détection des liens de traçabilité : Histrace, BCRTrace, Partrace et une dernière basée sur les connaissances des développeurs. Nous proposons un modèle de confiance, Trumo, inspiré des modèles de confiance Web, pour combiner les votes des experts. Nous proposons alors quatre approche de recouvrement des liens de traçabilité : Trustrace, LIBCROOS, COPARVO et une nouvelle méthode d'attribution de poids, qui utilisent les experts créés précédemment et Trumo. Les approches proposées utilisent une technique de recherche d'information pour créer des liens de référence et utilisent l'opinion des experts pour réévaluer ces liens de référence. Nous montrons que l'utilisation de plusieurs sources d'information améliore la précision et le rappel des techniques RI pour la traçabilité des exigences. Les résultats obtenus dans cette thèse montrent une amélioration de jusqu'à 22% de précision, 7% de rappel et 83% de réduction d'effort des développeurs pour la suppression manuelle de liens faux positifs. Les résultats obtenus dans cette thèse sont prometteurs et nous espérons que d'autres recherches dans ce domaine pourraient améliorer notre précision et rappel encore plus.----------ABSTRACT Traceability is the only means to ensure that the source code of a system is consistent with its requirements and that all and only the specified requirements have been implemented. During software maintenance and evolution, as developers add, remove, or modify features (including bugs), requirement traceability links become obsolete because developers do not/cannot devote effort to update them. Yet, recovering these traceability links later is a daunting and costly task for developers. Consequently, the literature proposed methods, techniques, and tools to recover semi-automatically or automatically these traceability links. Among the proposed techniques, the literature showed that information retrieval (IR) techniques can automatically recover traceability links between free-text requirements and source code. However, precision and recall of IR techniques are impacted by some factors, which impact the input of requirements traceability process. Due to low precision and--or recall, developers' confidence in the effectiveness of traceability link recovery technique is negatively affected. In this dissertation, our thesis is that adding more sources of information using a trust-based model and integrating developers' knowledge in automated IR-based requirements traceability approaches could mitigate the impact of the factors and improve the precision and recall of IR techniques. Our conjecture is that the accuracy of information retrieval techniques could be improved if two (or more) sources of information vote for a link. We use software repositories' data, binary class relationships, source code partitioning, and developer's knowledge as extra sources of information to confirm a link. We propose four approaches, Histrace, BCRTrace, Partrace, and developer's knowledge, to create experts out of the available extra sources of information. We propose a trust-model, Trumo, inspired by Web trust-models of users, to combine the experts' votes. We then propose four traceability link recovery approaches: Trustrace, LIBCROOS, COPARVO, and an improved term weighting scheme, which use the experts created by previous four techniques and Trumo. The proposed approaches use an IR technique to create the baseline links and use experts' opinions to reevaluate baseline links. We show that using more sources of information improve the accuracy of IR techniques for requirements traceability. The achieved results in this dissertation show up to 22% precision, 7% recall improvement and 83% reduction in developer's effort for manually removing false-positive links. The results achieved in this dissertation are promising and we hope that further research in this field might improve the accuracy of IR techniques more