An experimental study of the effectiveness of Panorama as a maintenance tool

It is a well known fact that the amount of time and money spent debugging and maintaining software far outstrips the amount spent developing it. Reports suggest that a majority of this effort is related to information-seeking, concept-location, and software comprehension. In addition to being inherently complex activities, these problems are exacerbated due to the fact that the original developers are often unavailable to help with the maintenance activities. To perform a maintenance task programmers first form dynamic and static mental models of the program based on their prior knowledge of programming and on information available about the program. Aside from high level design documents, Javadoc-like documentation has been the primary means of obtaining such information about programs. There are two severe shortcomings of this form of documentation: a) as recognized by proponents of AOP, the code for many of the concerns of interest to the maintainer are scattered across different classes and files and b) there are a lot of classes and files for large systems leading to information overload and it is very hard and time-consuming to identify the few segments of code and documentation that are relevant for a specific maintenance activity. To address these problems, we have developed an Eclipse plug-in, Panorama, which allow master developers to document sections of code related to a specific developer concern and provides ways to make such information readily available to maintainers. This tool helps directly with information-seeking, concept location, and software comprehension activities and focuses maintainer\u27s attention on the specific code and guidance necessary for a particular task. To validate the tool and our approach, we conducted an experimental study with 19 subjects assigning them several maintenance tasks on a fairly complex application (a multi-user Poker game). The study showed that the experimental group who had access to Panorama was able to complete more maintenance tasks in less time than the control group who did not have access to the tool. In addition, preliminary results indicate that the tool will have more impact on productivity of persons new to the application and programming environment than on experts

An empirically-based characterization and quantification of information seeking through mailing lists during Open Source developers' software evolution

Context Several authors have proposed information seeking as an appropriate perspective for studying software evolution. Empirical evidence in this area suggests that substantial time delays can accrue, due to the unavailability of required information, particularly when this information must travel across geographically distributed sites. Objective As a first step in addressing the time delays that can occur in information seeking for distributed Open Source (OS) programmers during software evolution, this research characterizes the information seeking of OS developers through their mailing lists. Method A longitudinal study that analyses 17 years of developer mailing list activity in total, over 6 different OS projects is performed, identifying the prevalent information types sought by developers, from a qualitative, grounded analysis of this data. Quantitative analysis of the number-of-responses and response time-lag is also performed. Results The analysis shows that Open Source developers are particularly implementation centric and team focused in their use of mailing lists, mirroring similar findings that have been reported in the literature. However novel findings include the suggestion that OS developers often require support regarding the technology they use during development, that they refer to documentation fairly frequently and that they seek implementation-oriented specifics based on system design principles that they anticipate in advance. In addition, response analysis suggests a large variability in the response rates for different types of questions, and particularly that participants have difficulty ascertaining information on other developer's activities. Conclusion The findings provide insights for those interested in supporting the information needs of OS developer communities: They suggest that the tools and techniques developed in support of co-located developers should be largely mirrored for these communities: that they should be implementation centric, and directed at illustrating "how" the system achieves its functional goals and states. Likewise they should be directed at determining the reason for system bugs: a type of question frequently posed by OS developers but less frequently responded to

Real-Time Reflexion Modelling in architecture reconciliation: A multi case study

Context Reflexion Modelling is considered one of the more successful approaches to architecture reconciliation. Empirical studies strongly suggest that professional developers involved in real-life industrial projects find the information provided by variants of this approach useful and insightful, but the degree to which it resolves architecture conformance issues is still unclear. Objective This paper aims to assess the level of architecture conformance achieved by professional architects using Reflexion Modelling, and to determine how the approach could be extended to improve its suitability for this task. Method An in vivo, multi-case-study protocol was adopted across five software systems, from four different financial services organizations. Think-aloud, video-tape and interview data from professional architects involved in Reflexion Modelling sessions were analysed qualitatively. Results This study showed that (at least) four months after the Reflexion Modelling sessions less than 50% of the architectural violations identified were removed. The majority of participants who did remove violations favoured changes to the architectural model rather than to the code. Participants seemed to work off two specific architectural templates, and interactively explored their architectural model to focus in on the causes of violations, and to assess the ramifications of potential code changes. They expressed a desire for dependency analysis beyond static-source-code analysis and scalable visualizations. Conclusion The findings support several interesting usage-in-practice traits, previously hinted at in the literature. These include (1) the iterative analysis of systems through Reflexion models, as a precursor to possible code change or as a focusing mechanism to identify the location of architecture conformance issues, (2) the extension of the approach with respect to dependency analysis of software systems and architectural modelling templates, (3) improved visualization support and (4) the insight that identification of architectural violations in itself does not lead to their removal in the majority of instances.This work was supported, in part, by Science Foundation Ireland Grants 12/IP/1351 and 10/CE/I1855 to Lero – the Irish Software Engineering Research Centre (www.lero.ie) and by the University of Brighton under the Rising Star Scheme awarded to Nour Ali

A study of novice programmer performance and programming pedagogy.

Identifying and mitigating the difficulties experienced by novice programmers is an active area of research that has embraced a number of research areas. The aim of this research was to perform a holistic study into the causes of poor performance in novice programmers and to develop teaching approaches to mitigate them. A grounded action methodology was adopted to enable the primary concepts of programming cognitive psychology and their relationships to be established, in a systematic and formal manner. To further investigate novice programmer behaviour, two sub-studies were conducted into programming performance and ability. The first sub-study was a novel application of the FP-Tree algorithm to determine if novice programmers demonstrated predictable patterns of behaviour. This was the first study to data mine programming behavioural characteristics rather than the learner’s background information such as age and gender. Using the algorithm, patterns of behaviour were generated and associated with the students’ ability. No patterns of behaviour were identified and it was not possible to predict student results using this method. This suggests that novice programmers demonstrate no set patterns of programming behaviour that can be used determine their ability, although problem solving was found to be an important characteristic. Therefore, there was no evidence that performance could be improved by adopting pedagogies to promote simple changes in programming behaviour beyond the provision of specific problem solving instruction. A second sub-study was conducted using Raven’s Matrices which determined that cognitive psychology, specifically working memory, played an important role in novice programmer ability. The implication was that programming pedagogies must take into consideration the cognitive psychology of programming and the cognitive load imposed on learners. Abstracted Construct Instruction was developed based on these findings and forms a new pedagogy for teaching programming that promotes the recall of abstract patterns while reducing the cognitive demands associated with developing code. Cognitive load is determined by the student’s ability to ignore irrelevant surface features of the written problem and to cross-reference between the problem domain and their mental program model. The former is dealt with by producing tersely written exercises to eliminate distractors, while for the latter the teaching of problem solving should be delayed until the student’s program model is formed. While this does delay the development of problem solving skills, the problem solving abilities of students taught using this pedagogy were found to be comparable with students taught using a more traditional approach. Furthermore, monitoring students’ understanding of these patterns enabled micromanagement of the learning process, and hence explanations were provided for novice behaviour such as difficulties using arrays, inert knowledge and “code thrashing”. For teaching more complex problem solving, scaffolding of practice was investigated through a program framework that could be developed in stages by the students. However, personalising the level of scaffolding required was complicated and found to be difficult to achieve in practice. In both cases, these new teaching approaches evolved as part of a grounded theory study and a clear progression of teaching practice was demonstrated with appropriate evaluation at each stage in accordance with action researc

Foundations of Empirical Software Engineering: The Legacy of Victor R. Basili

This book captures the main scientific contributions of Victor R. Basili, who has significantly shaped the field of empirical software engineering from its very start. He was the first to claim that software engineering needed to follow the model of other physical sciences and develop an experimental paradigm. By working on this postulate, he developed concepts that today are well known and widely used, including the Goal-Question-Metric method, the Quality-Improvement paradigm, and the Experience Factory. He is one of the few software pioneers who can aver that their research results are not just scientifically acclaimed but are also used as industry standards. On the occasion of his 65th birthday, celebrated with a symposium in his honor at the International Conference on Software Engineering in St. Louis, MO, USA in May 2005, Barry Boehm, Hans Dieter Rombach, and Marvin V. Zelkowitz, each a long-time collaborator of Victor R. Basili, selected the 20 most important research papers of their friend, and arranged these according to subject field. They then invited renowned researchers to write topical introductions. The result is this commented collection of timeless cornerstones of software engineering, hitherto available only in scattered publications

Understanding User Cognition: From Spatial Ability to Code Writing and Review

Understanding how developers carry out different computer science activities with objective measures can help to improve productivity and guide the use and development of supporting tools in software engineering. In this thesis, we present three research components using three different objective measures including neuroimaging (functional magnetic resonance imaging (fMRI) and functional near-infrared spectroscopy (fNIRS)) and eye tracking. We evaluate on over 140 human subjects to explore multiple computing activities, including data structure manipulations, code writing and code review. This thesis presents a systematic framework and shows that it is possible to conduct studies that acquire objective data in a natural setting to provide an understanding of users' underlying cognitive processes in software engineering tasks. We also provide basic principles and guidelines to adapt multiple psycho-physiological measures to software engineering.PHDComputer Science & EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/169678/1/yhhy_1.pd

Designing and Evaluating Accessible E-Learning for Students with Visual Impairments in K-12 Computing Education

This dissertation explores the pathways for making K-12 computing education more accessible for blind or visually impaired (BVI) learners. As computer science (CS) expands into K-12 education, more concerted efforts are required to ensure all students have equitable access to opportunities to pursue a career in computing. To determine their viability with BVI learners, I conducted three studies to assess current accessibility in CS curricula, materials, and learning environments. Study one was interviews with visually impaired developers; study two was interviews with K-12 teachers of visually impaired students; study three was a remote observation within a computer science course. My exploration revealed that most of CS education lacks the necessary accommodations for BVI students to learn at an equitable pace with sighted students. However, electronic learning (e-learning) was a theme that showed to provide the most accessible learning experience for BVI students, although even there, usability and accessibility challenges were present in online learning platforms. My dissertation engaged in a human-centered approach across three studies towards designing, developing, and evaluating an online learning management system (LMS) with the critical design elements to improve navigation and interaction with BVI users. Study one was a survey exploring the perception of readiness for taking online courses between sighted and visually impaired students. The findings from the survey fueled study two, which employed participatory design with storytelling with K-12 teachers and BVI students to learn more about their experiences using LMSs and how they imagine such systems to be more accessible. The findings led to developing the accessible learning content management system (ALCMS), a web-based platform for managing courses, course content, and course roster, evaluated in study three with high school students, both sighted and visually impaired, to determine its usability and accessibility. This research contributes with recommendations for including features and design elements to improve accessibility in existing LMSs and building new ones