Mining unstructured software data

Our thesis is that the analysis of unstructured data supports software understanding and evolution analysis, and complements the data mined from structured sources. To this aim, we implemented the necessary toolset and investigated methods for exploring, exposing, and exploiting unstructured data.To validate our thesis, we focused on development email data. We found two main challenges in using it to support program comprehension and software development: The disconnection between emails and code artifacts and the noisy and mixed-language nature of email content. We tackle these challenges proposing novel approaches. First, we devise lightweight techniques for linking email data to code artifacts. We use these techniques for creating a tool to support program comprehension with email data, and to create a new set of email based metrics to improve existing defect prediction approaches. Subsequently, we devise techniques for giving a structure to the content of email and we use this structure to conduct novel software analyses to support program comprehension. In this dissertation we show that unstructured data, in the form of development emails, is a valuable addition to structured data and, if correctly mined, can be used successfully to support software engineering activities

A Tool for Visual Understanding of Source Code Dependencies

Many program comprehension tools use graphs to visualize and analyze source code. The main issue is that existing approaches create graphs overloaded with too much information. Graphs contain hundreds of nodes and even more edges that cross each other. Understanding these graphs and using them for a given program comprehension task is tedious, and in the worst case developers stop using the tools. In this paper we present DA4Java, a graphbased approach for visualizing and analyzing static dependencies between Java source code entities. The main contribution of DA4Java is a set of features to incrementally compose graphs and remove irrelevant nodes and edges from graphs. This leads to graphs that contain significantly fewer nodes and edges and need less effort to understand

Autonomous Development of Algorithmic Concepts for Program Comprehension

A developmental model of algorithmic concepts is proposed here for program comprehension. Unlike traditional approaches that cannot do anything beyond their predesigned representation, this model can develop its internal representation autonomously from chaos into algorithmic concepts by mimicking concept formation in the brain under an uncontrollable environment that consists of program source codes from the Internet. The developed concepts can be employed to identify what algorithm a program performs. The accuracy of such identification reached 97.15% in a given experiment

An Examination of the Correlation of Exploring Computer Science Course Performance and the Development of Programming Expertise

This study investigated patterns in the development of computational thinking and programming expertise in the context of the Exploring Computer Science (ECS) program, a high school introductory CS course and professional development program designed to foster deep engagement through equitable inquiry around CS concepts. Prior research on programming expertise has identified three general areas of development --- program comprehension, program planning, and program generation. The pedagogical practices in ECS are consistent with problem solving approaches that support the development of programming expertise. The study took place in a large urban district during the 2016--17 school year with 28 ECS teachers and 1,931 students. A validated external assessment was used to measure the development of programming expertise. The results indicate that there were medium-sized, statistically significant increases from pretest to posttest, and there were no statistically significant differences by gender or race/ethnicity. After controlling for prior academic achievement, performance in the ECS course correlated with performance on the posttest. With respect to specific programming concepts, the results also provide evidence on the progression of the development of programming expertise. Students seem to develop comprehension and planning expertise prior to expertise in program generation. In addition, students seem to develop expertise with concrete tasks prior to abstract tasks

A trajectory-based strict semantics for program slicing

We define a program semantics that is preserved by dependence-based slicing algorithms. It is a natural extension, to non-terminating programs, of the semantics introduced by Weiser (which only considered terminating ones) and, as such, is an accurate characterisation of the semantic relationship between a program and the slice produced by these algorithms. Unlike other approaches, apart from Weiser’s original one, it is based on strict standard semantics which models the ‘normal’ execution of programs on a von Neumann machine and, thus, has the advantage of being intuitive. This is essential since one of the main applications of slicing is program comprehension. Although our semantics handles non-termination, it is defined wholly in terms of finite trajectories, without having to resort to complex, counter-intuitive, non-standard models of computation. As well as being simpler, unlike other approaches to this problem, our semantics is substitutive. Substitutivity is an important property becauseit greatly enhances the ability to reason about correctness of meaning-preserving program transformations such as slicing

Adolescent Literacy and the Achievement Gap: What Do We Know and Where Do We Go From Here?

Reviews research and program initiatives focused on improving adolescent academic achievement by targeting literacy. Provides ideas for collaboration and coordination of funding efforts to improve the literacy achievement of under-performing adolescents