Beyond Surveys: Analyzing Software Development Artifacts to Assess Teaching Efforts

This Innovative Practice Full Paper presents an approach of using software development artifacts to gauge student behavior and the effectiveness of changes to curriculum design. There is an ongoing need to adapt university courses to changing requirements and shifts in industry. As an educator it is therefore vital to have access to methods, with which to ascertain the effects of curriculum design changes. In this paper, we present our approach of analyzing software repositories in order to gauge student behavior during project work. We evaluate this approach in a case study of a university undergraduate software development course teaching agile development methodologies. Surveys revealed positive attitudes towards the course and the change of employed development methodology from Scrum to Kanban. However, surveys were not usable to ascertain the degree to which students had adapted their workflows and whether they had done so in accordance with course goals. Therefore, we analyzed students' software repository data, which represents information that can be collected by educators to reveal insights into learning successes and detailed student behavior. We analyze the software repositories created during the last five courses, and evaluate differences in workflows between Kanban and Scrum usage

Should I Bug You? Identifying Domain Experts in Software Projects Using Code Complexity Metrics

In any sufficiently complex software system there are experts, having a deeper understanding of parts of the system than others. However, it is not always clear who these experts are and which particular parts of the system they can provide help with. We propose a framework to elicit the expertise of developers and recommend experts by analyzing complexity measures over time. Furthermore, teams can detect those parts of the software for which currently no, or only few experts exist and take preventive actions to keep the collective code knowledge and ownership high. We employed the developed approach at a medium-sized company. The results were evaluated with a survey, comparing the perceived and the computed expertise of developers. We show that aggregated code metrics can be used to identify experts for different software components. The identified experts were rated as acceptable candidates by developers in over 90% of all cases

Challenges (and Opportunities!) of a Remote Agile Software Engineering Project Course During COVID-19

COVID-19 and its immediate impacts on teaching activities have required changes from computer science educators worldwide. We switched our on-site courses to remote setups without detailed knowledge of what tools, techniques, and methods would work in different teaching contexts. A growing amount of experience reports on general best practices for remote teaching in higher education are available. However, university courses featuring practical software development projects present unique challenges regarding remote learning, as effective student collaboration is vital. In these courses, students tackle situations in the project and their team meetings that would also occur in real software projects experienced in industry settings. In this paper, we share our experiences on how we successfully adapted our software engineering project course to a remote setup, which challenges we observed in student teams and how they can be mitigated, and what (surprisingly) worked better than expected. Finally, we propose improvements that we expect will be beneficial not only for future remote-only but also for hybrid or on-site courses

What Stays in Mind? - Retention Rates in Programming MOOCs

This work presents insights about the long-term effects and retention rates of knowledge acquired within MOOCs. In 2015 and 2017, we conducted two introductory MOOCs on object-oriented programming in Java with each over 10,000 registered participants. In this paper, we analyze course scores, quiz results and self-stated skill levels of our participants. The aim of our analysis is to uncover factors influencing the retention of acquired knowledge, such as time passed or knowledge application, in order to improve long-term success. While we know that some participants learned the programming basics within our course, we lack information on whether this knowledge was applied and fortified after the course's end. To fill this knowledge gap, we conducted a survey in 2018 among all participants of our 2015 and 2017 programming MOOCs. The first part of the survey elicits responses on whether and how MOOC knowledge was applied and gives participants opportunity to voice individual feedback. The second part of the survey contains several questions of increasing difficulty and complexity regarding course content in order to learn about the consolidation of the acquired knowledge. We distinguish three programming knowledge areas in the survey: First, understanding of concepts, such as loops and boolean algebra. Second, syntax knowledge, such as specific keywords. Third, practical skills including debugging and coding. We further analyzed the long-term effects separately per participant skill group. While answer rates were low, the collected data shows a decrease of knowledge over time, relatively unaffected by skill level. Application of the acquired knowledge improves the memory retention rates of MOOC participants across all skill levels

Interactive Strategy-Based Validation of Behavioral Models

When behavioral models are derived automatically based on observed stakeholder interactions, requirements engineers need to validate whether the stakeholders agree with the synthesized behavioral models. Allowing stakeholders to experience such models through simulation and animation allows them to comment on, amend to and correct these models. However, to ensure an efficient stakeholder validation, the simulation has to be guided instead of confronting the user with random situations over and over again. In this paper, we present a strategy-driven simulator capable of guiding the execution of behavioral models based on graph transformations. By analyzing either the overall structure of a partial state space (look ahead) or by performing an in-depth analysis of the states therein, the simulator is able to determine which transformations should be executed next to continue on the most promising path through the overall state space. The discussed implementation is illustrated with a case study

Cellular Automata as an Example for Advanced Beginners’ Level Coding Exercises in a MOOC on Test Driven Development

Programming tasks are an important part of teaching computer programming as they foster students to develop essential programming skills and techniques through practice.  The design of educational problems plays a crucial role in the extent to which the experiential knowledge is imparted to the learner both in terms of quality and quantity. Badly designed tasks have been known to put-off students from practicing programming. Hence, there is a need for carefully designed problems. Cellular Automata programming lends itself as a very suitable candidate among problems designed for programming practice. In this paper, we describe how various types of problems can be designed using concepts from Cellular Automata and discuss the features which make them good practice problems with regard to instructional pedagogy. We also present a case study on a Cellular Automata programming exercise used in a MOOC on Test Driven Development using JUnit, and discuss the automated evaluation of code submissions and the feedback about the reception of this exercise by participants in this course. Finally, we suggest two ideas to facilitate an easier approach of creating such programming exercises

Cellular Automata as an Example for Advanced Beginners’ Level Coding Exercises in a MOOC on Test Driven Development

Programming tasks are an important part of teaching computer programming as they foster students to develop essential programming skills and techniques through practice.  The design of educational problems plays a crucial role in the extent to which the experiential knowledge is imparted to the learner both in terms of quality and quantity. Badly designed tasks have been known to put-off students from practicing programming. Hence, there is a need for carefully designed problems. Cellular Automata programming lends itself as a very suitable candidate among problems designed for programming practice. In this paper, we describe how various types of problems can be designed using concepts from Cellular Automata and discuss the features which make them good practice problems with regard to instructional pedagogy. We also present a case study on a Cellular Automata programming exercise used in a MOOC on Test Driven Development using JUnit, and discuss the automated evaluation of code submissions and the feedback about the reception of this exercise by participants in this course. Finally, we suggest two ideas to facilitate an easier approach of creating such programming exercises

From Full-fledged ERP Systems Towards Process-centric Business Process Platforms

Enterprise Resource Planning (ERP) systems are critical to the success of enterprises, facilitating business operations through standardized digital processes. However, existing ERP systems are unsuitable for startups and small and medium-sized enterprises that grow quickly and require adaptable solutions with low barriers to entry. Drawing upon 15 explorative interviews with industry experts, we examine the challenges of current ERP systems using the task technology fit theory across companies of varying sizes. We describe high entry barriers, high costs of implementing implicit processes, and insufficient interoperability of already employed tools. We present a vision of a future business process platform based on three enablers: Business processes as first-class entities, semantic data and processes, and cloud-native elasticity and high availability. We discuss how these enablers address current ERP systems’ challenges and how they may be used for research on the next generation of business software for tomorrow’s enterprises