Peer Assessment in Experiential Learning : Assessing Tacit and Explicit Skills in Agile Software Engineering Capstone Projects

To prepare students for real-life software engineering projects, many higher-education institutions offer courses that simulate working life to varying degrees. As software engineering requires not only technical, but also inter- and intrapersonal skills, these skills should also be assessed. Assessing soft skills is challenging, especially when project-based and experiential learning are the primary pedagogical approaches. Previous work suggests that including students in the assessment process can yield a more complete picture of student performance. This paper presents experiences with developing and using a peer assessment framework that provides a 360-degree view on students' project performance. Our framework has been explicitly constructed to accommodate and evaluate tacit skills that are relevant in agile software development. The framework has been evaluated with 18 bachelors- and 11 masters-level capstone projects, totaling 176 students working in self-organized teams. We found that the framework eases teacher workload and allows a more thorough assessment of students' skills. We suggest including self- and peer assessment into software capstone projects alongside other, more traditional schemes like productivity metrics, and discuss challenges and opportunities in defining learning goals for tacit and social skills.Peer reviewe

Assessing collaborative and experiential learning

Collaborative and experiential learning has many proven merits. Team projects with real clients motivate students to put in the time for successfully completing demanding projects. However, assessing student performance where individual student contributions are separated from the collective contribution of the team as a whole is not a straightforward, simple task. Assessment data from multiple sources, including students as assessors of their own work and peers\u27 work, is critical to measuring certain student learning outcomes, such as responsible team work and timely communication. In this paper we present our experience with assessing collaborative and experiential learning in five Computer Information Systems courses. The courses were scheduled over three semesters and enrolled 57 students. Student performance and student feedback data were used to evaluate and refine our assessment methodology. We argue that assessment data analysis improved our understanding of (1) the assessment measures that support more closely targeted learning outcomes and (2) how those measures should be implemented

Mutation testing and self/peer assessment: analyzing their effect on students in a software testing course

Testing is a crucial activity in the development of software systems. With the increasing complexity of software projects, the industry requires incorporating graduates with adequate testing skills and preparation in this field. A challenge in software testing education is to make students perceive the benefits of writing tests and assess their quality with advanced testing techniques. In this paper, we present an experience integrating both mutation testing and self/peer assessment –two of the most used techniques to that end in the past– into a software testing course during three years. This experience allowed us to analyze the effect of applying these strategies on the students’ perception of their manually-written test suites. Noticeably, the computation of the mutation score significantly undermined the initial expectations they had on the developed test suites. Also, the application of peer testing helped them estimate the relative quality of two comparable test suites, as we found a notable correspondence with their respective mutation coverage. Besides, a more in-depth analysis revealed that the students' test suites with more test cases did not always achieve the highest scores, that they found more readable their own tests, and that they tended to cover the basic operations while forgetting about more advanced features. An opinion survey confirmed the impact that the use of mutants had on their perception about testing, and they mostly supported paying a higher level of attention to testing concepts in software engineering degree plans.The work was partially funded by the European Commission (FEDER), the Spanish Ministry of Science, Innovation and Universities under the project FAME (RTI2018-093608-B-C33), the European project ASSETs (612678-EPP-1-2019-1-IT-EPPKA2-SSA-B), and the University of Cádiz

A collaborative and experiential learning model powered by real-world projects

Information Technology (IT) curricula\u27s strong application component and its focus on user centeredness and team work require that students experience directly real-world projects for real users of IT solutions. Although the merit of this IT educational tenet is universally recognized, delivering collaborative and experiential learning has its challenges. Reaching out to identify projects formulated by actual organizations adds significantly to course preparation. There is a certain level of risk involved with delivering a useful solution while, at the same time, enough room should be allowed for students to experiment with, be wrong about, review, and learn. Challenges pertaining to the real-world aspect of problem-based learning are compounded by managing student teams and assessing their work such that both individual and collective contributions are taken into account. Finally, the quality of the project releases is not the only measure of student learning. Students should be given meaningful opportunities to practice, improve, and demonstrate their communication and interpersonal skills. In this paper we present our experience with two courses in which teams of students worked on real-world projects involving three external partners. We describe how each of the challenges listed above has impacted the course requirements, class instruction, team dynamics, assessment, and learning in these courses. Course assessment and survey data from students are linked to learning outcomes and point to areas where the collaborative and experiential learning model needs improvement

A preliminary evaluation of using WebPA for online peer assessment of collaborative performance by groups of online distance learners

Collaborative assessment has well-recognised benefits in higher education and, in online distance learning, this type of assessment may be integral to collaborative e-learning and may have a strong influence on the student’s relationship with learning. While there are known benefits associated with collaborative assessment, the main drawback is that students perceive that their individual contribution to the assessment is not recognised. Several methods can be used to overcome this; for example, something as simple as the teacher evaluating an individual’s contribution. However, teacher assessment can be deemed as unreliable by students, since the majority of group work is not usually done in the presence of the teacher (Loddington, Pond, Wilkinson, & Wilmot, 2009). Therefore, students’ assessment of performance/contribution of themselves and their peer group in relation to the assessment task, also known as peer moderation, can be a more suitable alternative. There are a number of tools that can be used to facilitate peer moderation online, such as WebPA, which is a free, open source, online peer assessment tool developed by Loughborough University. This paper is a preliminary evaluation of online peer assessment of collaborative work undertaken by groups of students studying online at a distance at a large UK university, where WebPA was used to facilitate this process. Students’ feedback on the use of WebPA was mixed, although most of the students found the software easy to use, with few technical issues and the majority reported that they would be happy to use this again. The authors reported WebPA as a beneficial peer assessment tool

Free and open source software development of IT systems

IT system development, integration, deployment, and administration benefit significantly from free and open source software (FOSS) tools and services. Affordability has been a compelling reason for adopting FOSS in computing curricula and equipping computing labs with support infrastructure. Using FOSS systems and services, however, is just the first step in taking advantage of how FOSS development principles and practices can impact student learning in IT degree programs. Above all, FOSS development of IT systems requires changes to how students, instructors, and other contributors work collaboratively and openly and get involved and invested in project activities. In this paper I examine the challenges to engage students in FOSS development projects proposed by real clients. A six-week course project revealed problems with adopting FOSS development and collaboration across different activities and roles that student team members have assumed. Despite these problems, students have showed a genuine and strong interest in gaining more practice with FOSS development. FOSS development teaching was further refined in two other courses to learn about adequate teaching strategies and the competencies that students achieve when they participate in FOSS development of IT systems

Beyond Surveys: Analyzing Software Development Artifacts to Assess Teaching Efforts

This Innovative Practice Full Paper presents an approach of using software development artifacts to gauge student behavior and the effectiveness of changes to curriculum design. There is an ongoing need to adapt university courses to changing requirements and shifts in industry. As an educator it is therefore vital to have access to methods, with which to ascertain the effects of curriculum design changes. In this paper, we present our approach of analyzing software repositories in order to gauge student behavior during project work. We evaluate this approach in a case study of a university undergraduate software development course teaching agile development methodologies. Surveys revealed positive attitudes towards the course and the change of employed development methodology from Scrum to Kanban. However, surveys were not usable to ascertain the degree to which students had adapted their workflows and whether they had done so in accordance with course goals. Therefore, we analyzed students' software repository data, which represents information that can be collected by educators to reveal insights into learning successes and detailed student behavior. We analyze the software repositories created during the last five courses, and evaluate differences in workflows between Kanban and Scrum usage

Strategies and mechanisms for electronic peer review

This journal article published at the October 2000 Frontiers in Education Conference discusses strategies and mechanisms for electronic peer review. It outlines a peer-grading system for review of student assignments over the World-Wide Web called Peer Grader. The system allows authors and reviewers to communicate and authors to update their submissions. This system facilitates collaborative learning and makes it possible to break up a large project into smaller portions. The article summarizes a unique and innovative method of peer-review. Educational levels: Graduate or professional