Peer Assessment in Experiential Learning : Assessing Tacit and Explicit Skills in Agile Software Engineering Capstone Projects

To prepare students for real-life software engineering projects, many higher-education institutions offer courses that simulate working life to varying degrees. As software engineering requires not only technical, but also inter- and intrapersonal skills, these skills should also be assessed. Assessing soft skills is challenging, especially when project-based and experiential learning are the primary pedagogical approaches. Previous work suggests that including students in the assessment process can yield a more complete picture of student performance. This paper presents experiences with developing and using a peer assessment framework that provides a 360-degree view on students' project performance. Our framework has been explicitly constructed to accommodate and evaluate tacit skills that are relevant in agile software development. The framework has been evaluated with 18 bachelors- and 11 masters-level capstone projects, totaling 176 students working in self-organized teams. We found that the framework eases teacher workload and allows a more thorough assessment of students' skills. We suggest including self- and peer assessment into software capstone projects alongside other, more traditional schemes like productivity metrics, and discuss challenges and opportunities in defining learning goals for tacit and social skills.Peer reviewe

Involving External Stakeholders in Project Courses

Problem: The involvement of external stakeholders in capstone projects and project courses is desirable due to its potential positive effects on the students. Capstone projects particularly profit from the inclusion of an industrial partner to make the project relevant and help students acquire professional skills. In addition, an increasing push towards education that is aligned with industry and incorporates industrial partners can be observed. However, the involvement of external stakeholders in teaching moments can create friction and could, in the worst case, lead to frustration of all involved parties. Contribution: We developed a model that allows analysing the involvement of external stakeholders in university courses both in a retrospective fashion, to gain insights from past course instances, and in a constructive fashion, to plan the involvement of external stakeholders. Key Concepts: The conceptual model and the accompanying guideline guide the teachers in their analysis of stakeholder involvement. The model is comprised of several activities (define, execute, and evaluate the collaboration). The guideline provides questions that the teachers should answer for each of these activities. In the constructive use, the model allows teachers to define an action plan based on an analysis of potential stakeholders and the pedagogical objectives. In the retrospective use, the model allows teachers to identify issues that appeared during the project and their underlying causes. Drawing from ideas of the reflective practitioner, the model contains an emphasis on reflection and interpretation of the observations made by the teacher and other groups involved in the courses. Key Lessons: Applying the model retrospectively to a total of eight courses shows that it is possible to reveal hitherto implicit risks and assumptions and to gain a better insight into the interaction...Comment: Abstract shortened since arxiv.org limits length of abstracts. See paper/pdf for full abstract. Paper is forthcoming, accepted August 2017. Arxiv version 2 corrects misspelled author nam

Proceedings of the 1st international workshop on software process education, training and professionalism (SPETP 2015)

These Proceedings contain the papers accepted for publication and presentation at the first 1st International Workshop on Software Process Education, Training and Professionalism (SPETP 2015) held in conjunction with the 15th International Conference on Software Process Improvement and Capability dEtermination (SPICE 2015), Gothenburg, Sweden, during June 15-17, 2015. During the 14th International Conference on Software Process Improvement and Capability dEtermination (SPICE 2014) held in Vilnius, Lithuania, at a post conference dinner, a group of key individuals from education and industry started to discuss the challenges faced for software process education, training and professionalism, especially with the background of the new modes of learning and teaching in higher education. Further discussions held post conference with key players in the relevant professional and personal certification fields led to a consensus that it is time for the industry to rise to the new challenges and set out in a manifesto a common vision for educators and trainers together with a set of recommendations to address the challenges faced. It was therefore agreed co-located the 1st International Workshop on Software Process Education, Training and Professionalism with the 15th International Conference on Software Process Improvement and Capability dEtermination. This workshop focused on the new challenges for and best practices in software process education, training and professionalism. The foundation for learning of software process should be part of a university or college education however software process is often treated as ‘add one’ module to the core curriculum. In a professional context, whilst there have been a number of initiatives focused on the certification related to the software process professional these have had little success for numerous reasons. Cooperation in education between industry, academia and professional bodies is paramount, together with the recognition of how the education world is changing and how education is resourced, delivered (with online and open learning) and taken up. Over the next 10 years on-line learning is projected to grow fifteen fold, accounting for 30% of all education provision, according to the recent report to the European Commission on New modes of learning and teaching in higher education. It is a great pleasure to see the varied contributions to this 1st International Workshop on Software Process Education, Training and Professionalism and we hope that our joint dedication, passion and innovation will lead to success for the profession through the publication of the manifesto as a key outcome from the workshop. On behalf of the SPETP 2015 conference Organizing Committee, we would like to thank all participants. Firstly all the authors, whose quality work is the essence of the conference, and the members of the Program Committee, who helped us with their expertise and diligence in reviewing all of the submissions. As we all know, organizing a conference requires the effort of many individuals. We wish to thank also all the members of our Organizing Committee, whose work and commitment were invaluable

Characterizing the Application of Design Ethnography Techniques to Improve Novice Human-Centered Design Processes.

Design is a central, distinguishing feature of engineering, requiring the development of technical solutions to societal problems. Successful design solutions must not only be technically sound, but also well-adapted to the context and culture in which they will be used. However, the most commonly used methods for eliciting and characterizing stakeholder preferences do not typically reveal critical stakeholder and contextual information. Through the studies described in this dissertation, I explore the use of design ethnography during front-end engineering design phases to capture both stakeholder preferences and contextual knowledge to inform engineering design decision making. Design ethnography is a set of primarily qualitative data collection and analysis techniques that have been adapted from the field of anthropology to augment the engineering design process. Studies from the fields of human-computer interaction and product development have demonstrated that design ethnography techniques are cost-effective and lead to more successful products. However, the design ethnography literature lacks critical understanding of the major barriers and factors that influence design ethnography success, methodologies for synthesizing and applying design ethnography data, best practices to engage with stakeholders, developmental trajectories of novice to expert skill acquisition, and case studies of how design ethnography has been implemented in diverse settings. The studies detailed in this dissertation employ a mixture of quantitative and qualitative research methods to address these gaps in the literature. Through this research, I have characterized novice design ethnography implementation strategies and identified internal and external factors that affect design ethnography execution; investigated correlations between information processing ability and the quality of product requirements developed; established a framework for evaluating and directing design ethnography stakeholder interviews; developed a case study within a global health design context; and interpreted the findings within a theoretically grounded model to represent novice to expert development. This body of work informs strategies and processes for engaging with stakeholders and understanding broader contexts in design work to improve design decision making within both design professional practice and engineering education.PhDMechanical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/133391/1/imohedas_1.pd

Privileging Tacit Knowledge within a Software Engineering Curriculum: A Living Educational Theory of Practice

Through action research I inquired into my pedagogical practice by questioning the nature of knowledge I valued as a lecturer. This questioning led me to develop my ‘living educational theory’ (Whitehead, 1989, p.41) of privileging tacit knowledge within a Software Engineering curriculum. My living educational theory is grounded in ideas of professional knowledge, relationships, competence, and expertise. In developing my theory, I explain how I transformed my ontological values of justice, democracy, and care in relation to students through standards of judgement that I developed to direct, test, and evaluate actions I took to improve my pedagogical practice. I describe how I experienced conflict between my existing practice and ontological values that led me to see myself as a ‘living contradiction’ (Whitehead, 1989, p.41) and to critique the dominant didactic perspectives located within my practice which privileged explicit disciplinary knowledge within a Software Engineering curriculum. To overcome feeling like a living contradiction, I researched and engaged with dialogical problem-posing pedagogies to encourage and support students to actively participate in their own development of becoming competent software engineering professionals. The dialogical problem-posing pedagogy I developed during this inquiry is constructed on the basis of just, democratic, and caring relationships with students, who are capable of exercising their agency and are constantly remaking their identity as they both create and use professional knowledge to solve Software Engineering problems. As I engaged with this dialogical problem-posing pedagogy, I re-conceptualised my identity as a pedagogical practitioner. I questioned the traditional and dominant orthodoxies that I subscribed to and which dictated that I positioned myself as the knowledge expert within the classroom. In doing so, I took action to move from being the knowledge expert to being a facilitator within the classroom to help students to realise their capacity to become competent software engineering practitioners

Towards a disruptive learning model in information systems education: a reflective student-dominant logic perspective

This thesis describes the research carried out to identify mechanisms as potential causes of action that can lead to improved learning outcomes. Identifying mechanisms requires an exploratory approach because mechanisms are not always directly perceivable. The potential mechanisms in this study were identified by analysing the reflections of students who participated in experiential learning projects during business process management education. The research was necessary due to a persistent need to minimise the gap between learning and practice. The research is unique because it takes a student-dominant approach to review students' reflective practices using a novel pragmatic-critical realist paradigm. Although reflection is central to experiential learning, there is limited understanding of how students reflect and what they consider to influence their learning outcomes. This limitation was explored using action research with mixed methods analysis which combined thematic analysis and partial least squares based structural equation modelling. During four action research cycles, student reflections on business process projects as part of higher education enterprise systems and business process management courses in a South African higher education institution were observed and analysed. Each action research cycle changed one aspect of the students' projects. The first intervention required students to reflect on action, the second required students to reflect in action, and the third required reflection for action. In the fourth cycle, the teaching staff changed. The findings showed that reflection is complex and must be linked to action to improve learning. Reflections were observed to positively influence learning outcomes when students apply effort to assigned tasks. On the basis of the observations, two models were proposed. The first model is a learning influence model embedded in the second disruptive learning model. The models show that reflective practices can improve learning outcomes by recognising that students learn at different levels but are predominantly absolutist. Students need to be inspired to apply effort in completing tasks and overcoming satisficing

Online peer assessment in higher education: a systematic review of literature in educational practices

Neste artigo procedemos a uma revisão sistemática da literatura sobre Online Peer Assessment (OPA) no Ensino Superior, mediadas por Tecnologias Digitais de Informação e Comunicação (TIC). Identificamos suas características, centrais e opcionais e mapeamos exemplos práticos (procedimentos e TIC) de OPA que poderão ser transversais, adaptáveis e aplicáveis em diversas unidades curriculares e regimes educacionais. Os resultados apontam para a utilização da OPA como uma estratégia que potencia a “avaliação para a aprendizagem”. Os referenciais teóricos subjacentes, os métodos de avaliação e os tipos de TIC utilizadas indicam seu direcionamento para um maior envolvimento e responsabilidade do aluno na sua aprendizagem. Evidenciamos na literatura que se busca desenvolver essas competências, dando ao aluno oportunidades com alguma regularidade para autoavaliar-se e avaliar seus pares por meio de feedback construtivo. Há também evidências de que dar ou produzir feedback é mais benéfico para a aprendizagem do que apenas recebê-lo, como também é cognitivamente mais exigente e envolve os alunos de forma mais ativa e os direciona ao pensamento crítico e a processos metacognitivos. A partir dos resultados alcançados, consideramos que a OPA, enquanto ferramenta cognitiva, contribui para a construção do conhecimento e para a reflexão sobre a aprendizagem. Um desafio que se coloca diz respeito ao desenvolvimento criativo com foco na diversificação e na inovação das práticas de avaliação no sentido de potenciar aprendizagens e resultados acadêmicos, em atenção às necessidades de aprendizagem que se manifestam frente às expectativas da educação atual e futura e às exigências da sociedade.In this research a systematic review of literature about Online Peer Assessment (OPA) in higher education, mediated by Information and Communication Technology (ICT), was carried out. Optional and central characteristics and its practices (procedures and ICT) were identified and mapping. These procedures and practices may be transversal, adaptable and applied in several curriculum units and educational regimes. The results point to the use of OPA as a strategy that enhances “assessment for learning”. The theoretical framework, the methods of assessment and the kinds of ICT used indicate directions for greater involvement and responsibility from the part of the student in his/her learning. Theoretical framework identifies the need to develop students’ skills, providing them with opportunities for self-assessment and peer assessment on a regular basis through constructive feedback. There is also evidence that to give or receive feedback is more beneficial for learning than just receiving it. It is also cognitively more demanding and directs the students to critical thinking and metacognitive processes. From the results obtained, OPA is seen as a cognitive tool, contributing to the building of knowledge and to reflection about learning. Issues for further reflection are also identified such as the creative development of assessment methods with a focus on diversity and innovation in order to enhance students’ learning and academic results, taking into account their learning needs in face of the expectations of current and future education and the demands of society.CIEC - Centro de Investigação em Estudos da Criança, IE, UMinho (UI 317 da FCT), PortugalFundos Nacionais através da FCT (Fundação para a Ciência e a Tecnologia) e cofinanciado pelo Fundo Europeu de Desenvolvimento Regional (FEDER) através do COMPETE 2020 – Programa Operacional Competitividade e Internacionalização (POCI) com a referência POCI-01-0145-FEDER-007562info:eu-repo/semantics/publishedVersio

A systemic study of learners' knowledge sharing and collaborative skills development : a case study in a British business school

Knowledge management, as Leistner (2010) argues, is a “misnomer”. Knowledge cannot be managed since it relates to prior experience and is present merely in the mind of individuals (p. 4). We can manage knowledge flow, but not the knowledge itself. Leistner states that ‘‘you can enable a flow by creating an environment that people find safe, attractive, and efficient, and that motivates them to share their knowledge. This could be either face-to-face or by recording relevant information that can be used by others to re-create knowledge in their own frame of reference’’ (p. 10). Therefore, managing the flow is “as much about creating conditions that will make sharing more likely as it is about trying to have a direct influence on people’’ (pp. 17-18).In the arena of creating such conditions, operational research (OR) is assumed to offer special capacities to lead the advancements in knowledge management and knowledge sharing research. However, the role of OR is not clear in knowledge management. There is also very little account of OR studies concerning knowledge management in combination with social network analysis. This situation has not changed over past years. In addition, although soft-OR tools promote specific solutions with which to tackle complexity management in organisations, there are very few studies concerning the use of action research and soft-OR tools such as the Viable System Model, which are designed specifically for knowledge sharing projects and simulating social networks.This research intends to design, develop and implement a soft canonical operational research (SCOR) methodological framework for the processes of knowledge sharing. The researcher combines Davison et al.’s (2012) canonical action research and Checkland’s (1985) F-M-A soft account of action research. The framework has, in itself, an embedded solution for skill development and performance improvement through collaborative knowledge sharing and experiential learning/practising. In this research, a combinative perspective of VSM and SNA is considered.Adopting a pragmatic philosophy with an interpretivist ontology and relativist epistemology, the researcher inductively conducted two cycles of action research and analysed the outcomes. Four types of transformations occurred in (1) individuals’ skill level, (2) performance, (3) knowledge network and (4) gradual development of strategies across levels. This research elucidates said transformations and explains the key mechanisms for facilitating collective knowledge sharing in order to develop skills and to improve performance. It also brings to light the evidence regarding two unplanned phenomena that occurred in both cycles: leadership development and autopoiesis.Reflection is provided on the design of the soft-OR multi-methodology and on how this design has been useful and effective in the present research. In addition, the study’s contributions to knowledge and practice are also explained. This research suggests that guided self-organisation is a more effective approach for skill development than traditional methods and that it can create an effective context in which a knowledge network is able to reproduce itself. Finally, the limitations of the research and implications for future studies are clarified