Tuning in zoning out : student assessment in teacher education

Assessment in higher education and teacher education in particular continues to be a prominent feature of the student experience. For teacher education students there is an immediate and future concern as they both experience assessment and go on to become assessors of learning within their profession. In the state of Queensland, Australia, assessment in teacher education has taken on renewed importance with pre-registration testing of new graduates. Assessment is a phenomenon that influences nearly all aspects of students’ experience of university – their learning, relationships with staff and peers, and their emotions. This paper reports on data from the author’s reflective journal where insights were derived from conversations about assessment during student interactions. Extracts from this reflective writing enabled extrapolation of key issues of student perceptions, learning, relationships and emotions. What students say and do in relation to assessment offers lecturers powerful insights into the central role it plays in their experience of university. Students are active in shaping their assessment experience but are clearly also shaped by it. Lecturers play important roles in sustaining and challenging assessment practices that influence students in complex ways

Contextualising research in AQF8 for engineering education

Context: Engineering education historically has been shaped by professional accreditation and internal university imperatives. The recent strengthening of the Australian Qualifications Framework (AQF) adds a new layer of thinking and has required many universities to re-examine how embedded honours degrees are awarded and the degree to which undergraduate programs reflect the level 8 requirements. The AQF is necessarily broad as it applies to all disciplines and fields of education. What is needed is a way of contextualising AQF requirements for engineering education and a means for understanding how these are similar to, or differ from, Engineers Australia’s Stage One Competencies. This paper represents a work in progress that is designed to ignite thinking about AQF8 implications – particularly those around research in the final year engineering project. Purpose: An Office for Learning and Teaching (OLT) project set out to map practices in relation to final year engineering projects and to consider the implications for compliance with AQF8 outcomes. One aspect of the study looked specifically at how AQF8 outcomes are understood by coordinators and the teamcame to examine the nature and role of research in final year projects. From here, the team sought to generate a contextualised definition for AQF8 outcomes as they might apply to engineering education. Although there are other points of differentiation between AQF7 and AQF8, the team sought, in particular, to develop a broadly accepted definition for research. Approach: This paper reports on one part of the larger project. The wider project methodology was largely qualitative, adopting a case study approach. Data was gathered from 16 universities across Australia (from all states and territories) and included university documentation such as subject outlines, rubrics and student guidelines. Additionally, interviews were conducted with coordinators of final year project courses. It was within these interviews that participants were asked specifically about AQF8 outcomes. Additional data was gathered from participants during a conference workshop designed to explore understanding of AQF8 outcomes. The notion of a contextualised definition of research is derived from this data together in collaboration with Engineers Australia. Results: The data revealed that universities are at different stages of AQF compliance. Further, there is mixed understanding in relation to AQF8 with degrees of uncertainty about some terms used in the document, particularly around what comprises research. There is some sense that some engineering programs are already meeting the higher order requirements of AQF8 outcomes and that compliance is merely procedural. Beyond this, however, is the revelation that there is a great deal of contention around research and autonomy as defined by AQF. It was in this context that the OLT team sought to develop a definition of research as applied to the final year or capstone project. The definition delineates specific engineering knowledge, skills and application as would be seen in the final year project. Conclusions: This paper proposes a nationally supported definition of what research means within the undergraduate engineering education context and in doing so assists linking AQF8 directly to final year projects specifically, but to embedded honours programs more generally

Final Year Engineering Projects : improving assessment, curriculum and supervision to meet AQF8 outcomes

Backround: A substantial project exploring the organisation, supervision and assessment of final year engineering projects in Australia has mapped practices across 16 universities. It addressed the need that although Australia has a strong history of developing Final Year Engineering Projects (FYEPs) as capstone courses in engineering education, there is no national approach to assessment or supervision. Practices are varied and the project team has worked towards identifying good practice such that universities are better positioned to meet Australian Qualification Framework (AQF) level 8 outcomes. Purpose: The study set out to firstly map practices in relation to final year engineering or capstone projects and to consider the implications for compliance with AQF8 outcomes. The study explored how projects are developed, how they are supervised and assessed and explored the challenges faced by universities in FYEP practice. Data from phase 1, together with the national and international literature, has been used to inform the development of good practice guidelines. These guidelines have been developed specifically to assist universities in ensuring their undergraduate engineering embedded honours degrees meet and reflect AQF level 8 outcomes in the key areas of curriculum, supervision and assessment. Design/Method: The project methodology was largely qualitative, adopting a case study approach. Data was gathered from 16 universities across Australia (from all states and territories) and included university documentation such as subject outlines, rubrics and student guidelines. Additionally, interviews were conducted with coordinators of final year project courses. Data was also gathered from participants during a conference workshop designed to explore understanding of AQF8. All data was coded and analysed inductively and deductively for themes. This data was then used to develop guidelines and resources for national dissemination. Results: The study has revealed great variation in assessment, curriculum and supervision practices in FYEPs across Australian universities and details of the nature of this variation is discussed. However, the study has also been able to identify good and next practice as seen by FYEP coordinators across a wide range of universities. Additionally, analysis in relation to AQF8 has enabled the team to propose guidelines and resources to support best practice in these areas and specifically in relation to meeting AQF8 outcomes. Conclusions: This paper presents the overarching findings of a large study and whilst it initially confirms what is already known – that there is variation in how FYEPs are taught, assessed and supervised across Australia – it moves into new space by articulating the nature of these variations and proposing good and next practice guidelines. It also sets the scene for the related and more specific papers on curriculum, assessment and supervision

Learning and teaching of Capstone Final Year Engineering Projects : an Australian study

The learning and teaching methodology of the capstone final year engineering project (FYEP) as employed in Australia is presented and discussed in this paper. A questionnaire was conducted to answer a broad research question: What is the current approach used in learning and teaching of capstone FYEPs? The questionnaire outcomes and a number of common issues, discrepancies and inconsistencies found are outlined in the paper. The study indicates the need to engage in further dialogue with supervisors, professors and students to develop best practice in the FYEP paradigm

Supporting students through the final year engineering project experience to achieve AQF8 outcomes

Backround: The final year engineering project is widely seen as the capstone experience in engineering programs. For four year engineering programs aspiring to AQF8 accreditation, it is essential that the final year projects provide students the opportunity to demonstrate many of the descriptors for Bachelor Honours degrees. However there is wide variation in both the type of project that students undertake and the level and style of support that they receive from their academic supervisors. This paper presents one set of guidelines to emerge from a national project focused on the assessment and supervision of final year projects. The guidelines are focused on best supervision practice. Purpose: This paper provides academic staff good practice guidelines on how to provide appropriate support to students through the final year project experience. Support provided should give the student an opportunity to demonstrate their ability to ‘provide solutions to complex problems with intellectual independence’ and to ‘design and use research in a project’ while maximising the chance of student success. The intention of the guidelines is to enhance academic project advisor capacity to ensure students meet the requirements of AQF8 outcomes. Design/Method: The wider project methodology was largely qualitative, adopting a case study approach. Data was gathered from 16 universities across Australia (from all states and territories) and included university documentation such as subject outlines, rubrics and student guidelines. Additionally, interviews were conducted with 16 coordinators of final year project courses. Within these interviews, participants were asked specifically about supervision practices and challenges. Additional data was gathered from participants during a conference workshop designed to explore understandings of AQF8.The guidelines presented in this paper were developed by analysing these collected data and comparing good practice outcomes with the AQF8 descriptors. Literature regarding the design process, project based-learning facilitation and the systems engineering approach also informed the framework. Results: This paper reports on one outcome from an Office for Learning and Teaching (OLT) sponsored projecton final year engineering projects. The framework and subsequent guidelines describe the process that the student will navigate and the behaviours that the student and the advisor should exhibit at each stage of the process in order to give the student an optimal chance to successfully complete their project while still demonstrating the level of independence characteristic of the AQF8 graduate. Conclusions: Final year engineering projects are a pivotal part of all engineering degrees. There is wide variation in the types of projects and levels of support that academic advisors provide to students throughout this process. The guidelines presented in this paper will assist academic advisors and final year project coordinators to provide quality support to students to meet AQF8 outcomes

Getting it right : assessment tasks and marking for capstone project courses

Capstone projects represent the culmination of an undergraduate engineering degree and are typically the last gatekeeping measure before students graduate and enter the engineering profession. In Australia there is a longstanding interest in and commitment to developing quality capstone experiences. A national study into the supervision and assessment of capstone projects has determined that whilst there is relative consistency in terms of what project tasks are set and assessed, there is not comparable consistency in how these tasks or assignments are marked. Two interconnected areas of assessing process and the role of the supervisor in marking are identified as contentious. This paper presents some findings of a national case study and concludes that whilst further investigation is warranted, assessing process as well as project products is valuable as is the need for greater acceptance of project supervisors as capable of making informed, professional judgments when marking significant project work

Guidelines for curriculum development of final year engineering projects to support achievement of AQF8 outcomes

A substantial project exploring the organisation, supervision and assessment of final year engineering projects in Australia has mapped practices across 16 universities. It addressed the need that although Australia has a strong history of developing FYEPs as capstone courses in engineering education, there is no national approach to curriculum development. Practices are varied and the project team has worked towards identifying good practice such that universities are better positioned to meet Australian Qualification Framework level 8 outcomes. PURPOSE. The study set out to firstly map practices in relation to final year engineering or capstone projects andto consider the implications for compliance with AQF8 outcomes. Data from phase 1, together with the national and international literature, has been used to inform the development of good practice guidelines. These guidelines have been developed specifically to assist universities in ensuring their undergraduate engineering embedded honours degrees meet and reflect AQF level 8 outcomes in the key areas of curriculum, supervision and assessment. DESIGN/METHOD. The project methodology was largely qualitative, adopting a case study approach. Data was gathered from 16 universities across Australia (from all states and territories) and included university documentation such as subject outlines, rubrics and student guidelines. Additionally, interviews were conducted with coordinators of final year project courses. Data was also gathered from participants during a conference workshop designed to explore understanding of AQF8. All data was coded and analysed inductively and deductively for themes. These themes were compared to the AQF8 outcomes, and the outcomes relevant to FYEP were identified. This data was then used to develop curriculum guidelines to support development of AQF8 outcomes. The guidelines have been workshopped at a range of sites throughout the second half of 2014. RESULTS. AQF 8 identifies a range of outcomes that can be demonstrated through the outcomes of FYEP. However the curriculum needs to be consciously developed to ensure that the outcomes are developed by the students. The study has revealed great variation in curriculum development practices in FYEPs across Australian universities. The study has been able to identify guidelines for curriculum development that will support the development of AQF8 outcomes, and identify good practice as seen by FYEP coordinators to achieve the curriculum development. CONCLUSIONS. This paper presents a summary of the guidelines and examples for curriculum development of FYEP to support AQF8 outcomes, which were developed from a large study. While there is variation in the curriculum of FYEPs across Australia, this paper gives examples of good practice that supports the development of AQF8 outcome

Learning and teaching approaches and methodologies of capstone final year engineering projects

Capstone projects represent the culmination of an undergraduate engineering degree and are typically the last checkpoint measure before students graduate and enter the engineering profession. In Australia there is a longstanding interest in and commitment to developing quality capstone experiences. A national study into the supervision and assessment of capstone projects has determined that whilst there is relative consistency in terms of what project tasks are set and assessed, there is not comparable consistency in how these tasks or assignments are marked. Two interconnected areas of assessing process and the role of the supervisor in marking are identified as contentious. This paper presents some findings of a national case study and concludes that whilst further investigation is warranted, assessing process as well as project products is valuable as is the need for greater acceptance of project supervisors as capable of making informed, professional judgments when marking significant project work

Assessment of final year engineering projects : an AQF8 perspective

Context: In undertaking the Office for Learning and Teaching (OLT) Project, Assessing Final Year EngineeringProjects (FYEPs): Ensuring Learning and Teaching Standards and AQF Level 8 Outcomes, theproject team identified three key areas which were common and most pertinent in the survey data: curriculum, supervision and assessment. This paper describes a set of assessment guidelines that were developed as a key outcome for the OLT Project in addition to the supporting body of knowledge, good practices and data collected. Purpose: The first phase of the broader project identified a need for greater consideration of how final year projects demonstrate Level 8 learning outcomes required by the Australian Qualifications Framework for 4-year Bachelor of Engineering (Honours) degrees. The purpose of the second phase was to develop and disseminate guidelines that assist academics to create and assess FYEP tasks in relation to the learning outcome areas of knowledge, skills and application. These guidelines are accompanied by exemplar practice as identified from the review of survey data and are designed to assist academics in the design, assessment and moderation of tasks associated with FYEP work. Approach: The wider project methodology was largely qualitative, adopting a case study approach. Data was gathered from 16 universities across Australia (from all states and territories) and included university documentation such as subject outlines, rubrics and student guidelines. Additionally, interviews were conducted with coordinators of final year project subjects. Within these interviews participants were asked specifically about their assessment practices and AQF level 8. Additional data was gathered from participants during a conference workshop designed to explore their understanding of the AQFLevel 8 learning outcome descriptors. The guidelines were developed after mapping this data against the sections of knowledge, skills and application described in the Level 8 learning outcomes. Results: The dissemination of the Assessment Guidelines and exemplar practice is designed to both capture some of the complexities around assessment of FYEPs and progress practice towards AQF8 compliance. It is anticipated that the adoption of the guidelines within institutions will lead to higher quality assessment practices and delivery of AQF level 8 outcomes. Conclusions: Assessment practices in FYEPs vary considerably across institutions and this variance is seen in both the types of tasks set for students and the ways in which they are marked and moderated. The project team has sought to delineate good practice in this area and disseminate guidelines designed to assist in careful thinking about the high standards implied by AQF level 8

Development and assessment of the final year engineering projects : a review

Accreditation requirements for undergraduate programs for professional engineers require final year students to complete capstone projects, but currently there is no gauge or guarantee of consistency. Practices differ greatly between universities and little work has been initiated that seeks to identify good practice. The literature shows that there are no definite or guaranteed assessment criteria for assessing the Final Year Engineering Projects (FYEPs) highlighting the need for the development of guidelines for the FYEPs and assessment criteria.This paper presents a review on the FYEPs learning and teaching methodologies as employed across several universities at national and international levels and is part of a wider Office for Learning and Teaching (OLT) commissioned study - Assessing FYEPs: Ensuring Learning and Teaching Standards and Australian Qualification Framework (AQF8) Outcomes. This study is intended to promote quality practice amongst supervisors and academics involved in teaching and facilitating FYEPs in Australian universities. This preliminary literature review comprises one part of this wider study