Are students in graduate programmes adequately attaining professional skills?

© 2017 WIETE. A method known as the computing professional skills assessment (CPSA), developed by this team, assesses the professional skills identified by ABET as essential for computing graduates. The method allows simultaneous evaluation of six skills: problem solving, teamwork, professional and ethical considerations, communication, local and global impacts, engagement in further research and learning. The range of the measurement scale is from a skill level of zero up to entry career. The instrument has been tested and validated with undergraduate students. As the top end of the scale corresponds to the attainment level for professionals in entry level positions, here the authors trialled it with a class of 16 postgraduate students who were expected to achieve at that level. The authors found that the students performed near to the target level. As the method is also well suited to teaching the skills, they conducted a survey of the students\u27 perceptions of its benefits. The results showed the students were very positive about the method and felt it was very beneficial for developing their professional skills

The computing professional skills assessment: An innovative method for assessing ABET\u27s student outcomes

© 2016 IEEE. ABET (formerly known as the Accreditation Board for Engineering and Technology Education) prescribes a number of outcomes that students should attain by graduation. Among these are six non-technical skills such as discerning ethical, legal, security and social issues to the ability to engage in continuing professional development. Here we present the first and only direct method and measurement tool in the literature to measure the six ABET Computing Accreditation Commission non-technical skills for both course and program level assessment. The Computing Professional Skills Assessment is a discussion-based performance task, conducted through asynchronous online discussions, designed to elicit students\u27 knowledge and application of professional skills. Our trial has shown that the method can successfully measure the skills. This paper outlines the method and its implementation and describes data collection and findings

Teaching and measuring the professional skills of information technology students using a learning oriented assessment task

© 2019 TEMPUS Publications. The Computing Professional Skills Assessment (CPSA) is a method developed by the authors for assessment of the nontechnical skills prescribed by ABET, the accreditation body for engineering and technology for computing students. These non-technical skills, referred to here as professional skills, include teamwork, communication and problem solving. With the CPSA, teams of five students analyse a complex, ill-defined problem over a 12-day period using an online, asynchronous discussion board. The discussion transcripts are subsequently examined using a rubric. This rigorous assessment evaluates all of the professional skills simultaneously and has been proven to be valid and reliable. As it is a demanding assessment running over a period of almost two weeks, the authors believe it to also be a very valuable learning activity representative of a learning oriented assessment. To ascertain the learning that occurs through use of the CPSA, it was implemented three times in three sections of a 3rd year computing course with a total of 56 students. The results which are presented here show that there was considerable learning and improvement in the students\u27 targeted skills over the semester. The students were surveyed on their perceptions of the CPSA as a learning tool at the end of semester. They strongly believed that it is an effective teaching and learning method and that they benefitted significantly

Student Perceptions of an Asychronous Online Discussion Board Used to Assess the Professional Skills

Though not originally developed for teaching, asynchronous online discussion boards (AODBs) have become important tools for online and blended learning, while also having expanded into some face-to-face teaching environments. Because of this, it is imperative that ongoing research is conducted into how they can be integrated in courses, how they are perceived by students, and how they operate when used as assessments. This paper examines student perceptions of a particular AODB activity, the Computing Professional Skills Assessment (CPSA). The professional skills are the 21st century, cross-disciplinary, or transferable learning outcomes including but not limited to problem-solving, communication, teamwork, and critical thinking, and they are globally recognized as essential for employment within a knowledge economy. Utilizing the CPSA as a direct assessment within a traditional face-to-face course, through an anonymous survey students provide their insights into the degree to which the AODB enhanced professional skills learning outcomes and shared what they liked, disliked, and would like to change about the AODB

Collaborative problem-solving through asynchronous discussion

© WIETE 2020 The reality of today’s workplace for engineers and computing professionals is that the ability to solve ill-defined and complex problems is a much-needed employment skill. The complexity of existing problems dictates that these can rarely be solved by an individual working alone. It is only by working in collaborative teams that such problems can be effectively addressed. This skill, encompassing the common 21st Century skills of problem-solving and teamwork, is often referred to as collaborative problem-solving (CPS). The authors examine the ability of computing students at a UAE university to collaboratively problem-solve through the use of a scenario-based teaching and assessment tool that is delivered as an asynchronous discussion board. It then describes the research method and how the analysis is utilised and discusses the impact of the findings. Results indicate that students performed better in the skill of teamwork when compared to problem-solving, and even though only two groups achieved or exceeded the target mean score for CPS, the overall mean was at the desired target. This suggests there is a need for expanded curricular opportunities for students to work on ill-defined, complex and multidimensional problems in a collaborative group setting

Concurrent direct assessment of foundation skills for general education

© 2018, Emerald Publishing Limited. Purpose: There is considerable agreement around the foundation skills required by employers that will enable graduates to integrate and devise promising solutions for the challenges faced by knowledge and globalized societies. These are life skills (communication skills, teamwork and leadership skills, language skills in reading and writing and information literacy), transferable skills (such as problem-solving, including critical thinking, creativity and quantitative reasoning) and technology skills (search for knowledge and build upon it). Foundation skills, however, are recognized to be difficult both to teach and assess. This paper aims to describe a performance assessment method to assess and measure these skills in a uniquely concurrent way – the General Education Foundation Skills Assessment (GEFSA). Design/methodology/approach: The GEFSA framework comprises a scenario/case describing an unresolved contemporary issue, which engages student groups in online discussions, and a task-specific analytic rubric to concurrently assess the extent to which students have attained the targeted foundation skills. The method was applied in three semesters – during 2016 and 2017. These students were non-native English speaking students in a General Education program at a university in the UAE. Findings: Results obtained from the rubric for each foundation skill were analyzed and interpreted to ensure robustness of method and tool usability and reliability, provide insight into, and commentary on, the respective skill attainment levels and assist in establishing realistic target ranges for General Education student skill attainment. The results showed that the method is valid and provides valuable data for curriculum development. Originality/value: This is the first method in published literature that directly assesses the foundation skills for General Education students simultaneously, thus providing educators with valuable data on the skill level of the students. Additionally, repeated use of the method is a valuable way of teaching skills

A new approach for assessing ABET\u27s professional skills in computing

© 2016 WIETE. In the fields of engineering and computing, the Accreditation Board for Engineering and Technology (ABET) education places much emphasis on professional skills, such as the ability to engage in lifelong learning and to function successfully on a multi-disciplinary team. The recently developed engineering professional skills assessment (EPSA) simultaneously measures ABET\u27s non-technical skills for programme and course level assessment. The EPSA is a discussion-based performance task designed to elicit students\u27 knowledge and application of professional skills. A research project is underway to adapt the method to the field of computing and develop the computing professional skills assessment (CPSA). The CPSA consists essentially of a scenario, a student discussion of the scenario and a rubric to grade the discussion. This article describes the work completed during the first year of the project and the results of the first complete iteration. The results demonstrate that the CPSA can successfully measure the professional skills

An Effective Rubric Norming Process

Within higher education, rubric use is expanding. Whereas some years ago the topic of rubrics may have been of interest only to faculty in colleges of education, in recent years the focus on teaching and learning and the emphasis from accrediting bodies has elevated the importance of rubrics across disciplines and different types of assessment. One of the key aspects to successful implementation of a shared rubric is the process known as norming, calibrating, or moderating rubrics, an oft-neglected area in rubric literature. Norming should be a collaborative process built around knowledge of the rubric and meaningful discussion leading to evidence-driven consensus, but actual examples of norming are rarely available to university faculty. This paper describes the steps involved in a successful consensus-driven norming process in higher education using one particular rubric, the Computing Professional Skills Assessment (CPSA). The steps are: 1) document preparation; 2) rubric review; 3) initial reading and scoring of one learning outcome; 4) initial sharing/recording of results; 5) initial consensus development and adjusting of results; 6) initial reading and scoring of remaining learning outcomes; 7) reading and scoring of remaining transcripts; 8) sharing/recording results; 9) development of consensus and adjusting of results. This norming process, though used for the CPSA, is transferable to other rubrics where faculty have come together to collaborate on grading a shared assignment. It is most appropriate for higher education where, more often than not, faculty independence requires consensus over directive. Accessed 1,689 times on https://pareonline.net from September 12, 2018 to December 31, 2019. For downloads from January 1, 2020 forward, please click on the PlumX Metrics link to the right

Assessment of learning outcomes through an asynchronous on-line discussion board

Asynchronous on-line discussion boards (AODBs) have become a standard pedagogical tool, and the use of learning outcomes has become normalised in much of higher education, yet little is known about using AODBs to assess learning outcomes. Hence, this article aims to assess the degree to which students demonstrate learning outcomes attainment through participation in an AODB. The research method implemented is built around a particular AODB instrument designed to teach and assess cross-disciplinary learning outcomes. Using a sample group of 30 computing students studying at a public university in the United Arab Emirates, this article demonstrates that a well-designed and structured AODB can be used to teach and assess important cross-disciplinary learning outcomes, and that discussions can be assessed in an efficient manner. By examining both student group and learning outcome scores, results indicate that students showed improvement in learning outcomes attainment across a semester. Given the recent growth in on-line learning, this study supports the need for more research into the use of AODBs as a method to teach and assess learning outcomes

An effective rubric norming process

© 2018 Practical Assessment, Research & Evaluation. Within higher education, rubric use is expanding. Whereas some years ago the topic of rubrics may have been of interest only to faculty in colleges of education, in recent years the focus on teaching and learning and the emphasis from accrediting bodies has elevated the importance of rubrics across disciplines and different types of assessment. One of the key aspects to successful implementation of a shared rubric is the process known as norming, calibrating, or moderating rubrics, an oft-neglected area in rubric literature. Norming should be a collaborative process built around knowledge of the rubric and meaningful discussion leading to evidence-driven consensus, but actual examples of norming are rarely available to university faculty. This paper describes the steps involved in a successful consensus-driven norming process in higher education using one particular rubric, the Computing Professional Skills Assessment (CPSA). The steps are: 1) document preparation; 2) rubric review; 3) initial reading and scoring of one learning outcome; 4) initial sharing/recording of results; 5) initial consensus development and adjusting of results; 6) initial reading and scoring of remaining learning outcomes; 7) reading and scoring of remaining transcripts; 8) sharing/recording results; 9) development of consensus and adjusting of results. This norming process, though used for the CPSA, is transferable to other rubrics where faculty have come together to collaborate on grading a shared assignment. It is most appropriate for higher education where, more often than not, faculty independence requires consensus over directive