Evaluating The Effectiveness of Live Peer Assessment as a Vehicle for The Development of Higher Order Practice in Computer Science Education

This thesis concerns a longitudinal study of the practice of Live Peer Assessment on two University courses in Computer Science. By Live Peer Assessment I mean a practice of whole-class collective marking using electronic devices of student artefacts demonstrated in a class or lecture theatre with instantaneous aggregated results displayed on screen immediately after each grading decision. This is radically different from historical peer-assessment in universities which has primarily been asynchronous process of marking of students’ work by small subsets of the cohort (e.g. 1 student artefact is marked by <3 fellow students). Live Peer Assessment takes place in public, is marked by (as far as practically possible) the whole cohort, and results are instantaneous. This study observes this practice, first on a level 4 course in E-Media Design where students’ main assignment is a multimedia CV (or resume) and secondly on a level 7 course in Multimedia Specification Design and Production where students produce a multimedia information artefact in both prototype and final versions. In both cases, students learned about these assignments from reviewing works done by previous students in Live Peer Evaluation events where they were asked to collectively publicly mark those works according to the same rubrics that the tutors would be using. In this level 4 course, this was used to help students get a better understanding of the marks criteria. In the level 7 course, this goal was also pursued, but was also used for the peer marking of students’ own work. Among the major findings of this study are: In the level 4 course student attainment in the final assessment improved on average by 13% over 4 iterations of the course, with very marked increase among students in the lower percentiles The effectiveness of Live Peer Assessment in improving student work comes from o Raising the profile of the marking rubric o Establishing a repertoire of example work o Modelling the “noticing” of salient features (of quality or defect) enabling students to self-monitor more effectively In the major accepted measure of peer-assessment reliability (correlation between student awarded marks and tutor awarded marks) Live Peer Assessment is superior to traditional peer assessment. That is to say, students mark more like tutors when using Live Peer Assessment In the second major measure (effect-size) which calculates if students are more strict or generous than tutors, (where the ideal would be no difference), Live Peer Assessment is broadly comparable with traditional peer assessment but this is susceptible to the conditions under which it takes place The reason for the better greater alignment of student and tutor marks comes from the training sessions but also from the public nature of the marking where individuals can compare their marking practice with that of the rest of the class on a criterion by criterion basis New measures proposed in this thesis to measure the health of peer assessment events comprise: Krippendorf’s Alpha, Magin’s Reciprocity Matrix, the median pairwise tutor student marks correlation, the Skewness and Kurtosis of the distribution of pairwise tutor student marking correlations Recommendations for practice comprise that: o summative peer assessment should not take place under conditions of anonymity but that very light conditions of marking competence should be enforced on student markers (e.g. >0.2 correlation between individual student marking and that of tutors) o That rubrics can be more suggestive and colloquial in the conditions of Live Peer Assessment because the marking criteria can be instantiated in specific examples of student attainment and therefore the criteria may be less legalistically drafted because a more holistic understanding of quality can be communicate

Assessing program code through static structural similarity

Learning to write software requires much practice and frequent assessment. Consequently, the use of computers to assist in the assessment of computer programs has been important in supporting large classes at universities. The main approaches to the problem are dynamic analysis (testing student programs for expected output) and static analysis (direct analysis of the program code). The former is very sensitive to all kinds of errors in student programs, while the latter has traditionally only been used to assess quality, and not correctness. This research focusses on the application of static analysis, particularly structural similarity, to marking student programs. Existing traditional measures of similarity are limiting in that they are usually only effective on tree structures. In this regard they do not easily support dependencies in program code. Contemporary measures of structural similarity, such as similarity flooding, usually rely on an internal normalisation of scores. The effect is that the scores only have relative meaning, and cannot be interpreted in isolation, ie. they are not meaningful for assessment. The SimRank measure is shown to have the same problem, but not because of normalisation. The problem with the SimRank measure arises from the fact that its scores depend on all possible mappings between the children of vertices being compared. The main contribution of this research is a novel graph similarity measure, the Weighted Assignment Similarity measure. It is related to SimRank, but derives propagation scores from only the locally optimal mapping between child vertices. The resulting similarity scores may be regarded as the percentage of mutual coverage between graphs. The measure is proven to converge for all directed acyclic graphs, and an efficient implementation is outlined for this case. Attributes on graph vertices and edges are often used to capture domain specific information which is not structural in nature. It has been suggested that these should influence the similarity propagation, but no clear method for doing this has been reported. The second important contribution of this research is a general method for incorporating these local attribute similarities into the larger similarity propagation method. An example of attributes in program graphs are identifier names. The choice of identifiers in programs is arbitrary as they are purely symbolic. A problem facing any comparison between programs is that they are unlikely to use the same set of identifiers. This problem indicates that a mapping between the identifier sets is required. The third contribution of this research is a method for applying the structural similarity measure in a two step process to find an optimal identifier mapping. This approach is both novel and valuable as it cleverly reuses the similarity measure as an existing resource. In general, programming assignments allow a large variety of solutions. Assessing student programs through structural similarity is only feasible if the diversity in the solution space can be addressed. This study narrows program diversity through a set of semantic preserving program transformations that convert programs into a normal form. The application of the Weighted Assignment Similarity measure to marking student programs is investigated, and strong correlations are found with the human marker. It is shown that the most accurate assessment requires that programs not only be compared with a set of good solutions, but rather a mixed set of programs of varying levels of correctness. This research represents the first documented successful application of structural similarity to the marking of student programs

Simulation in medical education : a case study evaluating the efficacy of high-fidelity patient simulation

Indiana University-Purdue University Indianapolis (IUPUI)High-fidelity patient simulation (HFPS) recreates clinical scenarios by combining mock patients and realistic environments to prepare learners with practical experience to meet the demands of modern clinical practice while ensuring patient safety. This research investigated the efficacy of HFPS in medical education through a case study of the Indiana University Bloomington Interprofessional Simulation Center. The goal of this research was to understand the role of simulated learning for attaining clinical selfefficacy and how HFPS training impacts performance. Three research questions were addressed to investigate HFPS in medical education using a mixed methods study design. Clinical competence and self-efficacy were quantified among medical students at IUSMBloomington utilizing HFPS compared to two IUSM campuses that did not incorporate this instructional intervention. Clinical competence was measured as performance on the Objective Structured Clinical Examination (OSCE), while self-efficacy of medical students was measured through a validated questionnaire. Although the effect of HFPS on quantitative results was not definitive, general trends allude to the ability of HFPS to recalibrate learners’ perceived and actual performance. Additionally, perceptual data regarding HFPS from both medical students and medical residents was analyzed. Qualitative results discovered the utility of HFPS for obtaining the clinical mental framework of a physician, fundamental psychomotor skills, and essential practice communicating and functioning as a healthcare team during interprofessional education simulations. Continued studies of HFPS are necessary to fully elucidate the value of this instructional adjunct, however positive outcomes of simulated learning on both medical students and medical residents were discovered in this study contributing to the existing HFPS literature

Critical Thinking Skills Profile of High School Students In Learning Science-Physics

This study aims to describe Critical Thinking Skills high school students in the city of Makassar. To achieve this goal, the researchers conducted an analysis of student test results of 200 people scattered in six schools in the city of Makassar. The results of the quantitative descriptive analysis of the data found that the average value of students doing the interpretation, analysis, and inference in a row by 1.53, 1.15, and 1.52. This value is still very low when compared with the maximum value that may be obtained by students, that is equal to 10.00. This shows that the critical thinking skills of high school students are still very low. One fact Competency Standards science subjects-Physics is demonstrating the ability to think logically, critically, and creatively with the guidance of teachers and demonstrate the ability to solve simple problems in daily life. In fact, according to Michael Scriven stated that the main task of education is to train students and or students to think critically because of the demands of work in the global economy, the survival of a democratic and personal decisions and decisions in an increasingly complex society needs people who can think well and make judgments good. Therefore, the need for teachers in the learning device scenario such as: driving question or problem, authentic Investigation: Science Processes

An analysis of 2005 NAEP 8th grade mathematics achievement items by content strand, problem type and language complexity

unavailabl

New Fundamental Technologies in Data Mining

The progress of data mining technology and large public popularity establish a need for a comprehensive text on the subject. The series of books entitled by "Data Mining" address the need by presenting in-depth description of novel mining algorithms and many useful applications. In addition to understanding each section deeply, the two books present useful hints and strategies to solving problems in the following chapters. The contributing authors have highlighted many future research directions that will foster multi-disciplinary collaborations and hence will lead to significant development in the field of data mining

Challenges for engineering students working with authentic complex problems

Engineers are important participants in solving societal, environmental and technical problems. However, due to an increasing complexity in relation to these problems new interdisciplinary competences are needed in engineering. Instead of students working with monodisciplinary problems, a situation where students work with authentic complex problems in interdisciplinary teams together with a company may scaffold development of new competences. The question is: What are the challenges for students structuring the work on authentic interdisciplinary problems? This study explores a three-day event where 7 students from Aalborg University (AAU) from four different faculties and one student from University College North Denmark (UCN), (6th-10th semester), worked in two groups at a large Danish company, solving authentic complex problems. The event was structured as a Hackathon where the students for three days worked with problem identification, problem analysis and finalizing with a pitch competition presenting their findings. During the event the students had workshops to support the work and they had the opportunity to use employees from the company as facilitators. It was an extracurricular activity during the summer holiday season. The methodology used for data collection was qualitative both in terms of observations and participants’ reflection reports. The students were observed during the whole event. Findings from this part of a larger study indicated, that students experience inability to transfer and transform project competences from their previous disciplinary experiences to an interdisciplinary setting

Exploring the practical use of a collaborative robot for academic purposes

This article presents a set of experiences related to the setup and exploration of potential educational uses of a collaborative robot (cobot). The basic principles that have guided the work carried out have been three. First and foremost, study of all the functionalities offered by the robot and exploration of its potential academic uses both in subjects focused on industrial robotics and in subjects of related disciplines (automation, communications, computer vision). Second, achieve the total integration of the cobot at the laboratory, seeking not only independent uses of it but also seeking for applications (laboratory practices) in which the cobot interacts with some of the other devices already existing at the laboratory (other industrial robots and a flexible manufacturing system). Third, reuse of some available components and minimization of the number and associated cost of required new components. The experiences, carried out following a project-based learning methodology under the framework of bachelor and master subjects and thesis, have focused on the integration of mechanical, electronic and programming aspects in new design solutions (end effector, cooperative workspace, artificial vision system integration) and case studies (advanced task programming, cybersecure communication, remote access). These experiences have consolidated the students' acquisition of skills in the transition to professional life by having the close collaboration of the university faculty with the experts of the robotics company.Postprint (published version