ProCAss: An intelligent assessment for computer programming corpus

Electronic assessment (e-assessment) becomes more popular in educational tools especially in e-learning environment.This is because it has some advantages such as reducing the staffs needed for assessment tasks, automated marking is not prone to human error and it gives instant feedback to the students.However, the e-assessment existed only more on assessing essays and lack of implementation in assessing computer program in computer science environment [Haley, et.al, 2003]. This paper will briefly describe on the techniques that have been used to implement e-assessment for essays such as Project Essay Grade (PEG), E-rater, Intelligent Essay Assessor (IEA) and Latent Semantic Analysis (LSA).Mainly, the discussion will concentrate on LSA to build the eassessment for computer programming corpus.One great advantage of LSA over the others is its ability to make absolute relative comparisons, where set of documents can be compared to each other, or to an answer schema.Another reason is the computer program produces programs in a subset of English-like words, a bit similar to an essay [Haley, et.al, 2003].This paper will propose system design integrated with LSA method to assess student’s programming assignments. Then, the ability of LSA algorithm in grading computer program corpus will be evaluated.The grading process will not limited on certain programming languages, but on any programming languages

Software Verification and Graph Similarity for Automated Evaluation of Students' Assignments

In this paper we promote introducing software verification and control flow graph similarity measurement in automated evaluation of students' programs. We present a new grading framework that merges results obtained by combination of these two approaches with results obtained by automated testing, leading to improved quality and precision of automated grading. These two approaches are also useful in providing a comprehensible feedback that can help students to improve the quality of their programs We also present our corresponding tools that are publicly available and open source. The tools are based on LLVM low-level intermediate code representation, so they could be applied to a number of programming languages. Experimental evaluation of the proposed grading framework is performed on a corpus of university students' programs written in programming language C. Results of the experiments show that automatically generated grades are highly correlated with manually determined grades suggesting that the presented tools can find real-world applications in studying and grading

Evaluation of a tool for Java structural specification checking

Although a number of tools for evaluating Java code functionality and style exist, little work has been done in a distance learning context on automated marking of Java programs with respect to structural specifications. Such automated checks support human markers in assessing students’ work and evaluating their own marking; online automated marking; students checking code before submitting it for marking; and question setters evaluating the completeness of questions set. This project developed and evaluated a prototype tool that performs an automated check of a Java program’s correctness with respect to a structural specification. Questionnaires and interviews were used to gather feedback on the usefulness of the tool as a marking aid to humans, and on its potential usefulness to students for self-assessment when working on their assignments. Markers were asked to compare the usefulness of structural specification testing as compared to other kinds of support, including syntax error assistance, style checking and functionality testing. Initial results suggest that most markers using the structural specification checking tool found it to be useful, and some reported that it increased their accuracy in marking. Reasons for not using the tool included lack of time and the simplicity of the assignment it was trialled on. Some reservations were expressed about reliance on tools for assessment, both for markers and for students. The need for advice on incorporating tools in marking workflow is suggested

Marking complex assignments using peer assessment with an electronic voting system and an automated feedback tool

The work described in this paper relates to the development and use of a range of initiatives in order to mark complex masters' level assignments related to the development of computer web applications. In the past such assignments have proven difficult to mark since they assess a range of skills including programming, human computer interaction and design. Based on the experience of several years marking such assignments, the module delivery team decided to adopt an approach whereby the students marked each other's practical work using an electronic voting system (EVS). The results of this are presented in the paper along with statistical comparison with the tutors' marking, providing evidence for the efficacy of the approach. The second part of the assignment related to theory and documentation. This was marked by the tutors using an automated feedback tool. It was found that the time to mark the work was reduced by more than 30% in all cases compared to previous years. More importantly it was possible to provide good quality individual feedback to learners rapidly. Feedback was delivered to all within three weeks of the test submission datePeer reviewe

An integrated approach to courseware

Software engineering is becoming increasingly important as an engineering discipline, and its teaching in universities and other higher education institutions should be of high quality. In this paper we describe a tool (BOSS — the Boss Online Submission System) which aids the education of software engineers. BOSS allows students to submit programming assignments online, and to run black-box tests on their programs prior to submission. Instructors can use BOSS to assist in marking such assignments by allowing submitted programs to be tested against multiple data sets. We describe how BOSS helps in the teaching of specific conceptual aspects of software engineering, and how it addresses some of the practical issues involved in teaching large student numbers in a pedagogically neutral manner