Automatic Assessment of Programming Assignments using Image Recognition

Programmeerimisülesannete automaatne kontrollimine on vaba juurdepääsuga e-kursuste ehk MOOCide (Massive Open Online Course) juures hädavajalik suure hulga esitatud lahenduste tõttu. Ülesannete nõuded peavad olema detailselt sõnastatud, et neid oleks võimalik automaatselt kontrollida. Väga täpsed nõuded ülesannetele piiravad jällegi nende loomingulisust. Probleemi leevendamiseks loodi antud lõputöö raames süsteem, mis suudab automaatselt hinnata programmeerimisalaste ülesannete graafilist väljundit, kasutades pildituvastust (image recognition). Süsteemi rakendatakse algajatele mõeldud programmeerimisalaste ülesannete puhul, mille lahenduseks on soovitud objekti graafilise väljundiga programmid. Lahendusprogrammist genereeritud pilti analüüsitakse pildituvastusega, mille tulemuseks on arv, mis näitab tõenäosust, et soovitud objekt oleks pildil. Esitus on arvestatud või mittearvestatud vastavalt eelmainitud tõenäosusele. Valminud süsteemi testiti MOOCi peal, täpsemini 2272 esitatud lahenduse peal. 4.6% tulemustest olid valenegatiivsed ning 0.5% tulemustest valepositiivsed. Kursuse jooksul läbi viidud vaheküsitlusest selgus, et 82.1% vastanute arvates töötas süsteem hästi või väga hästi ning keskmine hinnang süsteemile 5 palli skaalal oli 4.4.Automatic assessment of programming tasks in MOOCs (Massive Open Online Courses) is essential due to the large number of submissions. However, this often limits the scope of the assignments since task requirements must be strict for the solutions to be automatically gradable, reducing the opportunity for solutions to be creative. In order to alleviate this problem, we introduce a system capable of assessing the graphical output of a solution program using image recognition. This idea is applied to introductory computer graphics programming tasks which solutions are programs that produce images of a given object on the screen. The image produced by the solution program is analysed using image recognition, resulting in a probability of a given object appearing in the image. The solution is accepted or rejected based on this score. The system was tested in a MOOC on 2,272 solution submissions. The results contained 4.6% cases of false negative and 0.5% cases of false positive grades. A participant survey revealed that the system was perceived to be functioning well or very well by 82.1% of the respondents, with an average rating of 4.4 out of 5

Automatic Image Marking Process

Abstract-Efficient evaluation of student programs and timely processing of feedback is a critical challenge for faculty. Despite persistent efforts and significant advances in this field, there is still room for improvement. Therefore, the present study aims to analyse the system of automatic assessment and marking of computer science programming students’ assignments in order to save teachers or lecturers time and effort. This is because the answers are marked automatically and the results returned within a very short period of time. The study develops a statistical framework to relate image keywords to image characteristics based on optical character recognition (OCR) and then provides analysis by comparing the students’ submitted answers with the optimal results. This method is based on Latent Semantic Analysis (LSA), and the experimental results achieve high efficiency and more accuracy by using such a simple yet effective technique in automatic marking

Automatic Assessment in Undergraduate Level Engineering Drawing

AutoCAD is the most popular software used in undergraduate level Engineering Drawing education. Currently, there are no available methods to automate the marking process of AutoCAD assignments. This project is attempting to create a software prototype that can conceptually show how the marking process of AutoCAD assignments can be automated. The prototype will first convert the AutoCAD DXF file into SVG format and then evaluate each of the elements and attributes. The prototype is a web-based application and is able to achieve the aforementioned objectives. Given appropriate time and resources, this project can be extended into a full scale working application

Technology assessment of advanced automation for space missions

Six general classes of technology requirements derived during the mission definition phase of the study were identified as having maximum importance and urgency, including autonomous world model based information systems, learning and hypothesis formation, natural language and other man-machine communication, space manufacturing, teleoperators and robot systems, and computer science and technology

Markerless Motion Capture in the Crowd

This work uses crowdsourcing to obtain motion capture data from video recordings. The data is obtained by information workers who click repeatedly to indicate body configurations in the frames of a video, resulting in a model of 2D structure over time. We discuss techniques to optimize the tracking task and strategies for maximizing accuracy and efficiency. We show visualizations of a variety of motions captured with our pipeline then apply reconstruction techniques to derive 3D structure.Comment: Presented at Collective Intelligence conference, 2012 (arXiv:1204.2991