Semi-automatic assessment of unrestrained Java code: a Library, a DSL, and a workbench to assess exams and exercises

© ACM 2015. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in http://dx.doi.org/10.1145/2729094.2742615Automated marking of multiple-choice exams is of great interest in university courses with a large number of students. For this reason, it has been systematically implanted in almost all universities. Automatic assessment of source code is however less extended. There are several reasons for that. One reason is that almost all existing systems are based on output comparison with a gold standard. If the output is the expected, the code is correct. Otherwise, it is reported as wrong, even if there is only one typo in the code. Moreover, why it is wrong remains a mystery. In general, assessment tools treat the code as a black box, and they only assess the externally observable behavior. In this work we introduce a new code assessment method that also verifies properties of the code, thus allowing to mark the code even if it is only partially correct. We also report about the use of this system in a real university context, showing that the system automatically assesses around 50% of the work.This work has been partially supported by the EU (FEDER) and the Spanish Ministerio de Economíay Competitividad (Secretaría de Estado de Investigación, Desarrollo e Innovación) under grant TIN2013-44742-C4-1-R and by the Generalitat Valenciana under grant PROMETEOII2015/013. David Insa was partially supported by the Spanish Ministerio de Educación under FPU grant AP2010-4415.Insa Cabrera, D.; Silva, J. (2015). Semi-automatic assessment of unrestrained Java code: a Library, a DSL, and a workbench to assess exams and exercises. ACM. https://doi.org/10.1145/2729094.2742615SK. A Rahman and M. Jan Nordin. A review on the static analysis approach in the automated programming assessment systems. In National Conference on Programming 07, 2007.K. Ala-Mutka. A survey of automated assessment approaches for programming assignments. In Computer Science Education, volume 15, pages 83--102, 2005.C. Beierle, M. Kula, and M. Widera. Automatic analysis of programming assignments. In Proc. der 1. E-Learning Fachtagung Informatik (DeLFI '03), volume P-37, pages 144--153, 2003.J. Biggs and C. Tang. Teaching for Quality Learning at University : What the Student Does (3rd Edition). In Open University Press, 2007.P. Denny, A. Luxton-Reilly, E. Tempero, and J. Hendrickx. CodeWrite: Supporting student-driven practice of java. In Proceedings of the 42nd ACM technical symposium on Computer science education, pages 09--12, 2011.R. Hendriks. Automatic exam correction. 2012.P. Ihantola, T. Ahoniemi, V. Karavirta, and O. Seppala. Review of recent systems for automatic assessment of programming assignments. In Proceedings of the 10th Koli Calling International Conference on Computing Education Research, pages 86--93, 2010.H. Kitaya and U. Inoue. An online automated scoring system for Java programming assignments. In International Journal of Information and Education Technology, volume 6, pages 275--279, 2014.M.-J. Laakso, T. Salakoski, A. Korhonen, and L. Malmi. Automatic assessment of exercises for algorithms and data structures - a case study with TRAKLA2. In Proceedings of Kolin Kolistelut/Koli Calling - Fourth Finnish/Baltic Sea Conference on Computer Science Education, pages 28--36, 2004.Y. Liang, Q. Liu, J. Xu, and D. Wang. The recent development of automated programming assessment. In Computational Intelligence and Software Engineering, pages 1--5, 2009.K. A. Naudé, J. H. Greyling, and D. Vogts. Marking student programs using graph similarity. In Computers & Education, volume 54, pages 545--561, 2010.A. Pears, S. Seidman, C. Eney, P. Kinnunen, and L. Malmi. Constructing a core literature for computing education research. In SIGCSE Bulletin, volume 37, pages 152--161, 2005.F. Prados, I. Boada, J. Soler, and J. Poch. Automatic generation and correction of technical exercices. In International Conference on Engineering and Computer Education (ICECE 2005), 2005.M. Supic, K. Brkic, T. Hrkac, Z. Mihajlovic, and Z. Kalafatic. Automatic recognition of handwritten corrections for multiple-choice exam answer sheets. In Information and Communication Technology, Electronics and Microelectronics (MIPRO), pages 1136--1141, 2014.S. Tung, T. Lin, and Y. Lin. An exercise management system for teaching programming. In Journal of Software, 2013.T. Wang, X. Su, Y. Wang, and P. Ma. Semantic similarity-based grading of student programs. In Information and Software Technology, volume 49, pages 99--107, 2007

A Research Agenda for Studying Open Source I: A Multi-Level Framework

This paper presents a research agenda for studying information systems using open source software A multi-level research model is developed at five discrete levels of analysis: (1) the artifact; (2) the individual; (3) the team, project, and community; (4) the organization; and (5) society. Each level is discussed in terms of key issues within the level. Examples are based on prior research. In a companion paper, [Niederman, et al 2006], we view the agenda through the lens of referent discipline theories

The Use of Geographical Information Systems in the Urban Communes of Łódź Metropolitan Area

The article presents the results of pilot studies carried out in the cities of Łódź Metropolitan Area. The study concerned the use of geographical information system by the employees of offices of these cities. The interest in the subject of GIS in Polish urban communes results from the assumptions of the EU INSPIRE Directive and the necessity of its implementation in the basic units of territorial division of Poland. The aim of the research was the analysis of the impact of GIS on the sphere of the public participation in the process of local management and the possible use of GIS in the decision making in communes. Research showed what kind of software was used in analyzed communes

The rise of self-driving cars : is the private international law framework for non-contractual obligations posing a bump in the road?

This article focusses on some implications related to the commercialisation of self-driving or autonomous cars. Such vehicles are no longer a mere futuristic idea. They could soon be available on the market. Society in general and the applicable rules in particular will undergo a transformation following the introduction of autonomous vehicles. Despite the many benefits, self-driving cars also pose several challenges. These do not only relate to technical aspects but also to the influence of the autonomisation of traffic on infrastructure and employment in different sectors. More importantly, several legal challenges will need to be addressed as well before society will be able to fully enjoy the benefits of self-driving cars. The question as to who should be held liable for damage caused by self-driving car has already been addressed in academia. Less attention has been devoted to the relationship between autonomous vehicles and the existing private international law rules in the European Union. Although the application of the current jurisdictional and conflict of laws rules does not present problems, the membership of some EU Member States of the 1971 Hague Traffic Accidents Convention and/or the 1973 Hague Products Liability Convention impedes the harmonisation of conflict of laws rules in non-contractual matters as envisaged by the Rome II Regulation. In cases concerning liability arising from traffic accidents and in product liability cases, different Member States courts sometimes apply a different national law. This reduces foreseeability and legal certainty

Control, Process Facilitation, and Requirements Change in Offshore Requirements Analysis: The Provider Perspective

Process, technology, and project factors have been increasingly driving organizations to offshore early software development phases, such as requirements analysis. This emerging trend necessitates greater control and process facilitation between client and vendor sites. The effectiveness of control and facilitation has, however, not been examined within the context of requirements analysis and change. In this study, we examine the role of control and facilitation in managing changing requirements and on success of requirements gathering in the Indian offshore software development environment. Firms found that control by client-site coordinators had a positive impact on requirements analysis success while vender site-coordinators did not have similar influence. Process facilitation by client site-coordinators affected requirements phase success indirectly through control. The study concludes with recommendations for research and practice

An interactive learning environment in geographical information systems

The Unigis Learning Station is a computer‐based learning management tool for the Postgraduate Diploma in Geographical Information Systems by distance learning (correspondence). Unigis is an international network of universities co‐operating in the delivery of such courses. The students on Unigis courses are mature mid‐career professionals who study in addition to undertaking full time jobs. The Learning Station offers these students information about the course, resources for independent study, a structured set of exercises, assessments and feedback opportunities, and an integrated and easy way to interact with other course software. Following a brief introduction to the Unigis curriculum, this paper discusses the design of the Learning Station. The roles the Learning Station adopts are outlined, and the range of multimedia and communications tools used discussed. Evaluation of the Learning Station is presented and the issued raised by this provide useful lessons for other computer‐based learning management tools, and the adaptation of the Learning Station to other teaching and learning situations