Semi-automatic assessment of unrestrained Java code: a Library, a DSL, and a workbench to assess exams and exercises

© ACM 2015. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in http://dx.doi.org/10.1145/2729094.2742615Automated marking of multiple-choice exams is of great interest in university courses with a large number of students. For this reason, it has been systematically implanted in almost all universities. Automatic assessment of source code is however less extended. There are several reasons for that. One reason is that almost all existing systems are based on output comparison with a gold standard. If the output is the expected, the code is correct. Otherwise, it is reported as wrong, even if there is only one typo in the code. Moreover, why it is wrong remains a mystery. In general, assessment tools treat the code as a black box, and they only assess the externally observable behavior. In this work we introduce a new code assessment method that also verifies properties of the code, thus allowing to mark the code even if it is only partially correct. We also report about the use of this system in a real university context, showing that the system automatically assesses around 50% of the work.This work has been partially supported by the EU (FEDER) and the Spanish Ministerio de Economíay Competitividad (Secretaría de Estado de Investigación, Desarrollo e Innovación) under grant TIN2013-44742-C4-1-R and by the Generalitat Valenciana under grant PROMETEOII2015/013. David Insa was partially supported by the Spanish Ministerio de Educación under FPU grant AP2010-4415.Insa Cabrera, D.; Silva, J. (2015). Semi-automatic assessment of unrestrained Java code: a Library, a DSL, and a workbench to assess exams and exercises. ACM. https://doi.org/10.1145/2729094.2742615SK. A Rahman and M. Jan Nordin. A review on the static analysis approach in the automated programming assessment systems. In National Conference on Programming 07, 2007.K. Ala-Mutka. A survey of automated assessment approaches for programming assignments. In Computer Science Education, volume 15, pages 83--102, 2005.C. Beierle, M. Kula, and M. Widera. Automatic analysis of programming assignments. In Proc. der 1. E-Learning Fachtagung Informatik (DeLFI '03), volume P-37, pages 144--153, 2003.J. Biggs and C. Tang. Teaching for Quality Learning at University : What the Student Does (3rd Edition). In Open University Press, 2007.P. Denny, A. Luxton-Reilly, E. Tempero, and J. Hendrickx. CodeWrite: Supporting student-driven practice of java. In Proceedings of the 42nd ACM technical symposium on Computer science education, pages 09--12, 2011.R. Hendriks. Automatic exam correction. 2012.P. Ihantola, T. Ahoniemi, V. Karavirta, and O. Seppala. Review of recent systems for automatic assessment of programming assignments. In Proceedings of the 10th Koli Calling International Conference on Computing Education Research, pages 86--93, 2010.H. Kitaya and U. Inoue. An online automated scoring system for Java programming assignments. In International Journal of Information and Education Technology, volume 6, pages 275--279, 2014.M.-J. Laakso, T. Salakoski, A. Korhonen, and L. Malmi. Automatic assessment of exercises for algorithms and data structures - a case study with TRAKLA2. In Proceedings of Kolin Kolistelut/Koli Calling - Fourth Finnish/Baltic Sea Conference on Computer Science Education, pages 28--36, 2004.Y. Liang, Q. Liu, J. Xu, and D. Wang. The recent development of automated programming assessment. In Computational Intelligence and Software Engineering, pages 1--5, 2009.K. A. Naudé, J. H. Greyling, and D. Vogts. Marking student programs using graph similarity. In Computers & Education, volume 54, pages 545--561, 2010.A. Pears, S. Seidman, C. Eney, P. Kinnunen, and L. Malmi. Constructing a core literature for computing education research. In SIGCSE Bulletin, volume 37, pages 152--161, 2005.F. Prados, I. Boada, J. Soler, and J. Poch. Automatic generation and correction of technical exercices. In International Conference on Engineering and Computer Education (ICECE 2005), 2005.M. Supic, K. Brkic, T. Hrkac, Z. Mihajlovic, and Z. Kalafatic. Automatic recognition of handwritten corrections for multiple-choice exam answer sheets. In Information and Communication Technology, Electronics and Microelectronics (MIPRO), pages 1136--1141, 2014.S. Tung, T. Lin, and Y. Lin. An exercise management system for teaching programming. In Journal of Software, 2013.T. Wang, X. Su, Y. Wang, and P. Ma. Semantic similarity-based grading of student programs. In Information and Software Technology, volume 49, pages 99--107, 2007

Preface Volume 65, Issue 1

AbstractThis volume contains the Proceedings of the Fifth Workshop on Coalgebraic Methods in Computer Science (CMCS'2002). The Workshop was held in Grenoble, France on April 6--7 2002, as satellite event to ETAPS'2002.Over the last few years it has become clear that a great variety of state-based dynamical systems, like transition systems, automata, process calculi and class-based systems can be captured uniformly as coalgebras. The aim of the CMCS workshops is to bring together researchers with a common interest in the theory and application of coalgebras. The five CMCS volumes demonstrate that coalgebra is developing into a field of its own, presenting a deep mathematical foundation and a growing field of applications and interactions with various other fields, such as modal logic, category theory, dynamical systems, control systems, object-oriented and concurrent programming, formal systems specifications, algebra, analysis, combinatorics, and set theory.The papers in this volume were reviewed by the program committee: Jiri Adamek(Department of Computer Science, Technical University of Braunschweig)Alexandru Baltag(Department of Computer Science, Oxford University)H. Peter Gumm(Department of Mathematics and Computer Science, University of Marburg)Jesse Hughes(Department of Computer Science, University of Nijmegen)Bart Jacobs(Department of Computer Science, University of Nijmegen)Alexander Kurz(Department of Software Technology, CWI)Marina Lenisa(Department of Mathematics and Computer Science, University of Udine)Ugo Montanari(Department of Computer Science, University of Pisa)Larry Moss(Department of Mathematics, Indiana University)Ataru T. Nakagawa(SRA Key Technology Laboratory, Tokyo)John Power(Department of Computer Science, The University of Edinburgh)Horst Reichel(Institute of Theoretical Computer Science, Dresden University of Technology)Jan Rutten(Department of Software Technology, CWI)Several outside reviewers also assisted. CMCS received 20 submissions and accepted 15 of them. In addition, there were two invited speakers: Jose Meseguer and Luigi Santocanale. Their papers appear in this volume along with the 15 submitted contributions. We are grateful to everyone who sent us papers, and we regret that the length of the conference did not allow more papers to be presented.We thank the organizers of ETAPS'2002 for their help and encouragement. Special thanks to Rachid Echahed for his constant help with the organization of the workshop, and also to Mike Mislove for his work as a Managing Editor of the ENTCS series. Their efforts have been crucial for the success of CMCS'2002.September 1, 2002 Lawrence S. Mos

Getting Students Excited About Science With High Altitude Ballooning

Many of us dream of exploring space, but there are not many ways to do so. Although it is difficult to get into deep space, near space is within our grasp. High altitude balloons are released into the stratosphere, generally reaching between 60,000 to 120,000 feet before they burst and their payload is returned to earth by parachute. Modern balloon systems generally contain electronic equipment such as radio transmitters, cameras, and GPS receivers, as well as a variety of scientific instruments. Not only is high altitude ballooning a great way to introduce the electronics and programming skills needed to collect and analyze data from the spacecraft, it provides a fun way to explore scientific concepts from pressure, temperature and volume to cosmic radiation. We have begun offering summer camps to high school students in order to capitalize on the excitement of ballooning to get them interested in physics. In July of 2016, we will offer the five-day camp for the third time, with student numbers increasing from 2 to at least 6 with registrations still coming in. After a brief introduction to atmospheric science, students explore the relationship between rate of ascent, weight, and volume of helium, before using space flight prediction tools to determine potential flight paths. They also learn about a variety of cameras, sensors and data loggers and design and build data collection systems that their group will incorporate into their box for a flight near the end of the camp. Finally, each group prepares and presents a short report on the results of the flight. The numbers of students involved for any statistically significant results to be determined from surveys, but anecdotal data suggests that students’ interest in STEM fields and specifically physics and computer science is enhanced by the HAB camp experience

ElAM: A computer program for the analysis and representation of anisotropic elastic properties

Copyright © 2010 Elsevier. NOTICE: this is the author’s version of a work that was accepted for publication in Computer Physics Communications. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Computer Physics Communications, Volume 181, Issue 12 (2010), DOI: 10.1016/j.cpc.2010.08.033The continuum theory of elasticity has been used for more than a century and has applications in many fields of science and engineering. It is very robust, well understood and mathematically elegant. In the isotropic case elastic properties are easily represented, but for non-isotropic materials, even in the simple cubic symmetry, it can be difficult to visualise how properties such as Young's modulus or Poisson's ratio vary with stress/strain orientation. The ElAM (Elastic Anisotropy Measures) code carries out the required tensorial operations (inversion, rotation, diagonalisation) and creates 3D models of an elastic property's anisotropy. It can also produce 2D cuts in any given plane, compute averages following diverse schemes and query a database of elastic constants to support meta-analyses. Program summary Program title: ElAM1.0 Catalogue identifier: AEHB_v1_0 Program summary URL:http://cpc.cs.qub.ac.uk/summaries/AEHB_v1_0.html Program obtainable from: CPC Program Library, Queen's University, Belfast, N. Ireland Licensing provisions: Standard CPC licence, http://cpc.cs.qub.ac.uk/licence/licence.html No. of lines in distributed program, including test data, etc.: 43 848 No. of bytes in distributed program, including test data, etc.: 2 498 882 Distribution format: tar.gz Programming language: Fortran90 Computer: Any Operating system: Linux, Windows (XP, Vista) RAM: Depends chiefly on the size of the arrays representing elastic properties in 3D Classification: 7.7 Nature of problem: Representation of elastic moduli and ratios, and of wave velocities, in 3D; automatic discovery of unusual elastic properties. Solution method: Stiffness matrix (6×6)(6×6) inversion and conversion to compliance tensor (3×3×3×3)(3×3×3×3), tensor rotation, dynamic matrix diagonalisation, simple optimisation, postscript and VRML output preparation. Running time: Dependent on angular accuracy and size of elastic constant database (from a few seconds to a few hours). The tests provided take from a few seconds for test0 to approximately 1 hour for test4

Introductory programming: a systematic literature review

As computing becomes a mainstream discipline embedded in the school curriculum and acts as an enabler for an increasing range of academic disciplines in higher education, the literature on introductory programming is growing. Although there have been several reviews that focus on specific aspects of introductory programming, there has been no broad overview of the literature exploring recent trends across the breadth of introductory programming. This paper is the report of an ITiCSE working group that conducted a systematic review in order to gain an overview of the introductory programming literature. Partitioning the literature into papers addressing the student, teaching, the curriculum, and assessment, we explore trends, highlight advances in knowledge over the past 15 years, and indicate possible directions for future research

Abstract State Machines 1988-1998: Commented ASM Bibliography

An annotated bibliography of papers which deal with or use Abstract State Machines (ASMs), as of January 1998.Comment: Also maintained as a BibTeX file at http://www.eecs.umich.edu/gasm

Emergent requirements for supporting introductory programming

The problems associated with learning and teaching first year University Computer Science (CS1) programming classes are summarized showing that various support tools and techniques have been developed and evaluated. From this review of applicable support the paper derives ten requirements that a support tool should have in order to improve CS1 student success rate with respect to learning and understanding