Semi-automatic assessment of unrestrained Java code: a Library, a DSL, and a workbench to assess exams and exercises

© ACM 2015. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in http://dx.doi.org/10.1145/2729094.2742615Automated marking of multiple-choice exams is of great interest in university courses with a large number of students. For this reason, it has been systematically implanted in almost all universities. Automatic assessment of source code is however less extended. There are several reasons for that. One reason is that almost all existing systems are based on output comparison with a gold standard. If the output is the expected, the code is correct. Otherwise, it is reported as wrong, even if there is only one typo in the code. Moreover, why it is wrong remains a mystery. In general, assessment tools treat the code as a black box, and they only assess the externally observable behavior. In this work we introduce a new code assessment method that also verifies properties of the code, thus allowing to mark the code even if it is only partially correct. We also report about the use of this system in a real university context, showing that the system automatically assesses around 50% of the work.This work has been partially supported by the EU (FEDER) and the Spanish Ministerio de Economíay Competitividad (Secretaría de Estado de Investigación, Desarrollo e Innovación) under grant TIN2013-44742-C4-1-R and by the Generalitat Valenciana under grant PROMETEOII2015/013. David Insa was partially supported by the Spanish Ministerio de Educación under FPU grant AP2010-4415.Insa Cabrera, D.; Silva, J. (2015). Semi-automatic assessment of unrestrained Java code: a Library, a DSL, and a workbench to assess exams and exercises. ACM. https://doi.org/10.1145/2729094.2742615SK. A Rahman and M. Jan Nordin. A review on the static analysis approach in the automated programming assessment systems. In National Conference on Programming 07, 2007.K. Ala-Mutka. A survey of automated assessment approaches for programming assignments. In Computer Science Education, volume 15, pages 83--102, 2005.C. Beierle, M. Kula, and M. Widera. Automatic analysis of programming assignments. In Proc. der 1. E-Learning Fachtagung Informatik (DeLFI '03), volume P-37, pages 144--153, 2003.J. Biggs and C. Tang. Teaching for Quality Learning at University : What the Student Does (3rd Edition). In Open University Press, 2007.P. Denny, A. Luxton-Reilly, E. Tempero, and J. Hendrickx. CodeWrite: Supporting student-driven practice of java. In Proceedings of the 42nd ACM technical symposium on Computer science education, pages 09--12, 2011.R. Hendriks. Automatic exam correction. 2012.P. Ihantola, T. Ahoniemi, V. Karavirta, and O. Seppala. Review of recent systems for automatic assessment of programming assignments. In Proceedings of the 10th Koli Calling International Conference on Computing Education Research, pages 86--93, 2010.H. Kitaya and U. Inoue. An online automated scoring system for Java programming assignments. In International Journal of Information and Education Technology, volume 6, pages 275--279, 2014.M.-J. Laakso, T. Salakoski, A. Korhonen, and L. Malmi. Automatic assessment of exercises for algorithms and data structures - a case study with TRAKLA2. In Proceedings of Kolin Kolistelut/Koli Calling - Fourth Finnish/Baltic Sea Conference on Computer Science Education, pages 28--36, 2004.Y. Liang, Q. Liu, J. Xu, and D. Wang. The recent development of automated programming assessment. In Computational Intelligence and Software Engineering, pages 1--5, 2009.K. A. Naudé, J. H. Greyling, and D. Vogts. Marking student programs using graph similarity. In Computers & Education, volume 54, pages 545--561, 2010.A. Pears, S. Seidman, C. Eney, P. Kinnunen, and L. Malmi. Constructing a core literature for computing education research. In SIGCSE Bulletin, volume 37, pages 152--161, 2005.F. Prados, I. Boada, J. Soler, and J. Poch. Automatic generation and correction of technical exercices. In International Conference on Engineering and Computer Education (ICECE 2005), 2005.M. Supic, K. Brkic, T. Hrkac, Z. Mihajlovic, and Z. Kalafatic. Automatic recognition of handwritten corrections for multiple-choice exam answer sheets. In Information and Communication Technology, Electronics and Microelectronics (MIPRO), pages 1136--1141, 2014.S. Tung, T. Lin, and Y. Lin. An exercise management system for teaching programming. In Journal of Software, 2013.T. Wang, X. Su, Y. Wang, and P. Ma. Semantic similarity-based grading of student programs. In Information and Software Technology, volume 49, pages 99--107, 2007

Diagrams in Essays: Exploring the Kinds of Diagrams Students Generate and How Well They Work

Part of the Lecture Notes in Computer Science book series (LNAI, volume 12909)12th International Conference, Diagrams 2021, Virtual, September 28–30, 2021, ProceedingsUsing appropriate diagrams is generally considered efficacious in communication. However, although diagrams are extensively used in printed and digital media, people in general rarely construct diagrams to use in common everyday communication. Furthermore, instruction on diagram use for communicative purposes is uncommon in formal education and, when students are required to communicate what they have learned, the usual expectation is they will use words --not diagrams. Requiring diagram inclusion in essays, for example, would be almost unheard of. Consequently, current understanding about student capabilities in this area is very limited. The aim of this study therefore was to contribute to addressing this gap: it comprised a qualitative exploration of 12 undergraduate students’ diagram use in two essays (in which they were asked to include at least one diagram). Analysis focused on identifying the kinds of diagrams produced, and the effectiveness with which those diagrams were used. Useful functions that the diagrams served included clarification, summarization, integration of points, and provision of additional information and/or perspectives in visual form. However, there were also redundancies, as well as unclear, schematically erroneous, and overly complicated representations in some of the diagrams that the students constructed. These findings are discussed in terms of needs, opportunities, and challenges in instructional provision

Introductory programming: a systematic literature review

As computing becomes a mainstream discipline embedded in the school curriculum and acts as an enabler for an increasing range of academic disciplines in higher education, the literature on introductory programming is growing. Although there have been several reviews that focus on specific aspects of introductory programming, there has been no broad overview of the literature exploring recent trends across the breadth of introductory programming. This paper is the report of an ITiCSE working group that conducted a systematic review in order to gain an overview of the introductory programming literature. Partitioning the literature into papers addressing the student, teaching, the curriculum, and assessment, we explore trends, highlight advances in knowledge over the past 15 years, and indicate possible directions for future research

Bibliometric studies on single journals: a review

This paper covers a total of 82 bibliometric studies on single journals (62 studies cover unique titles) published between 1998 and 2008 grouped into the following fields; Arts, Humanities and Social Sciences (12 items); Medical and Health Sciences (19 items); Sciences and Technology (30 items) and Library and Information Sciences (21 items). Under each field the studies are described in accordance to their geographical location in the following order, United Kingdom, United States and Americana, Europe, Asia (India, Africa and Malaysia). For each study, elements described are (a) the journal’s publication characteristics and indexation information; (b) the objectives; (c) the sampling and bibliometric measures used; and (d) the results observed. A list of journal titles studied is appended. The results show that (a)bibliometric studies cover journals in various fields; (b) there are several revisits of some journals which are considered important; (c) Asian and African contributions is high (41.4 of total studies; 43.5 covering unique titles), United States (30.4 of total; 31.0 on unique titles), Europe (18.2 of total and 14.5 on unique titles) and the United Kingdom (10 of total and 11 on unique titles); (d) a high number of bibliometrists are Indians and as such coverage of Indian journals is high (28 of total studies; 30.6 of unique titles); and (e) the quality of the journals and their importance either nationally or internationally are inferred from their indexation status

Transparent authentication methodology in electronic education

In the context of on-line assessment in e-learning, a problem arises when a student taking an exam may wish to cheat by handing over personal credentials to someone else to take their place in an exam, Another problem is that there is no method for signing digital content as it is being produced in a computerized environment. Our proposed solution is to digitally sign the participant’s work by embedding voice samples in the transcript paper at regular intervals. In this investigation, we have demonstrated that a transparent stenographic methodology will provide an innovative and practical solution for achieving continuous authentication in an online educational environment by successful insertion and extraction of audio digital signatures

Courseware in academic library user education: A literature review from the GAELS Joint Electronic Library Project

The use of courseware for information skills teaching in academic libraries has been growing for a number of years. In order to create effective courseware packages to support joint electronic library activity at Glasgow and Strathclyde Universities, the GAELS project conducted a literature review of the subject. This review discovered a range of factors common to successful library courseware implementations, such as the need for practitioners to feel a sense of ownership of the medium, a need for courseware customization to local information environments, and an emphasis on training packages for large bodies of undergraduates. However, we also noted underdeveloped aspects worthy of further attention, such as treatment of pedagogic issues in library computer‐aided learning (CAL) implementations and use of hypertextual learning materials for more advanced information skills training. We describe how these findings shaped the packages produced by the project and suggest ways forward for similar types of implementation

The Role of Technology in Music Education: a Survey of Computer Usage in Teaching Music in Colleges of Education in The Volta Region, Ghana

The study sought to find out the role of computer technology in music education in Colleges of Education in the Volta Region of Ghana. It aimed at surveying the use of computer technology for teaching music and exploring the instructional prospects for computer technology usage in music in Colleges of Education. The study employed Rogers’ Diffusion Innovation theory and descriptive survey research method. Data was collected from the respondents using questionnaire, interview, and observation. The study revealed that even though about 90% of the music tutors have good academic qualification and over five years teaching experience, lack of competence in handling computer technology in teaching music among some music tutors and incoherent ICT initiatives hindered proper application of computer technology in the field of music education. It is however envisaged that increasing access and coherent computer technology initiatives will be paramount for the teaching of music in the Colleges of Education

Emergent requirements for supporting introductory programming

The problems associated with learning and teaching first year University Computer Science (CS1) programming classes are summarized showing that various support tools and techniques have been developed and evaluated. From this review of applicable support the paper derives ten requirements that a support tool should have in order to improve CS1 student success rate with respect to learning and understanding

Multi-layer Architecture For Storing Visual Data Based on WCF and Microsoft SQL Server Database

In this paper we present a novel architecture for storing visual data. Effective storing, browsing and searching collections of images is one of the most important challenges of computer science. The design of architecture for storing such data requires a set of tools and frameworks such as SQL database management systems and service-oriented frameworks. The proposed solution is based on a multi-layer architecture, which allows to replace any component without recompilation of other components. The approach contains five components, i.e. Model, Base Engine, Concrete Engine, CBIR service and Presentation. They were based on two well-known design patterns: Dependency Injection and Inverse of Control. For experimental purposes we implemented the SURF local interest point detector as a feature extractor and $K$-means clustering as indexer. The presented architecture is intended for content-based retrieval systems simulation purposes as well as for real-world CBIR tasks.Comment: Accepted for the 14th International Conference on Artificial Intelligence and Soft Computing, ICAISC, June 14-18, 2015, Zakopane, Polan