Semi-automatic assessment of unrestrained Java code: a Library, a DSL, and a workbench to assess exams and exercises

© ACM 2015. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in http://dx.doi.org/10.1145/2729094.2742615Automated marking of multiple-choice exams is of great interest in university courses with a large number of students. For this reason, it has been systematically implanted in almost all universities. Automatic assessment of source code is however less extended. There are several reasons for that. One reason is that almost all existing systems are based on output comparison with a gold standard. If the output is the expected, the code is correct. Otherwise, it is reported as wrong, even if there is only one typo in the code. Moreover, why it is wrong remains a mystery. In general, assessment tools treat the code as a black box, and they only assess the externally observable behavior. In this work we introduce a new code assessment method that also verifies properties of the code, thus allowing to mark the code even if it is only partially correct. We also report about the use of this system in a real university context, showing that the system automatically assesses around 50% of the work.This work has been partially supported by the EU (FEDER) and the Spanish Ministerio de Economíay Competitividad (Secretaría de Estado de Investigación, Desarrollo e Innovación) under grant TIN2013-44742-C4-1-R and by the Generalitat Valenciana under grant PROMETEOII2015/013. David Insa was partially supported by the Spanish Ministerio de Educación under FPU grant AP2010-4415.Insa Cabrera, D.; Silva, J. (2015). Semi-automatic assessment of unrestrained Java code: a Library, a DSL, and a workbench to assess exams and exercises. ACM. https://doi.org/10.1145/2729094.2742615SK. A Rahman and M. Jan Nordin. A review on the static analysis approach in the automated programming assessment systems. In National Conference on Programming 07, 2007.K. Ala-Mutka. A survey of automated assessment approaches for programming assignments. In Computer Science Education, volume 15, pages 83--102, 2005.C. Beierle, M. Kula, and M. Widera. Automatic analysis of programming assignments. In Proc. der 1. E-Learning Fachtagung Informatik (DeLFI '03), volume P-37, pages 144--153, 2003.J. Biggs and C. Tang. Teaching for Quality Learning at University : What the Student Does (3rd Edition). In Open University Press, 2007.P. Denny, A. Luxton-Reilly, E. Tempero, and J. Hendrickx. CodeWrite: Supporting student-driven practice of java. In Proceedings of the 42nd ACM technical symposium on Computer science education, pages 09--12, 2011.R. Hendriks. Automatic exam correction. 2012.P. Ihantola, T. Ahoniemi, V. Karavirta, and O. Seppala. Review of recent systems for automatic assessment of programming assignments. In Proceedings of the 10th Koli Calling International Conference on Computing Education Research, pages 86--93, 2010.H. Kitaya and U. Inoue. An online automated scoring system for Java programming assignments. In International Journal of Information and Education Technology, volume 6, pages 275--279, 2014.M.-J. Laakso, T. Salakoski, A. Korhonen, and L. Malmi. Automatic assessment of exercises for algorithms and data structures - a case study with TRAKLA2. In Proceedings of Kolin Kolistelut/Koli Calling - Fourth Finnish/Baltic Sea Conference on Computer Science Education, pages 28--36, 2004.Y. Liang, Q. Liu, J. Xu, and D. Wang. The recent development of automated programming assessment. In Computational Intelligence and Software Engineering, pages 1--5, 2009.K. A. Naudé, J. H. Greyling, and D. Vogts. Marking student programs using graph similarity. In Computers & Education, volume 54, pages 545--561, 2010.A. Pears, S. Seidman, C. Eney, P. Kinnunen, and L. Malmi. Constructing a core literature for computing education research. In SIGCSE Bulletin, volume 37, pages 152--161, 2005.F. Prados, I. Boada, J. Soler, and J. Poch. Automatic generation and correction of technical exercices. In International Conference on Engineering and Computer Education (ICECE 2005), 2005.M. Supic, K. Brkic, T. Hrkac, Z. Mihajlovic, and Z. Kalafatic. Automatic recognition of handwritten corrections for multiple-choice exam answer sheets. In Information and Communication Technology, Electronics and Microelectronics (MIPRO), pages 1136--1141, 2014.S. Tung, T. Lin, and Y. Lin. An exercise management system for teaching programming. In Journal of Software, 2013.T. Wang, X. Su, Y. Wang, and P. Ma. Semantic similarity-based grading of student programs. In Information and Software Technology, volume 49, pages 99--107, 2007

Partial Correctness of a Power Algorithm

This work continues a formal verification of algorithms written in terms of simple-named complex-valued nominative data [6],[8],[15],[11],[12],[13]. In this paper we present a formalization in the Mizar system [3],[1] of the partial correctness of the algorithm: i := val.1 j := val.2 b := val.3 n := val.4 s := val.5 while (i n) i := i + j s := s * b return s computing the natural n power of given complex number b, where variables i, b, n, s are located as values of a V-valued Function, loc, as: loc/.1 = i, loc/.3 = b, loc/.4 = n and loc/.5 = s, and the constant 1 is located in the location loc/.2 = j (set V represents simple names of considered nominative data [17]).The validity of the algorithm is presented in terms of semantic Floyd-Hoare triples over such data [9]. Proofs of the correctness are based on an inference system for an extended Floyd-Hoare logic [2],[4] with partial pre- and post-conditions [14],[16],[7],[5].Institute of Informatics, University of Białystok, PolandGrzegorz Bancerek, Czesław Byliński, Adam Grabowski, Artur Korniłowicz, Roman Matuszewski, Adam Naumowicz, and Karol Pąk. The role of the Mizar Mathematical Library for interactive proof development in Mizar. Journal of Automated Reasoning, 61(1):9–32, 2018. doi:10.1007/s10817-017-9440-6.R.W. Floyd. Assigning meanings to programs. Mathematical aspects of computer science, 19(19–32), 1967.Adam Grabowski, Artur Korniłowicz, and Adam Naumowicz. Four decades of Mizar. Journal of Automated Reasoning, 55(3):191–198, 2015. doi:10.1007/s10817-015-9345-1.C.A.R. Hoare. An axiomatic basis for computer programming. Commun. ACM, 12(10): 576–580, 1969.Ievgen Ivanov and Mykola Nikitchenko. On the sequence rule for the Floyd-Hoare logic with partial pre- and post-conditions. In Proceedings of the 14th International Conference on ICT in Education, Research and Industrial Applications. Integration, Harmonization and Knowledge Transfer. Volume II: Workshops, Kyiv, Ukraine, May 14–17, 2018, volume 2104 of CEUR Workshop Proceedings, pages 716–724, 2018.Ievgen Ivanov, Mykola Nikitchenko, Andrii Kryvolap, and Artur Korniłowicz. Simple-named complex-valued nominative data – definition and basic operations. Formalized Mathematics, 25(3):205–216, 2017. doi:10.1515/forma-2017-0020.Ievgen Ivanov, Artur Korniłowicz, and Mykola Nikitchenko. Implementation of the composition-nominative approach to program formalization in Mizar. The Computer Science Journal of Moldova, 26(1):59–76, 2018.Ievgen Ivanov, Artur Korniłowicz, and Mykola Nikitchenko. On an algorithmic algebra over simple-named complex-valued nominative data. Formalized Mathematics, 26(2):149–158, 2018. doi:10.2478/forma-2018-0012.Ievgen Ivanov, Artur Korniłowicz, and Mykola Nikitchenko. An inference system of an extension of Floyd-Hoare logic for partial predicates. Formalized Mathematics, 26(2): 159–164, 2018. doi:10.2478/forma-2018-0013.Ievgen Ivanov, Artur Korniłowicz, and Mykola Nikitchenko. Partial correctness of GCD algorithm. Formalized Mathematics, 26(2):165–173, 2018. doi:10.2478/forma-2018-0014.Ievgen Ivanov, Artur Korniłowicz, and Mykola Nikitchenko. On algebras of algorithms and specifications over uninterpreted data. Formalized Mathematics, 26(2):141–147, 2018. doi:10.2478/forma-2018-0011.Artur Kornilowicz, Andrii Kryvolap, Mykola Nikitchenko, and Ievgen Ivanov. Formalization of the algebra of nominative data in Mizar. In Maria Ganzha, Leszek A. Maciaszek, and Marcin Paprzycki, editors, Proceedings of the 2017 Federated Conference on Computer Science and Information Systems, FedCSIS 2017, Prague, Czech Republic, September 3–6, 2017., pages 237–244, 2017. ISBN 978-83-946253-7-5. doi:10.15439/2017F301.Artur Kornilowicz, Andrii Kryvolap, Mykola Nikitchenko, and Ievgen Ivanov. Formalization of the nominative algorithmic algebra in Mizar. In Leszek Borzemski, Jerzy Świątek, and Zofia Wilimowska, editors, Information Systems Architecture and Technology: Proceedings of 38th International Conference on Information Systems Architecture and Technology – ISAT 2017 – Part II, Szklarska Poręba, Poland, September 17–19, 2017, volume 656 of Advances in Intelligent Systems and Computing, pages 176–186. Springer, 2017. ISBN 978-3-319-67228-1. doi:10.1007/978-3-319-67229-8_16.Artur Korniłowicz, Andrii Kryvolap, Mykola Nikitchenko, and Ievgen Ivanov. An approach to formalization of an extension of Floyd-Hoare logic. In Vadim Ermolayev, Nick Bassiliades, Hans-Georg Fill, Vitaliy Yakovyna, Heinrich C. Mayr, Vyacheslav Kharchenko, Vladimir Peschanenko, Mariya Shyshkina, Mykola Nikitchenko, and Aleksander Spivakovsky, editors, Proceedings of the 13th International Conference on ICT in Education, Research and Industrial Applications. Integration, Harmonization and Knowledge Transfer, Kyiv, Ukraine, May 15–18, 2017, volume 1844 of CEUR Workshop Proceedings, pages 504–523. CEUR-WS.org, 2017.Artur Korniłowicz, Ievgen Ivanov, and Mykola Nikitchenko. Kleene algebra of partial predicates. Formalized Mathematics, 26(1):11–20, 2018. doi:10.2478/forma-2018-0002.Andrii Kryvolap, Mykola Nikitchenko, and Wolfgang Schreiner. Extending Floyd-Hoare logic for partial pre- and postconditions. In Vadim Ermolayev, Heinrich C. Mayr, Mykola Nikitchenko, Aleksander Spivakovsky, and Grygoriy Zholtkevych, editors, Information and Communication Technologies in Education, Research, and Industrial Applications: 9th International Conference, ICTERI 2013, Kherson, Ukraine, June 19–22, 2013, Revised Selected Papers, pages 355–378. Springer International Publishing, 2013. ISBN 978-3-319-03998-5. doi:10.1007/978-3-319-03998-5_18.Volodymyr G. Skobelev, Mykola Nikitchenko, and Ievgen Ivanov. On algebraic properties of nominative data and functions. In Vadim Ermolayev, Heinrich C. Mayr, Mykola Nikitchenko, Aleksander Spivakovsky, and Grygoriy Zholtkevych, editors, Information and Communication Technologies in Education, Research, and Industrial Applications – 10th International Conference, ICTERI 2014, Kherson, Ukraine, June 9–12, 2014, Revised Selected Papers, volume 469 of Communications in Computer and Information Science, pages 117–138. Springer, 2014. ISBN 978-3-319-13205-1. doi:10.1007/978-3-319-13206-8_6.27218919

Chemistry education students’ perception toward their learning outcomes during online learning

This study aims to determine the self-perception of chemistry education students related to online learning in the UIN Ar-Raniry. This descriptive research involved chemistry education students of the 2019 batch. Their perception was obtained using an online questionnaire with 15 questions. It can be concluded that implementing the online learning system does not entirely affect students' learning outcomes. REFERENCESAl-Idrus, S. W., Muti’ah, M., & Rahmawati, R. (2021). Analisis Proses Pembelajaran Daring Selama Pandemi Covid-19 pada Mahasiswa Program Studi Pendidikan Kimia FKIP UNRAM. PENSA, 3(1 SE-Articles). https://ejournal.stitpn.ac.id/index.php/pensa/article/view/1246Arisandi, Y., Dasna, I. W., Sumari, S., Habiddin, H., Ibnu, S., & Subandi. (2021). Promoting vocational students' perception and achievement towards Chemistry: SPARE Learning Strategy for students majoring in Automotive Engineering. Turkish Journal of Computer and Mathematics Education (TURCOMAT), 12(13), 6357–6365.Belawati, T. (2019). Pembelajaran Online. Universitas Terbuka.Habiddin, H., Yahmin, Y., Retnosari, R., Muarifin, M., Aziz, A. N., Husniah, I., & Anwar, L. (2020). Chemistry students' attitude towards chemistry. AIP Conference Proceedings, 2215(1), 20005. https://doi.org/10.1063/5.0000496Mandailina, V., Pramita, D., & Haifaturrahmah, H. (2021). Pembelajaran Daring Dalam Meningkatkan Motivasi dan Hasil Belajar Peserta Didik Selama Pandemi Covid-19: Sebuah Meta-Analisis. Indonesian Journal of Educational Science (IJES), 3(2), 120–129.Pratiwi, N. N., & Puspasari, D. (2021). PENGARUH PENGGUNAAN PEMBELAJARAN DARING TERHADAP HASIL BELAJAR SISWA. JAMP : Jurnal Administrasi Dan Manajemen Pendidikan; Vol 4, No 4 (2021): Volume 4 No 4 Desember 2021. https://doi.org/10.17977/um027v4i42021p320Purwanto, A., Pramono, R., Asbari, M., Hyun, C. C., Wijayanti, L. M., Putri, R. S., & Santoso,  priyono B. (2020). Studi Eksploratif Dampak Pandemi COVID-19 Terhadap Proses Pembelajaran Online di Sekolah Dasar. EduPsyCouns: Journal of Education, Psychology and Counseling, 2(1), 1–12.Putri, M., & Kurniawati, Y. (2021). Students' learning interest using computer and android in acid base teaching. J-PEK (Jurnal Pembelajaran Kimia), 6(2), 63–71. http://dx.doi.org/10.17977/um026v6i22021p063Putria, H., Maula, L. H., & Uswatun, D. A. (2020). Analisis Proses Pembelajaran dalam Jaringan (DARING) Masa Pandemi Covid- 19 Pada Guru Sekolah Dasar. Jurnal Basicedu, 4(4), 861–872.Riyana, C. (2019). Produksi Bahan Pembelajaran Berbasis Online. Universitas Terbuka.Wardani, S., Haryani, S., Harmiasri, R., & Sari, N. N. (2022). Implementation of Online Learning to Prepare the Youth Generation in the Disruptive Era. J-PEK (Jurnal Pembelajaran Kimia), 7(1), 9–21. https://doi.org/http://dx.doi.org/10.17977/um026v7i12022p00

Introductory programming: a systematic literature review

As computing becomes a mainstream discipline embedded in the school curriculum and acts as an enabler for an increasing range of academic disciplines in higher education, the literature on introductory programming is growing. Although there have been several reviews that focus on specific aspects of introductory programming, there has been no broad overview of the literature exploring recent trends across the breadth of introductory programming. This paper is the report of an ITiCSE working group that conducted a systematic review in order to gain an overview of the introductory programming literature. Partitioning the literature into papers addressing the student, teaching, the curriculum, and assessment, we explore trends, highlight advances in knowledge over the past 15 years, and indicate possible directions for future research

Flexible learning in computer science

This paper outlines the concept of Flexible Pedagogy and how it can assist in addressing some of the issues facing STEM disciplines in general, and Computer Science in particular. The paper considers what flexible pedagogy is and how technologies developed by Computer Science can enable flexibility. It then describes some of the issues facing STEM education, with a particular focus on Computer Science education in Higher Education. Finally, it considers how flexible approaches to teaching and learning are particularly pertinent to the issues faced in Computer Science and future opportunities

The social impact and cultural issues affecting the e-learning performance in Libyan Higher Education institutes

This paper analyses the social impact and cultural issues which affect the e-learning performance in Libyan Higher Education institutes (HEIs). It is described the development and implementation of e-learning systems in various HEIs with the emphasis on the digital gap in Libya and barriers to successful e-learning implementation in these institutions. Also the social impact of using e-learning packages and Internet by young people in Libya is studied and a SWOT analysis of ICT and e-learning in Tripoli University is performed in order to improve the effectiveness of the use of e-learning systems in Libyan HEIs

An Evaluation of eScience Lab Kits for Online Learning

Higher education online science courses generally lack the hands-on components essential in understanding theories, methods, and techniques in chemistry and biology. Companies like eScience Labs construct kits to facilitate online learning, which provide students with hands-on activities relevant to their science courses. In order to evaluate ease, efficacy, and comprehension of the forensic science kits by eScience Labs was completed while writing observations of the activities during and after completion; the lab manual learning objectives were compared to results of activities and two stopwatches took elapsed time of each activity to compare with the stated times in the kit manual. This method determined that the eScience manual does not provide enough information for a college freshman to fully understand the topic; however, combining these labs with professor provided online lectures would allow full comprehension of the forensic science applications or techniques. Recommendations to obtain maximum learning outcomes include requiring the completion of prerequisites like algebra and general chemistry. With these aspects combined, the eScience lab kit is a great addition to an introductory forensic science course as it provides safe and interactive hands-on activities

Aligning operational and corporate goals: a case study in cultivating a whole-of-business approach using a supply chain simulation game

This paper outlines the development and use of an interactive computer-based supply chain game to facilitate the alignment of disconnected operational and corporate goals. A multi-enterprise internal cattle supply chain was simulated targeting the operational property managers and the overall impacts of their decision making on corporate goals A three stage multidisciplinary approach was used. A case study based financial analysis was undertaken across the internal cattle supply chain, a participative action research component (developing the game to simulate the flow of product and associated decisions and financial transactions through the internal supply chain of the company for different operational scenarios using measurable and familiar operational and financial criteria as tracking tools), and a qualitative analysis of organisational learning through player debriefing following playing the game. Evaluation of the managers' learning around the need for a change in general practice to address goal incongruence was positive evidenced by changes in practice and the game regarded by the users as a useful form of organisational training. The game provided property managers with practical insights into the strategic implications of their enterprise level decisions on the internal supply chain and on overall corporate performance. The game is unique and is a tool that can be used to help address an endemic problem across multi-enterprise industries in the agrifood sector in Australia