Software Verification and Graph Similarity for Automated Evaluation of Students' Assignments

In this paper we promote introducing software verification and control flow graph similarity measurement in automated evaluation of students' programs. We present a new grading framework that merges results obtained by combination of these two approaches with results obtained by automated testing, leading to improved quality and precision of automated grading. These two approaches are also useful in providing a comprehensible feedback that can help students to improve the quality of their programs We also present our corresponding tools that are publicly available and open source. The tools are based on LLVM low-level intermediate code representation, so they could be applied to a number of programming languages. Experimental evaluation of the proposed grading framework is performed on a corpus of university students' programs written in programming language C. Results of the experiments show that automatically generated grades are highly correlated with manually determined grades suggesting that the presented tools can find real-world applications in studying and grading

Preface : CAA 2010: Computer assisted assessment: supporting student learning

The Preface for the first special issue for the International Journal of e-Assessment highlights the role of the assessment for learning agenda. It also points to papers within the special issue which relate to: (a) Overcoming the barriers to supporting student learning with computer-assisted assessment; (b) Marking and computer-assisted assessment; (c) Harnessing the technology to improve efficiency and pedagogy. The papers in this issue stress the general recognition that times are changing and that assessment needs to become embedded in the teaching/learning cycle and not purely as a checking device for the awarding institution. These papers encourage us to rise to the challenge of developing more sophisticated computer-assisted assessment systems that support student learning and there is still a lot to do

Trials & Tribulations Encountered During the Development & Teaching of a Dual-Delivery Format Research Methods Course

This paper focuses on developmental and pedagogical/sociological issues related to a doctoral level research methodology course. This course is delivered in two formats, resident (face-to-face) and distance (web-based on Blackboard). Pedagogical, sociological, course development, course delivery, issues and challenges for both formats are discussed. An annotated bibliography is also included

Feedback by automatic assessment systems used in mathematics homework in the engineering field

This research presents the results of the teaching innovation Dynamic Online Assessment System in Mathematics, which is implemented in higher education to promote self‐study by students outside the classroom. The WIRIS calculator was integrated into the Moodle platform to create questions with random elements, for example, students had access to different variants of the same question. The effect of the type of feedback (immediate or deferred) on the work of the students on the platform, measured by means of participation, time spent, and grades obtained, was evaluated. We used a quasi‐experimental methodology for a population of 5,507 students, distributed in 229 courses on four campuses that learn Mathematics I in engineering programs. Immediate feedback exhibits better work of students on the platform, but this work is not necessarily more efficient in comparison with the work performed by students using assessments online assessment with deferred feedback

Effectiveness of Automatic Formative Assessment for learning Mathematics in Higher Education

[EN] The health emergency due to COVID-19 has highlighted the need of new and flexible digital methodologies for learning and teaching Mathematics, which can support the individual student’s needs and help shape education. In this paper, we propose to use Automatic Formative Assessment (AFA) activities designed according to an innovative model, framed on the theories of formative assessment and feedback. The model includes: availability of the activities and multiple attempts; algorithmic questions; open mathematical answers; contextualized tasks; immediate and interactive feedback. We analyzed an experience using AFA in a blended module in Mathematics for 96 first-year students in Biotechnology. We collected data from the platform on grades and attempts of AFA activities and cross-checked them with the final exam grades. The results show that the feedback gained from AFA helped students improve their performance and supported them in the exam preparation. In light of these results, we can conclude that similar activities can be an effective solution to support students’ self-study during and after the pandemic.Barana, A.; Marchisio, M.; Sacchet, M. (2021). Effectiveness of Automatic Formative Assessment for learning Mathematics in Higher Education. En 7th International Conference on Higher Education Advances (HEAd'21). Editorial Universitat Politècnica de València. 1-8. https://doi.org/10.4995/HEAd21.2021.13030OCS1

Effective computer-aided assessment of mathematics; principles, practice and results

This article outlines some key issues for writing effective computer-aided assessment (CAA) questions in subjects with substantial mathematical or statistical content, especially the importance of control of random parameters and the encoding of wrong methods of solution (mal-rules) commonly used by students. The pros and cons of using CAA and different question types are discussed. Issues surrounding the selection and encoding of mal-rules are highlighted, especially for multi-choice and responsive numerical input questions. These generate mal-rule-specific feedback, the mal-rule used being deduced 15 from the student’s selection or input. Student answer file data from the use of over 800 questions and their embedding within an overall assessment regime is analysed and presented to show that this has had a very beneficial effect on the examination performance of a large cohort of first-year economics students in their mathematics module over the last 6 years. Question analysis of over 270,000 question attempts, identifying the most 20 difficult/discriminating questions, shows that the questions are robust, valid and span an appropriate range of difficulties. The idea of underlying mal-rules is examined to see how far this explains this range

Strategies of formative assessment enacted through automatic assessment in blended modality

International audienceThis paper intends to contribute to the research on formative assessment in Mathematics providing a model of automatic assessment aimed at enhancing learning and self-regulation. The model was developed at the Department of Mathematics of the University of Turin (Italy). The main features of the model are: availability, algorithmic questions, open answers, immediate feedback, interactive feedback, and real-life contextualization. The effectiveness of the model to enact formative strategies is discussed though the results of a didactic experimentation involving 299 students of 8th grade, where automatically assessed assignments have been used both during Mathematics classes and as online homework

Long term effects on technology enhanced learning : The use of weekly digital lessons in mathematics

In this study we investigate the effects of long-term technology enhanced learning (TEL) in mathematics learning performance and fluency, and how technology enhanced learning can be integrated into regular curriculum. The study was conducted in five second grade classes. Two of the classes formed a treatment group and the remaining three formed a control group. The treatment group used TEL in one mathematics lesson per week for 18 to 24 months. Other lessons were not changed. The difference in learning performance between the groups tested using a post-test; for that, we used a mathematics performance test and a mathematics fluency test. The results showed that the treatment group using TEL got statistically significantly higher learning performance results compared to the control group. The difference in arithmetic fluency was not statistically significant even though there was a small difference in favor of the treatment group. However, the difference in errors made in the fluency test was statistically significant in favor of the treatment group.Peer reviewe

Effects of regular use of scalable, technology enhanced solution for primary mathematics education

Mathematics is one of the key subjects in any school curriculum and most teachers agree that mathematical skills are important for students to master. There is an abundance of research in learning mathematics and a consensus exists among researchers that technology can enhance the learning process. However, many factors need to be taken into consideration when introducing technology into teaching mathematics. Developing a more natural collaboration between learning technology experts, teachers, and students ensures all stakeholders are considered. Involving teachers early on helps develop enduring commitment to innovations and practical solutions. Moreover, creating a culture of collaboration between experts in the field and teachers brings to bear the best of what both worlds have to offer. This thesis synthesizes six papers and offers additional findings that focus on how technology experts can collaborate with elementary teachers to improve student learning outcomes. We focus on managing educational change in ways that improve the sustainability of innovations. We also explore how technical and teaching experts co-create effective lesson plans. In one of the six papers we collected and reported teachers’ responses to survey questions covering typical usage patterns on a platform. Teachers’ direct feedback was collected and incorporated to improve technical solutions. Moreover, one study was conducted abroad to measure the effect of culture on the teaching and learning process. Evidence of effectiveness of technologically enhanced lessons and corresponding homework was based on multiple studies in grades 1 - 3, covering 379 students. The effectiveness of educational technology was measured based on two variables: student performance in mathematics, based on the learning objectives specified in the curriculum, and arithmetic fluency measured by how rapidly and accurately students solved basic arithmetic operations. Statistically significant findings show that educational technology can improve two target variables when comparing students who did not use educational technology to students who did. An additional effect size analysis was conducted to verify and compare results with previous research. Based on these results, platform use produced the same or better effect than previous studies. Based on teacher feedback and user growth on the platform, we managed to integrate technology into the regular school classroom in meaningful and sustainable ways. We were clearly able to support teachers in their practice in a manner that resulted in noticeable student achievement gains. A survey revealed a need to emphasize new features that were introduced to the platform in teacher training programs. Teachers also reported having a positive attitude towards the platform and the initiative gained wide acceptance among their peers.Matematiikka on yksi tärkeimmistä kouluaineista pelkästään tuntimääräisesti mitattunakin. Matematiikan osaamista ja oppimista pidetään yleisesti tärkeänä ja arvostettuna taitona. Matematiikan oppimisesta on valtavasti tutkimusta ja tutkijoiden keskuudessa vallitsee yhteisymmärrys tietotekniikan positiivisista mahdollisuuksista edistää matematiikan oppimista. Tietotekniikan ja oppimisen vuorovaikutus on kuitenkin monisyinen vyyhti ja sen onnistunut hyödyntäminen vaatii tutkijoiden, opettajien ja oppilaiden välistä tiivistä ja vuorovaikutteista yhteistyötä. Uusien innovaatioiden ja kokeilujen onnistumiselle ja niihin sitoutumiselle luodaan vahva pohja, kun opettajat otetaan mukaan kehitystyöhön ensimetreiltä lähtien. Tällaisen tiiviin yhteistyökulttuurin vaaliminen mahdollistaa käytännön työn ja teorian vahvuuksien hyödyntämisen. Tämä väitöstyö koostuu kuudesta artikkelista. Artikkelit kuvaavat, kuinka tutkijat ja opettajat työskentelivät yhdessä parantaakseen oppilaiden matematiikan oppimista. Tavoitteenamme oli muuttaa koulun käytänteitä pitkäjänteisesti ja kestävällä tavalla. Tutkimme kuinka tutkijat ja opettajat pystyivät yhdessä luomaan onnistuneita ja tehokkaita oppimiskokonaisuuksia. Opettajat olivat koko ajan kehitystyön keskiössä. Yhdessä kuudesta artikkelista tutkittiin kyselytutkimuksen avulla opettajien kokemuksia ja käyttötottumuksia. Näitä vastauksia hyödynnettiin teknisessä kehitystyössä ja hyvien käytänteiden hiomisessa. Yksi väitöskirjan tutkimuksista tehtiin ulkomailla opetus- ja oppimiskulttuureista vaikutusten huomioimiseksi. Sähköisten oppituntien ja kotitehtävien vaikuttavuuden arviointi perustuu useisiin 1.-3. luokilla tehtyihin tutkimuksiin ja kaikkiaan 379 oppilaan vastauksiin. Sähköisten oppituntien vaikuttavuutta arvioitiin kahden eri mittarin perusteella. Ensin matematiikan taitojen perusteella, eli kuinka hyvin kunkin luokka-asteen oppimistavoitteet olivat täyttyneet ja myöhemmin myös laskusujuvuuden perusteella, eli kuinka nopeasti ja tarkasti oppilaat pystyivät laskemaan peruslaskutoimituksia. Tulokset osoittavat, että opetusteknologian avulla pystytään parantamaan oppilaiden suoriutumista edellä mainittujen osa-alueiden osalta verrattuna oppilaisiin, jotka eivät käyttäneet opetusteknologiaa. Tulokset olivat tilastollisesti merkitseviä. Näiden tulosten varmistamiseksi laskettiin vaikuttavuuden suuruus ja sitä verrattiin aiempiin alan tutkimuksiin. Tulosten perusteella sähköisillä oppitunneilla oli sama tai parempi vaikuttavuus kuin aiemmissa tutkimuksissa. Opettajien palautteiden ja kasvavan käyttäjämäärän perusteella voidaan sanoa, että onnistuimme tavoitteessamme integroida opetusteknologiaa mielekkäällä tavalla osaksi koulutyötä. Onnistuimme myös tukemaan ja auttamaan opettajia opetustyössään ja samalla merkittävästi parantamaan oppilaiden suoriutumista. Kyselytutkimuksen perusteella huomasimme, että uusien ominaisuuksien kouluttamiseen tulee kiinnittää enemmän huomiota. Samassa tutkimuksessa opettajat raportoivat olevansa tyytyväisiä alustaan ja sähköiset oppitunnit näyttävät saaneen vankan jalansijan suomalaisessa opettajakunnassa