Classifying and recognizing students' sorting algorithm implementations in a data structures and algorithms course

We discuss an instrument for recognizing and classifying algorithms (Aari) in terms of whiteboxtesting. We examined freshmen students' sorting algorithm implementations in a datastructures and algorithms course in two rounds: at the beginning of the course before thestudents received any instruction on sorting algorithms, and after taking lectures on sortingalgorithms. We evaluated the performance of Aari with the implementations of each roundseparately. The results show that the sorting algorithms (in Java), which Aari has been trainedto recognize (Insertion sort, Selection sort, Bubble sort, Quicksort and Mergesort), arerecognized with an average accuracy of about 90%. When considering all the submitted sortingalgorithm implementations (including the variations of the standard algorithms), Aari achievedan overall accuracy being 71% and 81% for the first and second round, respectively. The manual analysis of the implementations revealed that students have manymisconceptions related to sorting algorithms. For example, they include unnecessary swaps intheir Insertion or Selection sort implementations. Based on the data, we present acategorization of these types of variations. We also discuss how these categories can be used toimprove education of sorting methods, and to further develop Aari as a white-box testing tool,which gives feedback to the students on their inappropriate solutions and asks them to correctthe problem; something black-box testing is not capable of doing

Tools and Environments

Teaching and learning how to build software are central aspects of computing education, and the tools which we use to support this are themselves a focus of research and innovation. This chapter considers tools designed or predominately used for education; from software development environments to automatic assessment tools, visualization, and educational games platforms. It looks at not just the history and state-of-the-art of these tools, but also at the challenges and opportunities in researching with and about them

Sistema de apoio à submissão e avaliação de trabalhos académicos com componentes gráficas

Tese de mestrado integrado. Engenharia Informática e Computação. Faculdade de Engenharia. Universidade do Porto. 201

Promoting Programming Learning. Engagement, Automatic Assessment with Immediate Feedback in Visualizations

The skill of programming is a key asset for every computer science student. Many studies have shown that this is a hard skill to learn and the outcomes of programming courses have often been substandard. Thus, a range of methods and tools have been developed to assist students’ learning processes. One of the biggest fields in computer science education is the use of visualizations as a learning aid and many visualization based tools have been developed to aid the learning process during last few decades. Studies conducted in this thesis focus on two different visualizationbased tools TRAKLA2 and ViLLE. This thesis includes results from multiple empirical studies about what kind of effects the introduction and usage of these tools have on students’ opinions and performance, and what kind of implications there are from a teacher’s point of view. The results from studies in this thesis show that students preferred to do web-based exercises, and felt that those exercises contributed to their learning. The usage of the tool motivated students to work harder during their course, which was shown in overall course performance and drop-out statistics. We have also shown that visualization-based tools can be used to enhance the learning process, and one of the key factors is the higher and active level of engagement (see. Engagement Taxonomy by Naps et al., 2002). The automatic grading accompanied with immediate feedback helps students to overcome obstacles during the learning process, and to grasp the key element in the learning task. These kinds of tools can help us to cope with the fact that many programming courses are overcrowded with limited teaching resources. These tools allows us to tackle this problem by utilizing automatic assessment in exercises that are most suitable to be done in the web (like tracing and simulation) since its supports students’ independent learning regardless of time and place. In summary, we can use our course’s resources more efficiently to increase the quality of the learning experience of the students and the teaching experience of the teacher, and even increase performance of the students. There are also methodological results from this thesis which contribute to developing insight into the conduct of empirical evaluations of new tools or techniques. When we evaluate a new tool, especially one accompanied with visualization, we need to give a proper introduction to it and to the graphical notation used by tool. The standard procedure should also include capturing the screen with audio to confirm that the participants of the experiment are doing what they are supposed to do. By taken such measures in the study of the learning impact of visualization support for learning, we can avoid drawing false conclusion from our experiments. As computer science educators, we face two important challenges. Firstly, we need to start to deliver the message in our own institution and all over the world about the new – scientifically proven – innovations in teaching like TRAKLA2 and ViLLE. Secondly, we have the relevant experience of conducting teaching related experiment, and thus we can support our colleagues to learn essential know-how of the research based improvement of their teaching. This change can transform academic teaching into publications and by utilizing this approach we can significantly increase the adoption of the new tools and techniques, and overall increase the knowledge of best-practices. In future, we need to combine our forces and tackle these universal and common problems together by creating multi-national and multiinstitutional research projects. We need to create a community and a platform in which we can share these best practices and at the same time conduct multi-national research projects easily.Siirretty Doriast

Automatic Feedback for UML Modeling Exercises as an Extension of INLOOP

In recent years, e-learning systems have become an important part of normal and university education. One reason for this is the growth in student numbers. INLOOP is an assessment system for object-oriented programming and is used in a beginner software engineering course of the TU Dresden. Unfortunately, for the course, there is no assessment system for object-oriented modeling. This thesis extends the INLOOP concept with functionality to assess object-oriented modeling. For that, it introduces INLOOM as a constraint-based model assessment system that works well with the architecture and workflow of INLOOP. It is based on a two-stage system that generates constraint-based test sets out of expert solution models and uses these to generate feedback and a score for a student's solution model. The system was designed by first surveying the literature on model assessment systems and then creating a design that is easy to include into INLOOP. To assess the new INLOOM system, a proof-of-concept realization for analysis UML class models is introduced. For evaluation, this realization is used to assess student solutions for tasks of multiple exams. The results of the assessment system are then compared with the assessments of human instructors. It could be shown that the INLOOM system works as well as other comparable systems. In conclusion, the INLOOM system can be used alongside INLOOP to improve the student feedback in the beginner software engineering course

Ohjelman staattinen analyysi lajittelualgoritmien tunnistamiseksi

Automaattinen ohjelman analyysi ja ohjelmakoodin tunnistus on kiinnostava ongelma tietotekniikka-alalla. Syy tähän löytyy ohjelmistoteollisuudesta ja varsinkin ohjelmiston elinkaaren yhdestä vaiheesta: ylläpidosta. Automaattisella ohjelman ymmärtämisellä ja ohjelmakoodin tunnistuksella tarkoitetaan järjestelmää, joka voisi kertoa käyttäjälle, mitä syötteenä saatu ohjelma näyttäisi tekevän, mitä algoritmia se muistuttaa ja kuinka paljon tai millainen rakenne ja tyyli ohjelmalla on. Tällainen järjestelmä voi - hyvin ja laajasti kehitettynä - auttaa ohjelmistokehittäjiä ymmärtämään ylläpidettävän järjestelmän toimintaa ja säästämään heitä lukemasta ohjelmakoodia, joka on aikaa vievää toimintaa. Tällaisella järjestelmällä olisi käyttöä myös muissa ohjelmiston elinkaaren vaiheissa: järjestelmä voisi helpottaa ohjelmistokehittäjien työtä ohjelmiston verifiointi- ja validointitehtävissä huomattavasti. Kyseisestä järjestelmästä olisi apua myös yliopistoissa ja korkeakouluissa tarjolla olevien tietotekniikan kurssien yhteydessä suoritettavien tehtävien automaattisessa tarkastuksessa. Tietyt tietotekniikan kurssit ovat yleensä laajoja, ja niiden suorittaminen vaatii opiskelijoilta monen tehtävän suorittamista. Järjestelmä voisi ottaa vastaan opiskelijan lähettämän tiettyyn aiheeseen liittyvän tehtävän, ja kertoa sitten opettajalle, onko tehtävä ratkaistu oikein, ja jos ei, niin kuinka lähellä oikeaa ratkaisua se on. Tässä diplomityössä tutkitaan aikaisempia automaattiseen ohjelman analyysiin ja ohjelmakoodin tunnistukseen liittyviä töitä ja esitetään erilaisia lähestymistapoja ongelmaan. Uutena lähestymistapana esitetään mm ohjelmakoodissa käytettyjen muuttujien lukumäärään ja rooliin pohjautuva staattinen ohjelman analyysi ja koodin tunnistus. Tämä työ rajataan koskemaan ainoastaan tunnettuja lajittelualgoritmeja, ja järjestelmän jatkokehitys ja sen toiminnan laajentaminen muiden algoritmien käsittelemiseksi jätetään tulevalle tutkimukselle. Lopuksi kerrotaan työn muista rajauksista ja esitetään muutama ehdotus tulevalle tutkimukselle

Optical Character Recognition based approach for automatic Image Marking Process

In today's world, programming teachers perform tedious tasks, which are time consuming; for instance, preparing and marking daily assignments, preparing and marking programming projects, preparing and marking short exams, etc. These tasks distract programming teachers from fulfilling their key role – teaching. Therefore, using automated marking approach with ability to communicate with students is highly desirable. Despite the existing approaches for automated student marking, there is still a need for more improvement. An automated program marking approach is proposed in this study based on a proposal by iMarking®. This approach automates the process of marking and assignments submission and facilitates the communication between teachers and students by designing and implementing a web-based application. In addition, the proposed approach adopts Optical Character Recognition (OCR) to extract the text from images to be evaluated using novel evaluation metrics. The novel evaluation metrics are formulated based on observation and experiment and aim to calculate the matching similarity and mismatching percentage of the submitted student answers when compared with the optimal answers. Evaluation results from a sample of 100 different programming questions show that the proposed approach is efficient in automatically marking the student answers with 100% accuracy. Furthermore, it is found to be time saving – approximately 197 seconds for marking ten questions – which is in line with the objective of creating a more efficient system for teachers