Computer-based assessment system for e-learning applied to programming education

Tese de mestrado integrado. Engenharia Informática e Computação. Faculdade de Engenharia. Universidade do Porto. 201

Computer-based assessment system for e-learning applied to programming education

Tese de Mestrado Integrado. Engenharia Informática e Computação. Faculdade de Engemharia. Universidade do Porto. 201

Programming exercises evaluation systems: an interoperability survey

Learning computer programming requires solving programming exercises. In computer programming courses teachers need to assess and give feedback to a large number of exercises. These tasks are time consuming and error-prone since there are many aspects relating to good programming that should be considered. In this context automatic assessment tools can play an important role helping teachers in grading tasks as well to assist students with automatic feedback. In spite of its usefulness, these tools lack integration mechanisms with other eLearning systems such as Learning Management Systems, Learning Objects Repositories or Integrated Development Environments. In this paper we provide a survey on programming evaluation systems. The survey gathers information on interoperability features of these systems, categorizing and comparing them regarding content and communication standardization. This work may prove useful to instructors and computer science educators when they have to choose an assessment system to be integrated in their e-Learning environment

Exploring Automated Code Evaluation Systems and Resources for Code Analysis: A Comprehensive Survey

The automated code evaluation system (AES) is mainly designed to reliably assess user-submitted code. Due to their extensive range of applications and the accumulation of valuable resources, AESs are becoming increasingly popular. Research on the application of AES and their real-world resource exploration for diverse coding tasks is still lacking. In this study, we conducted a comprehensive survey on AESs and their resources. This survey explores the application areas of AESs, available resources, and resource utilization for coding tasks. AESs are categorized into programming contests, programming learning and education, recruitment, online compilers, and additional modules, depending on their application. We explore the available datasets and other resources of these systems for research, analysis, and coding tasks. Moreover, we provide an overview of machine learning-driven coding tasks, such as bug detection, code review, comprehension, refactoring, search, representation, and repair. These tasks are performed using real-life datasets. In addition, we briefly discuss the Aizu Online Judge platform as a real example of an AES from the perspectives of system design (hardware and software), operation (competition and education), and research. This is due to the scalability of the AOJ platform (programming education, competitions, and practice), open internal features (hardware and software), attention from the research community, open source data (e.g., solution codes and submission documents), and transparency. We also analyze the overall performance of this system and the perceived challenges over the years

Selection of the Best K-Gram Value on Modified Rabin-Karp Algorithm

The Rabin-Karp algorithm is used to detect similarity using hashing techniques, from related studies modifications have been made in the hashing process but in previous studies have not been conducted research for the best k value in the K-Gram process. At the stage of stemming the Nazief & Adriani algorithm is used to transform the words into basic words. The researcher uses several variations of K-Gram values to determine the best K-Gram values. The analysis was performed using Ukara Enhanced public data obtained from the Kaggle with a total of 12215 data. The student essay answers data totaled to 258 data in the group A and 305 in the group B, every student essay answers data in each group will be compared with the answers of other fellow group member. Research results are the value of k = 3 has the best performance which has the highest some interpretations of 1-14%  (Little degree of similarity) and 15-50% (Medium level of similarity) compared to values of k = 5, 7, and 9 which have the highest number of interpretation results 0%-0.99% (Document is different). However, if the students essay answers compared have 100% (Exactly the same) interpretations, the k value on K-Gram does not affect the results

Source code analysis on student assignments using machine learning techniques

Abstract. To increase the success in computer programming courses, it is important to understand the learning process and common difficulties faced by students. Although several studies have investigated possible relationships between students performance and self-regulated learning characteristics, little attention has been given the source code produced by students in this regard. Such source code might contain valuable information about their learning process, specially in a context where practical programming assignments are frequent and students write source code constantly during the course. This poses the following research questions: What is the relationship between the characteristics of students source code and their performance in a computer programming course?. What is the relationship between source code features and self-regulated learning characteristics (i.e., motivation and learning strategies) in a computer programming course?. How the source code and self-regulated features can predict the students' performance? In order to answer these questions, a strategy to support the correlation analysis among students performance, motivation, use of learning strategies, and source code metrics in computer programming courses is proposed. A comprehensive case study is presented to evaluate the strategy. Additionally, an automatic grading tool for programming assignments was used, which facilitated to obtain the source code of the participants for further automatic source code analysis. Moreover, self-regulated learning characteristics were collected using the Motivated Strategies for Learning Questionnaire (MSLQ). Results show that the main features from source code which are significantly related to students performance and self-regulated learning features are: length-related metrics, with mainly positive correlations; and Halstead complexity measures, correlated negatively. In the light of the findings of this study, it is possible to understand better students source code as an artifact that can be used to monitorize several characteristics related to self-regulated learning, course performance, and in general, their learning process. In this way, more research in the area is required to verify if these relationships could give to computing educators new ways to identify and help students with problems.Para mejorar el éxito de los estudiantes en los cursos de programación, es importante entender el proceso de aprendizaje y las dificultades comunes que enfrentan los estudiantes. Aunque muchos estudios han investigado las posibles relaciones entre el rendimiento de los estudiantes y aspectos de la auto-regulación del aprendizaje, poca atención se le ha dado al código fuente producido por los estudiantes. El cual puede contener información valiosa acerca de su proceso de aprendizaje. Esto es especialmente cierto en contextos donde las actividades prácticas de programación son frecuentes y los estudiantes escriben código fuente constantemente durante el desarrollo del curso. Lo anterior, plantea las siguientes preguntas de investigación: ¿Cuál es la relación entre las características del código fuente de los estudiantes y su rendimiento en un curso de programación de computadores?. ¿Cuál es la relación entre las características del código fuente y características de aprendizaje auto-regulado (motivación y estrategias de aprendizaje) en un curso de programación de computadores?. ¿Cómo el código fuente y las características de aprendizaje auto-regulado pueden predecir el rendimiento de los estudiantes? Para responder estas preguntas, se presenta una estrategia para realizar el análisis de correlaciones entre el rendimiento de los estudiantes, motivación, el uso de estrategias de aprendizaje, y las métricas de código fuente en cursos de programación de computadores. Un caso de estudio exhaustivo es presentado para evaluar la estrategia propuesta usando datos recolectados de estudiantes. Además se usaba una herramienta de calificación automática para evaluar las practicas, lo cual facilitaba la obtención de código fuente de estudiantes para su análisis posterior. Las características de aprendizaje auto-regulado fueron obtenidas usando el cuestionario: Motivated Strategies for Learning Questionnaire Colombia (MSLQColombia). Los resultados muestran que las principales características del código fuente que están relacionadas con el rendimiento de los estudiantes y características auto-reguladas son: las métricas de longitud, que se correlaciona positivamente; y las medidas de complejidad de Halstead, las cuales se correlacionan negativamente. Dados los resultados, es posible entender mejor el código fuente de los estudiantes como un artefacto que puede ser usado para monitorear características relacionadas con el aprendizaje auto-regulado, rendimiento en el curso, y en general, su proceso de aprendizaje. De esta forma, investigaciones adicionales son necesarias para verificar si dichas relaciones pueden dar a los educadores nuevas herramientas para identificar y ayudar a estudiantes con problemas.Maestrí

Evaluación Automática de Resultados de Aprendizaje como un Nuevo Paradigma en la enseñanza de un Curso de Programación: La ingeniería en la sociedad 5.0

Programming education goes through the transition from the content model to the learning outcomes model and the integration of technology, where automatic assessment tools allow students to support them in practice; This makes learning inclusive by proposing a transversal approach so that the necessary programming skills are achieved. In this sense, this paper presents a strategy that evaluates the source code and analyzes it using software metrics to identify students' learning results in a programming course. A strategy was developed that integrates an automatic source code evaluation tool, which allowed us to identify How an evaluation-based approach supports the learning process, time, and impact in a computer programming course? The results show that the strategy helps the transition from a programming course to a learning outcomes model and reduces the evaluation time without affecting students' grades, compared to the traditional way. Finally, it is essential to highlight that the development of strategies that integrate tools that support the teaching-learning and evaluation process in programming courses has a positive impact on academic training and decision-making, seeking to improve students' weaknesses through the analysis of the outcomes obtained