Introductory programming: a systematic literature review

As computing becomes a mainstream discipline embedded in the school curriculum and acts as an enabler for an increasing range of academic disciplines in higher education, the literature on introductory programming is growing. Although there have been several reviews that focus on specific aspects of introductory programming, there has been no broad overview of the literature exploring recent trends across the breadth of introductory programming. This paper is the report of an ITiCSE working group that conducted a systematic review in order to gain an overview of the introductory programming literature. Partitioning the literature into papers addressing the student, teaching, the curriculum, and assessment, we explore trends, highlight advances in knowledge over the past 15 years, and indicate possible directions for future research

Programming Process, Patterns and Behaviors: Insights from Keystroke Analysis of CS1 Students

With all the experiences and knowledge, I take programming as granted. But learning to program is still difficult for a lot of introductory programming students. This is also one of the major reasons for a high attrition rate in CS1 courses. If instructors were able to identify struggling students then effective interventions can be taken to help them. This thesis is a research done on programming process data that can be collected non-intrusively from CS1 students when they are programming. The data and their findings can be leveraged in understanding students’ thought process, detecting patterns and identifying behaviors that could possibly help instructors to identify struggling students, help them and design better courses

Multivariate Gradient Analysis for Evaluating and Visualizing a Learning System Platform for Computer Programming

This paper explores the application of canonical gradient analysis to evaluate and visualize student performance and acceptance of a learning system platform. The subject of evaluation is a first year BSc module for computer programming. This uses ‘Ceebot’, an animated and immersive game-like development environment. Multivariate ordination approaches are widely used in ecology to explore species distribution along environmental gradients. Environmental factors are represented here by three ‘assessment’ gradients; one for the overall module mark and two independent tests of programming knowledge and skill. Response data included Likert expressions for behavioral, acceptance and opinion traits. Behavioral characteristics (such as attendance, collaboration and independent study) were regarded to be indicative of learning activity. Acceptance and opinion factors (such as perceived enjoyment and effectiveness of Ceebot) were treated as expressions of motivation to engage with the learning environment. Ordination diagrams and summary statistics for canonical analyses suggested that logbook grades (the basis for module assessment) and code understanding were weakly correlated. Thus strong module performance was not a reliable predictor of programming ability. The three assessment indices were correlated with behaviors of independent study and peer collaboration, but were only weakly associated with attendance. Results were useful for informing teaching practice and suggested: (1) realigning assessments to more fully capture code-level skills (important in the workplace); (2) re-evaluating attendance-based elements of module design; and (3) the overall merit of multivariate canonical gradient approaches for evaluating and visualizing the effectiveness of a learning system platform

Celebrating Faculty Achievement 2015

https://digitalcommons.lasalle.edu/celebratingfaculty/1003/thumbnail.jp

Learning Transfer in Novice Programmers: A Preliminary Study

Learning transfer refers to the ability to correctly apply learned skills, knowledge and behaviors to new situations or contexts. This paper explores novice programmers' transfer through the analysis of two related coding tasks completed by CS1 students as part of their assessment. The first task was a take-home practical and the second task was a lab practical exam; both tasks requested the implementation of a C function with an integer parameter from which the digits are to be extracted and operated on. The solution set generated from each task by a cohort of 255 CS1 students has been explored and classified in order to determine the extent of transfer from the practice task to the later assessment task. This classification shows 36.5% of students consolidated or extended the acquired skills and 13% at least partly; 38%, on the other hand, failed to recall their previous valid strategy or to devise a better one, and were unsuccessful in the second task. On the positive side, 9% of students devised a different and improved strategy in the exam, indicating additional learning had occurred in between the two tasks. Peer review of key coding tasks could improve transfer by forcing weaker students to compare and evaluate different design strategies

An algorithm and a tool for the automatic grading of MOOC learners from their contributions in the discussion forum

MOOCs (massive open online courses) have a built-in forum where learners can share experiences as well as ask questions and get answers. Nevertheless, the work of the learners in the MOOC forum is usually not taken into account when calculating their grade in the course, due to the difficulty of automating the calculation of that grade in a context with a very large number of learners. In some situations, discussion forums might even be the only available evidence to grade learners. In other situations, forum interactions could serve as a complement for calculating the grade in addition to traditional summative assessment activities. This paper proposes an algorithm to automatically calculate learners' grades in the MOOC forum, considering both the quantitative dimension and the relevance in their contributions. In addition, the algorithm has been implemented within a web application, providing instructors with a visual and a numerical representation of the grade for each learner. An exploratory analysis is carried out to assess the algorithm and the tool with a MOOC on programming, obtaining a moderate positive correlation between the forum grades provided by the algorithm and the grades obtained through the summative assessment activities. Nevertheless, the complementary analysis conducted indicates that this correlation may not be enough to use the forum grades as predictors of the grades obtained through summative assessment activities.This work was supported in part by the FEDER/Ministerio de Ciencia, Innovación y Universidades;Agencia Estatal de Investigación, through the Smartlet Project under Grant TIN2017-85179-C3-1-R, and in part by the Madrid Regional Government through the e-Madrid-CM Project under Grant S2018/TCS-4307, a project which is co-funded by the European Structural Funds (FSE and FEDER). Partial support has also been received from the European Commission through Erasmus+ Capacity Building in the Field of Higher Education projects, more specifically through projects LALA (586120-EPP-1-2017-1-ES-EPPKA2-CBHE-JP), InnovaT (598758-EPP-1-2018-1-AT-EPPKA2-CBHE-JP), and PROF-XXI (609767-EPP-1-2019-1-ES-EPPKA2-CBHE-JP)

O uso da robótica na intervenção com crianças com autismo em macau: um estudo exploratório com o milo

Robotics are being used in the intervention with children with Autism Spectrum Disorder (ASD) in many places and already for many years. Many robots were developed and different studies are being made in order to evaluate its effectiveness. “Socially Assistive Robotics” is shown to be effective in different areas mainly in social and emotional development. Milo, a robot developed by a team led by Richard Margolin for the Robots4Autism program (RoboKind, 2020), is one of the robots whose use is reported to be successful. In Macao there is no report of studies or experiences on the use of robots in the intervention with children with ASD. In a collaboration between the Macao Science Centre, the Macao Autism Association (MAA) and the University of Saint Joseph, an exploratory study was developed to understand the applicability of Milo to the work with children with ASD in Macao. The study showed that the robot is able to facilitate social and emotional competences of children with ASD. However, several limitations including language, cultural differences, the inexperienced facilitators and the level of sessions are too simple for the participants to be aware of that may affect the effectiveness of the intervention. It is important to show that the adoption of Milo in Macao for intervening children with ASD can be further implemented, with better practical solutions.A robótica tem sido utilizada na intervenção com crianças com Transtorno do Espectro do Autismo (TEA) em muitos lugares e há já muitos anos. Muitos robôs foram desenvolvidos e diversos estudos foram feitos para avaliar a sua eficácia. A “Robótica Socialmente Assistida” mostra-se eficaz em diferentes áreas, principalmente no desenvolvimento social e emocional. Milo, um robô desenvolvido por uma equipe liderada por Richard Margolin para o programa Robots4Autism (RoboKind, 2020), é um dos robôs cuja utilização é considerada como bem-sucedida. Em Macau não há relato de estudos ou experiências sobre a utilização de robôs na intervenção com crianças com ASD. Numa colaboração entre o Centro de Ciência de Macau, a Associação de Autismo de Macau (MAA) e a Universidade de São José, foi desenvolvido um estudo exploratório para compreender a aplicabilidade de Milo ao trabalho com crianças com TEA em Macau. O estudo mostrou que o robô é capaz de facilitar as competências sociais e emocionais de crianças com TEA. No entanto, várias limitações, incluindo o idioma, diferenças culturais, a falta de experiência dos facilitadores e o nível de dificuldade dos exercícios podem afetar a eficácia da intervenção. O estudo permite concluir que o Milo pode ser utilizado na intervenção com crianças com EA em Macau desde que com melhores soluções práticas

Analyzing the behavior of students regarding learning activities, badges, and academic dishonesty in MOOC environment

Mención Internacional en el título de doctorThe ‘big data’ scene has brought new improvement opportunities to most products and services, including education. Web-based learning has become very widespread over the last decade, which in conjunction with the Massive Open Online Course (MOOC) phenomenon, it has enabled the collection of large and rich data samples regarding the interaction of students with these educational online environments. We have detected different areas in the literature that still need improvement and more research studies. Particularly, in the context of MOOCs and Small Private Online Courses (SPOCs), where we focus our data analysis on the platforms Khan Academy, Open edX and Coursera. More specifically, we are going to work towards learning analytics visualization dashboards, carrying out an evaluation of these visual analytics tools. Additionally, we will delve into the activity and behavior of students with regular and optional activities, badges and their online academically dishonest conduct. The analysis of activity and behavior of students is divided first in exploratory analysis providing descriptive and inferential statistics, like correlations and group comparisons, as well as numerous visualizations that facilitate conveying understandable information. Second, we apply clustering analysis to find different profiles of students for different purposes e.g., to analyze potential adaptation of learning experiences and pedagogical implications. Third, we also provide three machine learning models, two of them to predict learning outcomes (learning gains and certificate accomplishment) and one to classify submissions as illicit or not. We also use these models to discuss about the importance of variables. Finally, we discuss our results in terms of the motivation of students, student profiling, instructional design, potential actuators and the evaluation of visual analytics dashboards providing different recommendations to improve future educational experiments.Las novedades en torno al ‘big data’ han traído nuevas oportunidades de mejorar la mayoría de productos y servicios, incluyendo la educación. El aprendizaje mediante tecnologías web se ha extendido mucho durante la última década, que conjuntamente con el fenómeno de los cursos abiertos masivos en línea (MOOCs), ha permitido que se recojan grandes y ricas muestras de datos sobre la interacción de los estudiantes con estos entornos virtuales de aprendizaje. Nosotros hemos detectado diferentes áreas en la literatura que aún necesitan de mejoras y del desarrollo de más estudios, específicamente en el contexto de MOOCs y cursos privados pequeños en línea (SPOCs). En la tesis nos hemos enfocado en el análisis de datos en las plataformas Khan Academy, Open edX y Coursera. Más específicamente, vamos a trabajar en interfaces de visualizaciones de analítica de aprendizaje, llevando a cabo la evaluación de estas herramientas de analítica visual. Además, profundizaremos en la actividad y el comportamiento de los estudiantes con actividades comunes y opcionales, medallas y sus conductas en torno a la deshonestidad académica. Este análisis de actividad y comportamiento comienza primero con análisis exploratorio proporcionando variables descriptivas y de inferencia estadística, como correlaciones y comparaciones entre grupos, así como numerosas visualizaciones que facilitan la transmisión de información inteligible. En segundo lugar aplicaremos técnicas de agrupamiento para encontrar distintos perfiles de estudiantes con diferentes propósitos, como por ejemplo para analizar posibles adaptaciones de experiencias educativas y sus implicaciones pedagógicas. También proporcionamos tres modelos de aprendizaje máquina, dos de ellos que predicen resultados finales de aprendizaje (ganancias de aprendizaje y la consecución de certificados de terminación) y uno para clasificar que ejercicios han sido entregados de forma deshonesta. También usaremos estos tres modelos para analizar la importancia de las variables. Finalmente, discutimos todos los resultados en términos de la motivación de los estudiantes, diferentes perfiles de estudiante, diseño instruccional, posibles sistemas actuadores, así como la evaluación de interfaces de analítica visual, proporcionando recomendaciones que pueden ayudar a mejorar futuras experiencias educacionales.Programa Oficial de Doctorado en Ingeniería TelemáticaPresidente: Davinia Hernández Leo.- Secretario: Luis Sánchez Fernández.- Vocal: Adolfo Ruiz Callej

Identifying evidences of computer programming skills through automatic source code evaluation

Orientador: Roberto PereiraCoorientador: Eleandro MaschioTese (doutorado) - Universidade Federal do Paraná, Setor de Ciências Exatas, Programa de Pós-Graduação em Informática. Defesa : Curitiba, 27/03/2020Inclui referências: p. 98-106Área de concentração: Ciência da ComputaçãoResumo: Esta tese e contextualizada no ensino de programacao de computadores em cursos de Computacao e investiga aspectos e estrategias para avaliacao automatica e continua de codigos fonte desenvolvidos pelos alunos. O estado da arte foi identificado por meio de revisao sistematica de literatura e revelou que as pesquisas anteriores tendem a realizar avaliacoes baseadas em aspectos tecnicos de codigos fonte, como a avaliacao de corretude funcional e a deteccao de erros. Avaliacoes baseadas em habilidades, por outro lado, sao pouco exploradas e possuem potencial para fornecer detalhes a respeito de habilidades representadas por conceitos de alto nivel, como desvios condicionais e estruturas de repeticao. Um metodo de identificacao automatica de evidencias de aprendizado e entao proposto como uma abordagem baseada em habilidades para a avaliacao automatica de codigos fonte de programacao. O metodo e caracterizado pela implementacao de diferentes estrategias para avaliacao de codigos fonte, identificacao de evidencias de habilidades de programacao, e representacao destas habilidades em um modelo do aluno. Experimentos realizados em ambientes controlados (bases de dados artificiais) mostraram que estrategias automaticas de avaliacao de codigo fonte sao viaveis. Experimentos conduzidos em ambientes reais (codigos fonte produzidos por alunos) produziram resultados semelhantes aos ambientes controlados, entretanto revelaram limitacoes relacionadas a implementacao das estrategias, como vulnerabilidades a sintaxes inesperadas e falhas em expressoes regulares. Um conjunto de habilidades foi selecionado para compor o modelo do aluno, representado por uma rede bayesiana dinamica. Por meio de experimentos foi demonstrado que a alimentacao do modelo com evidencias resultantes da avaliacao automatica de codigos fonte permite o acompanhamento do progresso das habilidades dos alunos. Finalmente, as estrategias automaticas em conjunto com os recursos do modelo do aluno permitiram a demonstracao da avaliacao baseada em habilidades, que se mostrou um recurso valioso para identificacao de solucoes funcionalmente corretas, porem conceitualmente incorretas; quando o programa e funcionalmente correto, retornando resultados esperados a determinadas entradas, porem foi construido com recursos e conceitos incorretos. Palavras-chave: Programacao de Computadores, Avaliacao Automatica, Avaliacao Baseada em HabilidadesAbstract: This thesis is contextualized in the teaching of computer programming in Computing courses and investigates aspects and strategies for automatic and continuous evaluation of student developed source codes. The state of the art was identified through systematic literature review and revealed previous research tends to perform evaluations based on source codes technical aspects, such as functional correctness assessment and error detection. Skills-based assessments, in turn, are less explored although having potential to provide details of skills represented by high-level concepts, such as conditionals and repetition structures. A method for automatic identification of learning evidences is then proposed as a skills-based approach to automatic evaluation of programming source codes. The method is characterized by implementing different strategies for source code evaluation, identifying evidences of programming skills, and representing these skills in a student model. Experiments conducted in controlled scenarios (testing datasets) have shown automatic source code evaluation strategies are viable. Experiments conducted in real scenarios (student-made source codes) produced results similar to controlled scenarios, however, implementation-related limitations were revealed for some strategies, such as vulnerabilities to unexpected syntax and flaws in regular expressions. A skill set was selected to compose our student model, represented by a Dynamic Bayesian Network. Experiments have shown feeding the model with evidences resulting from source codes automatic evaluation allows monitoring students' skills progress. Finally, automatic strategies coupled with student model capabilities enabled demonstrating skills-based assessment, which showed a valuable resource for identifying functionally correct source codes, but conceptually incorrect; when a program is correct functionally, returning expected results to specific inputs, but it was built with erroneous concepts and resources. Keywords: Computer Programming, Automatic Evaluation, Skills-Based Assessmen

Affective modelling and feedback in programming practice systems

指導教員：角