Automatic Algorithm Recognition Based on Programming Schemas and Beacons - A Supervised Machine Learning Classification Approach

In this thesis, we present techniques to recognize basic algorithms covered in computer science education from source code. The techniques use various software metrics, language constructs and other characteristics of source code, as well as the concept of schemas and beacons from program comprehension models. Schemas are high level programming knowledge with detailed knowledge abstracted out. Beacons are statements that imply specific structures in a program. Moreover, roles of variables constitute an important part of the techniques. Roles are concepts that describe the behavior and usage of variables in a program. They have originally been introduced to help novices learn programming. We discuss two methods for algorithm recognition. The first one is a classification method based on a supervised machine learning technique. It uses the vectors of characteristics and beacons automatically computed from the algorithm implementations of a training set to learn what characteristics and beacons can best describe each algorithm. Based on these observed instance-class pairs, the system assigns a class to each new input algorithm implementation according to its characteristics and beacons. We use the C4.5 algorithm to generate a decision tree that performs the task. In the second method, the schema detection method, algorithms are defined as schemas that exist in the knowledge base of the system. To identify an algorithm, the method searches the source code to detect schemas that correspond to those predefined schemas. Moreover, we present a method that combines these two methods: it first applies the schema detection method to extract algorithmic schemas from the given program and then proceeds to the classification method applied to the schema parts only. This enhances the reliability of the classification method, as the characteristics and beacons are computed only from the algorithm implementation code, instead of the whole given program. We discuss several empirical studies conducted to evaluate the performance of the methods. Some results are as follows: evaluated by leave-one-out cross-validation, the estimated classification accuracy for sorting algorithms is 98,1%, for searching, heap, basic tree traversal and graph algorithms 97,3% and for the combined method (on sorting algorithms and their variations from real student submissions) 97,0%. For the schema detection method, the accuracy is 88,3% and 94,1%, respectively. In addition, we present a study for categorizing student-implemented sorting algorithms and their variations in order to find problematic solutions that would allow us to give feedback on them. We also explain how these variations can be automatically recognized

Computing Education Research Compiled: Keyword Trends, Building Blocks, Creators, and Dissemination

The need for organized computing education efforts dates back to the 1950s. Since then, computing education research (CER) has evolved and matured from its early initiatives and separation from mathematics education into a respectable research specialization of its own. In recent years, a number of meta-research papers, reviews, and scientometric studies have built overviews of CER from various perspectives. This paper continues that approach by offering new perspectives on the past and present state of CER: analyses of influential papers throughout the years, of the theoretical backgrounds of CER, of the institutions and authors who create CER, and finally of the top publication venues and their citation practices. The results reveal influential contributions from early curriculum guidelines to rigorous empirical research of today, the prominence of computer programming as a topic of research, evolving patterns of learning-theory usage, the dominance of high-income countries and a cluster of 52 elite institutions, and issues regarding citation practices within the central venues of dissemination.</p

Personality representation: predicting behaviour for personalised learning support

The need for personalised support systems comes from the growing number of students that are being supported within institutions with shrinking resources. Over the last decade the use of computers and the Internet within education has become more predominant. This opens up a range of possibilities in regard to spreading that resource further and more effectively. Previous attempts to create automated systems such as intelligent tutoring systems and learning companions have been criticised for being pedagogically ineffective and relying on large knowledge sources which restrict their domain of application. More recent work on adaptive hypermedia has resolved some of these issues but has been criticised for the lack of support scope, focusing on learning paths and alternative content presentation. The student model used within these systems is also of limited scope and often based on learning history or learning styles.This research examines the potential of using a personality theory as the basis for a personalisation mechanism within an educational support system. The automated support system is designed to utilise a personality based profile to predict student behaviour. This prediction is then used to select the most appropriate feedback from a selection of reflective hints for students performing lab based programming activities. The rationale for the use of personality is simply that this is the concept psychologists use for identifying individual differences and similarities which are expressed in everyday behaviour. Therefore the research has investigated how these characteristics can be modelled in order to provide a fundamental understanding of the student user and thus be able to provide tailored support. As personality is used to describe individuals across many situations and behaviours, the use of such at the core of a personalisation mechanism may overcome the issues of scope experienced by previous methods.This research poses the following question: can a representation of personality be used to predict behaviour within a software system, in such a way, as to be able to personalise support?Putting forward the central claim that it is feasible to capture and represent personality within a software system for the purpose of personalising services.The research uses a mixed methods approach including a number and combination of quantitative and qualitative methods for both investigation and determining the feasibility of this approach.The main contribution of the thesis has been the development of a set of profiling models from psychological theories, which account for both individual differences and group similarities, as a means of personalising services. These are then applied to the development of a prototype system which utilises a personality based profile. The evidence from the evaluation of the developed prototype system has demonstrated an ability to predict student behaviour with limited success and personalise support.The limitations of the evaluation study and implementation difficulties suggest that the approach taken in this research is not feasible. Further research and exploration is required –particularly in the application to a subject area outside that of programming

Predicting programming assignment difficulty

Teaching programming is increasingly more widespread and starts at primary school level on some countries. Part of that teaching consist of students writing small programs that will demonstrate learned theory and how various things fit together to form a functional program. Multiple studies indicate that programming is difficult skill to learn and master. Some part of difficulty comes from plethora of concepts that students are expected to learn in relatively short time. Part of practicing to write programs involves feedback, which aids students’ learning of assignment’s topic, and motivation, which encourages students to continue the course and their studies. For feedback it would be helpful to know students’ opinion of a programming assignment difficulty. There are few studies that attempt to find out if there is correlation between metrics that are obtained from students’ writing a program and their reported difficulty of it. These studies use statistical models on data after the course is over. This leads to an idea if such a thing could be done while students are working on programming assignments. To do this some sort of machine learning model would be possible solution but as of now no such models exist. Due to this we will utilize idea from one of these studies to create a model, which could do such prediction. We then improve that model, which is coarse, with two additional models that are more tailored for the job. Our main results indicate that this kind of models show promise in their prediction of a programming assignment difficulty based on collected metrics. With further work these models could provide indication of a student struggling on some assignment. Using this kind of model as part of existing tools we could provide a student subtle help before his frustration grows too much. Further down the road such a model could be used to provide further exercises, if need by a student, or progress forward once he masters certain topic

Utilizing educational technology in computer science and programming courses : theory and practice

There is one thing the Computer Science Education researchers seem to agree: programming is a difficult skill to learn. Educational technology can potentially solve a number of difficulties associated with programming and computer science education by automating assessment, providing immediate feedback and by gamifying the learning process. Still, there are two very important issues to solve regarding the use of technology: what tools to use, and how to apply them? In this thesis, I present a model for successfully adapting educational technology to computer science and programming courses. The model is based on several years of studies conducted while developing and utilizing an exercise-based educational tool in various courses. The focus of the model is in improving student performance, measured by two easily quantifiable factors: the pass rate of the course and the average grade obtained from the course. The final model consists of five features that need to be considered in order to adapt technology effectively into a computer science course: active learning and continuous assessment, heterogeneous exercise types, electronic examination, tutorial-based learning, and continuous feedback cycle. Additionally, I recommend that student mentoring is provided and cognitive load of adapting the tools considered when applying the model. The features are classified as core components, supportive components or evaluation components based on their role in the complete model. Based on the results, it seems that adapting the complete model can increase the pass rate statistically significantly and provide higher grades when compared with a “traditional” programming course. The results also indicate that although adapting the model partially can create some improvements to the performance, all features are required for the full effect to take place. Naturally, there are some limits in the model. First, I do not consider it as the only possible model for adapting educational technology into programming or computer science courses. Second, there are various other factors in addition to students’ performance for creating a satisfying learning experience that need to be considered when refactoring courses. Still, the model presented can provide significantly better results, and as such, it works as a base for future improvements in computer science education.Ohjelmoinnin oppimisen vaikeus on yksi harvoja asioita, joista lähes kaikki tietojenkäsittelyn opetuksen tutkijat ovat jokseenkin yksimielisiä. Opetusteknologian avulla on mahdollista ratkaista useita ohjelmoinnin oppimiseen liittyviä ongelmia esimerkiksi hyödyntämällä automaattista arviointia, välitöntä palautetta ja pelillisyyttä. Teknologiaan liittyy kuitenkin kaksi olennaista kysymystä: mitä työkaluja käyttää ja miten ottaa ne kursseilla tehokkaasti käyttöön? Tässä väitöskirjassa esitellään malli opetusteknologian tehokkaaseen hyödyntämiseen tietojenkäsittelyn ja ohjelmoinnin kursseilla. Malli perustuu tehtäväpohjaisen oppimisjärjestelmän runsaan vuosikymmenen pituiseen kehitys- ja tutkimusprosessiin. Mallin painopiste on opiskelijoiden suoriutumisen parantamisessa. Tätä arvioidaan kahdella kvantitatiivisella mittarilla: kurssin läpäisyprosentilla ja arvosanojen keskiarvolla. Malli koostuu viidestä tekijästä, jotka on otettava huomioon tuotaessa opetusteknologiaa ohjelmoinnin kursseille. Näitä ovat aktiivinen oppiminen ja jatkuva arviointi, heterogeeniset tehtävätyypit, sähköinen tentti, tutoriaalipohjainen oppiminen sekä jatkuva palautesykli. Lisäksi opiskelijamentoroinnin järjestäminen kursseilla ja järjestelmän käyttöönottoon liittyvän kognitiivisen kuorman arviointi tukevat mallin käyttöä. Malliin liittyvät tekijät on tässä työssä lajiteltu kolmeen kategoriaan: ydinkomponentteihin, tukikomponentteihin ja arviontiin liittyviin komponentteihin. Tulosten perusteella vaikuttaa siltä, että mallin käyttöönotto parantaa kurssien läpäisyprosenttia tilastollisesti merkittävästi ja nostaa arvosanojen keskiarvoa ”perinteiseen” kurssimalliin verrattuna. Vaikka mallin yksittäistenkin ominaisuuksien käyttöönotto voi sinällään parantaa kurssin tuloksia, väitöskirjaan kuuluvien tutkimusten perusteella näyttää siltä, että parhaat tulokset saavutetaan ottamalla malli käyttöön kokonaisuudessaan. On selvää, että malli ei ratkaise kaikkia opetusteknologian käyttöönottoon liittyviä kysymyksiä. Ensinnäkään esitetyn mallin ei ole tarkoituskaan olla ainoa mahdollinen tapa hyödyntää opetusteknologiaa ohjelmoinnin ja tietojenkäsittelyn kursseilla. Toiseksi tyydyttävään oppimiskokemukseen liittyy opiskelijoiden suoriutumisen lisäksi paljon muitakin tekijöitä, jotka tulee huomioida kurssien uudelleensuunnittelussa. Esitetty malli mahdollistaa kuitenkin merkittävästi parempien tulosten saavuttamisen kursseilla ja tarjoaa sellaisena perustan entistä parempaan opetukseen

Towards a mood sensitive integrated development environment to enhance the performance of programmers

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.The aim of the research was to analyze the possibility of developing an Integrated Development Environment (IDE) that could improve a programmer‘s performance by considering their current mood. Various experiments were conducted to study this idea. However, the impact of moods on programmer performance was initially examined in the literature. Based on this, a Cognitive Programming Task Model (CPTM) was developed showing that various cognitive functions and programming activities are interrelated. A second model derived from the literature, the Cognitive Mood Model (CMM), suggested that moods are also interrelated with various cognitive functions. Combining these two models indirectly suggests a relation between moods and programming tasks, which was presented as the Mood Programming Model (MPM). As direct empirical support was lacking for this relation, two experiments were conducted to study the effect mood could have on performance in a debug task. Validated mood-inducing movie clips were used to induce specific moods along two-mood dimensions: valence and arousal. The first study was conducted online. The results showed that arousal is a significant factor when considering programmer performance whereas valence was found to have no significant effect. The second study was a continuation study to validate the findings from the first study within lab conditions. The results were not able to confirm the findings of the first experiment. The reasons for these findings were explained accordingly. As mood was found to have an effect on a programmer‘s coding and debugging performance, this factor might be considered when developing a support system. The next step in the research was therefore to consider mood measuring in a non-interruptive way. The next two experiments were based around the hypothesis that ―moods can be measured from the keyboard and mouse interaction of the computer user‖. In the first experiment an application was installed on participants‘ computers to record their key presses and mouse clicks in a log file. Their self reported moods in intervals of 20 minutes were also stored in the same file over an average period of eight days. Correlations between participants‘ self reported moods and their keyboard and mouse use revealed that it might be possible to measure moods of the some of the participants. The second experiment took place in the lab, where participants were asked to perform programming like tasks while listening to ii mood inducing background music. Their moods were measured with a Galvanic Skin Response (GSR) meter whereas key presses and mouse clicks again were recorded in log files. The correlations between GSR measurements and keyboard and mouse interaction validated the findings of the experiment in the field that it might be possible to measure the mood of some users from their computer use. Analyzing participants‘ personality traits showed dutifulness and self discipline as indicators that a person‘s mood correlates with his/her interaction behaviour. Considering that mood has an effect on programmer performance and that it might be possible to measure mood in a non-intrusive manner, the last question to focus on was whether a computer-generated intervention could change a programmer‘s mood and consequently improve their performance. In the final experiment programmers had to dry run algorithms for 16 minutes with the expectation that a level of boredom would set in. After this the video clip instructed them to participate in some physical exercises. Participants continued tracing algorithms for 8 minutes after the intervention. Results showed that the mood change after the intervention coincided with a programmers improved ability to provide the correct output of the algorithms. Together these findings lay the foundation for developing an IDE that can measure the programmer mood in a non-intrusive way and make effective interventions to improve programmer performance

Empirical modelling as a new paradigm for educational technology

Educational technology has yet to deliver the benefits or successes that were expected in educational practice, especially in relation to issues other than the communication and delivery of teaching materials. Evidence suggests that these difficulties stem from the mismatch between formalised virtual learning environments and everyday sensemaking and between the rich potential for enhanced learning afforded by new technology and the constraints of old-style educational practice. In addressing this mismatch, some commentators suggest that the primary need is for a new culture of educational practice-and even that such a culture is already emerging, and others identify the need for a new paradigm for educational technology. The aim of this thesis is to explore the potential for a new paradigm for educational technology based on the principles and tools of Empirical Modelling (see http://dcs.warwick.ac.uk/modelling). The thesis builds upon previous research on Empirical Modelling as a constructionist approach to learning, and in particular Roe's doctoral thesis 'Computers for learning: an Empirical Modelling perspective'. Roe's treatment of Empirical Modelling can be viewed as generalising the use of spreadsheets for learning through applying 'programming by dependency' within the framework of existing educational practice. In contrast, this thesis is concerned at a more fundamental level with the contribution that Empirical Modelling can make to technology enhanced learning that may lead to new educational practices. In particular, it identifies eight significant characteristics of learning that are well-matched to Empirical Modelling activity, and associates these with experimental, flexible and meaningful strands in learning. The credentials of Empirical Modelling as a potential new foundation for educational technology are enhanced by demonstrating that Empirical Modelling is radically different from traditional software development and use. It provides a methodology for modelling with dependency that is more closely related to the use of spreadsheets for learning. The thesis elaborates on the relationship between Empirical Modelling and learning in a variety of different contexts, ways and applications. Three examples drawn from computer science higher education are explored to emphasise the experimental, flexible and meaningful characteristics of Empirical Modelling. This discussion of Empirical Modelling in a specific educational context is complemented by an investigation of its relevance to learning in a wider context, with reference to a broad range of subjects, to specific issues in language learning, and to the topics of lifelong learning and collaborative learning. Although the application of Empirical Modelling for learning is as yet too immature for large scale empirical studies, its potential is evaluated using informal empirical evidence arising from Empirical Modelling practice at Warwick. The sources for this evaluation are well-established teaching activities relating to Empirical Modelling in Computer Science at the University of 'Warwick, comprising an introductory module and a number of final year undergraduate projects. The thesis concludes by considering the extent to which Empirical Modelling can go beyond the support for constructionism envisaged by Roe, to address the broader agenda of supporting constructivist learning using computers. To this end, a close relationship between Empirical Modelling and a vision of constructivism recently set out by Bruno Latour in his paper 'The Promises of Constructivism' is demonstrated

Reverse Engineering Heterogeneous Applications

Nowadays a large majority of software systems are built using various technologies that in turn rely on different languages (e.g. Java, XML, SQL etc.). We call such systems heterogeneous applications (HAs). By contrast, we call software systems that are written in one language homogeneous applications. In HAs the information regarding the structure and the behaviour of the system is spread across various components and languages and the interactions between different application elements could be hidden. In this context applying existing reverse engineering and quality assurance techniques developed for homogeneous applications is not enough. These techniques have been created to measure quality or provide information about one aspect of the system and they cannot grasp the complexity of HAs. In this dissertation we present our approach to support the analysis and evolution of HAs based on: (1) a unified first-class description of HAs and, (2) a meta-model that reifies the concept of horizontal and vertical dependencies between application elements at different levels of abstraction. We implemented our approach in two tools, MooseEE and Carrack. The first is an extension of the Moose platform for software and data analysis and contains our unified meta-model for HAs. The latter is an engine to infer derived dependencies that can support the analysis of associations among the heterogeneous elements composing HA. We validate our approach and tools by case studies on industrial and open-source JEAs which demonstrate how we can handle the complexity of such applications and how we can solve problems deriving from their heterogeneous nature

Challenges of rapid migration to fully virtual education in the age of the Corona virus pandemic: experiences from across the world

The social disruption caused by the sudden eruption of the Corona Virus pandemic has shaken the whole world, influencing all levels of education immensely. Notwithstanding there was a lack of preparedness for this global public health emergency which continues to affect all aspects of work and life. The problem is, naturally, multifaceted, fast evolving and complex, affecting everyone, threatening our well-being, the global economy, the environment and all societal and cultural norms and our everyday activities. In a recent UNESCO report it is noted that nearly a billion and a quarter (which is 67,7 % of the total number) of learners have been affected by the Corona Virus pandemic worldwide. The education sector at all levels has been one of the hardest hit sectors particularly as the academic/school year was in full swing. The impact of the pandemic is widespread, representing a health hazard worldwide. Being such, it profoundly affects society as a whole, and its members that are, in particular, i) individuals (the learners, their parents, educators, support staff), ii) schools, training organisations, pedagogical institutions and education systems, iii) quickly transformed policies, methods and pedagogies to serve the newly appeared needs of the latter. Lengthy developments of such scale usually take years of consultation, strategic planning and implementation. In addition to raising awareness across the population of the dangers of the virus transmission and instigating total lockdown, it has been necessary to develop mechanisms for continuing the delivery of education as well as demanding mechanisms for assuring the quality of the educational experience and educational results. There is often scepticism about securing quality standards in such a fast moving situation. Often in the recent past, the perception was that courses and degrees leading to an award are inferior if the course modules (and sometimes its assessment components) were wholly online. Over the last three decades most Higher Education institutions developed both considerable infrastructure and knowhow enabling distance mode delivery schools (Primary and Secondary) had hardly any necessary infrastructure nor adequate knowhow for enabling virtual education. In addition, community education and various training providers were mainly delivered face-to-face and that had to either stop altogether or rapidly convert materials, exercises and tests for online delivery and testing. A high degree of flexibility and commitment was demanded of all involved and particularly from the educators, who undertook to produce new educational materials in order to provide online support to pupils and students. Apart from the delivery mode of education, which is serving for certificated programmes, it is essential to ensure that learners’ needs are thoroughly and continuously addressed and are efficiently supported throughout the Coronavirus or any other future lockdown. The latter can be originated by various causes and reasons that vary in nature, such as natural or socioeconomical. Readiness, thus, in addition to preparedness, is the primary key question and solution when it comes to quality education for any lockdown. In most countries, the compulsory primary and secondary education sectors have been facing a more difficult challenge than that faced by Higher Education. The poor or in many cases non-existent technological infrastructure and low technological expertise of the teachers, instructors and parents, make the delivery of virtual education difficult or even impossible. The latter, coupled with phenomena such as social exclusion and digital divide where thousands of households do not have adequate access to broadband Internet, Wi-Fi infrastructure and personal computers hamper the promising and strenuous virtual solutions. The shockwaves of the sudden demands on all sectors of society and on individuals required rapid decisions and actions. We will not attempt to answer the question “Why was the world unprepared for the onslaught of the Coronavirus pandemic” but need to ascertain the level of preparedness and readiness particularly of the education sector, to effect the required rapid transition. We aimed to identify the challenges, and problems faced by the educators and their institutions. Through first-hand experiences we also identify best practices and solutions reached. Thus we constructed a questionnaire to gather our own responses but also experiences from colleagues and members of our environment, family, friends, and colleagues. This paper reports the first-hand experiences and knowledge of 33 co-authors from 27 institutions and from 13 different countries from Europe, Asia, and Africa. The communication technologies and development platforms used are identified; the challenges faced as well as solutions and best practices are reported. The findings are consolidated into the four areas explored i.e. Development Platforms, Communications Technologies, Challenges/Problems and Solutions/Best Practices. The conclusion summarises the findings into emerging themes and similarities. Reflections on the lasting impact of the effect of Coronavirus on education, limitations of study, and indications of future work complete the paper

Empirical modelling as a new paradigm for educational technology

Educational technology has yet to deliver the benefits or successes that were expected in educational practice, especially in relation to issues other than the communication and delivery of teaching materials. Evidence suggests that these difficulties stem from the mismatch between formalised virtual learning environments and everyday sensemaking and between the rich potential for enhanced learning afforded by new technology and the constraints of old-style educational practice. In addressing this mismatch, some commentators suggest that the primary need is for a new culture of educational practice-and even that such a culture is already emerging, and others identify the need for a new paradigm for educational technology. The aim of this thesis is to explore the potential for a new paradigm for educational technology based on the principles and tools of Empirical Modelling (see http://dcs.warwick.ac.uk/modelling). The thesis builds upon previous research on Empirical Modelling as a constructionist approach to learning, and in particular Roe's doctoral thesis 'Computers for learning: an Empirical Modelling perspective'. Roe's treatment of Empirical Modelling can be viewed as generalising the use of spreadsheets for learning through applying 'programming by dependency' within the framework of existing educational practice. In contrast, this thesis is concerned at a more fundamental level with the contribution that Empirical Modelling can make to technology enhanced learning that may lead to new educational practices. In particular, it identifies eight significant characteristics of learning that are well-matched to Empirical Modelling activity, and associates these with experimental, flexible and meaningful strands in learning. The credentials of Empirical Modelling as a potential new foundation for educational technology are enhanced by demonstrating that Empirical Modelling is radically different from traditional software development and use. It provides a methodology for modelling with dependency that is more closely related to the use of spreadsheets for learning. The thesis elaborates on the relationship between Empirical Modelling and learning in a variety of different contexts, ways and applications. Three examples drawn from computer science higher education are explored to emphasise the experimental, flexible and meaningful characteristics of Empirical Modelling. This discussion of Empirical Modelling in a specific educational context is complemented by an investigation of its relevance to learning in a wider context, with reference to a broad range of subjects, to specific issues in language learning, and to the topics of lifelong learning and collaborative learning. Although the application of Empirical Modelling for learning is as yet too immature for large scale empirical studies, its potential is evaluated using informal empirical evidence arising from Empirical Modelling practice at Warwick. The sources for this evaluation are well-established teaching activities relating to Empirical Modelling in Computer Science at the University of 'Warwick, comprising an introductory module and a number of final year undergraduate projects. The thesis concludes by considering the extent to which Empirical Modelling can go beyond the support for constructionism envisaged by Roe, to address the broader agenda of supporting constructivist learning using computers. To this end, a close relationship between Empirical Modelling and a vision of constructivism recently set out by Bruno Latour in his paper 'The Promises of Constructivism' is demonstrated.EThOS - Electronic Theses Online ServiceGBUnited Kingdo