Utilizing educational technology in computer science and programming courses : theory and practice

There is one thing the Computer Science Education researchers seem to agree: programming is a difficult skill to learn. Educational technology can potentially solve a number of difficulties associated with programming and computer science education by automating assessment, providing immediate feedback and by gamifying the learning process. Still, there are two very important issues to solve regarding the use of technology: what tools to use, and how to apply them? In this thesis, I present a model for successfully adapting educational technology to computer science and programming courses. The model is based on several years of studies conducted while developing and utilizing an exercise-based educational tool in various courses. The focus of the model is in improving student performance, measured by two easily quantifiable factors: the pass rate of the course and the average grade obtained from the course. The final model consists of five features that need to be considered in order to adapt technology effectively into a computer science course: active learning and continuous assessment, heterogeneous exercise types, electronic examination, tutorial-based learning, and continuous feedback cycle. Additionally, I recommend that student mentoring is provided and cognitive load of adapting the tools considered when applying the model. The features are classified as core components, supportive components or evaluation components based on their role in the complete model. Based on the results, it seems that adapting the complete model can increase the pass rate statistically significantly and provide higher grades when compared with a “traditional” programming course. The results also indicate that although adapting the model partially can create some improvements to the performance, all features are required for the full effect to take place. Naturally, there are some limits in the model. First, I do not consider it as the only possible model for adapting educational technology into programming or computer science courses. Second, there are various other factors in addition to students’ performance for creating a satisfying learning experience that need to be considered when refactoring courses. Still, the model presented can provide significantly better results, and as such, it works as a base for future improvements in computer science education.Ohjelmoinnin oppimisen vaikeus on yksi harvoja asioita, joista lähes kaikki tietojenkäsittelyn opetuksen tutkijat ovat jokseenkin yksimielisiä. Opetusteknologian avulla on mahdollista ratkaista useita ohjelmoinnin oppimiseen liittyviä ongelmia esimerkiksi hyödyntämällä automaattista arviointia, välitöntä palautetta ja pelillisyyttä. Teknologiaan liittyy kuitenkin kaksi olennaista kysymystä: mitä työkaluja käyttää ja miten ottaa ne kursseilla tehokkaasti käyttöön? Tässä väitöskirjassa esitellään malli opetusteknologian tehokkaaseen hyödyntämiseen tietojenkäsittelyn ja ohjelmoinnin kursseilla. Malli perustuu tehtäväpohjaisen oppimisjärjestelmän runsaan vuosikymmenen pituiseen kehitys- ja tutkimusprosessiin. Mallin painopiste on opiskelijoiden suoriutumisen parantamisessa. Tätä arvioidaan kahdella kvantitatiivisella mittarilla: kurssin läpäisyprosentilla ja arvosanojen keskiarvolla. Malli koostuu viidestä tekijästä, jotka on otettava huomioon tuotaessa opetusteknologiaa ohjelmoinnin kursseille. Näitä ovat aktiivinen oppiminen ja jatkuva arviointi, heterogeeniset tehtävätyypit, sähköinen tentti, tutoriaalipohjainen oppiminen sekä jatkuva palautesykli. Lisäksi opiskelijamentoroinnin järjestäminen kursseilla ja järjestelmän käyttöönottoon liittyvän kognitiivisen kuorman arviointi tukevat mallin käyttöä. Malliin liittyvät tekijät on tässä työssä lajiteltu kolmeen kategoriaan: ydinkomponentteihin, tukikomponentteihin ja arviontiin liittyviin komponentteihin. Tulosten perusteella vaikuttaa siltä, että mallin käyttöönotto parantaa kurssien läpäisyprosenttia tilastollisesti merkittävästi ja nostaa arvosanojen keskiarvoa ”perinteiseen” kurssimalliin verrattuna. Vaikka mallin yksittäistenkin ominaisuuksien käyttöönotto voi sinällään parantaa kurssin tuloksia, väitöskirjaan kuuluvien tutkimusten perusteella näyttää siltä, että parhaat tulokset saavutetaan ottamalla malli käyttöön kokonaisuudessaan. On selvää, että malli ei ratkaise kaikkia opetusteknologian käyttöönottoon liittyviä kysymyksiä. Ensinnäkään esitetyn mallin ei ole tarkoituskaan olla ainoa mahdollinen tapa hyödyntää opetusteknologiaa ohjelmoinnin ja tietojenkäsittelyn kursseilla. Toiseksi tyydyttävään oppimiskokemukseen liittyy opiskelijoiden suoriutumisen lisäksi paljon muitakin tekijöitä, jotka tulee huomioida kurssien uudelleensuunnittelussa. Esitetty malli mahdollistaa kuitenkin merkittävästi parempien tulosten saavuttamisen kursseilla ja tarjoaa sellaisena perustan entistä parempaan opetukseen

Evolutionary design of deep neural networks

Mención Internacional en el título de doctorFor three decades, neuroevolution has applied evolutionary computation to the optimization of the topology of artificial neural networks, with most works focusing on very simple architectures. However, times have changed, and nowadays convolutional neural networks are the industry and academia standard for solving a variety of problems, many of which remained unsolved before the discovery of this kind of networks. Convolutional neural networks involve complex topologies, and the manual design of these topologies for solving a problem at hand is expensive and inefficient. In this thesis, our aim is to use neuroevolution in order to evolve the architecture of convolutional neural networks. To do so, we have decided to try two different techniques: genetic algorithms and grammatical evolution. We have implemented a niching scheme for preserving the genetic diversity, in order to ease the construction of ensembles of neural networks. These techniques have been validated against the MNIST database for handwritten digit recognition, achieving a test error rate of 0.28%, and the OPPORTUNITY data set for human activity recognition, attaining an F1 score of 0.9275. Both results have proven very competitive when compared with the state of the art. Also, in all cases, ensembles have proven to perform better than individual models. Later, the topologies learned for MNIST were tested on EMNIST, a database recently introduced in 2017, which includes more samples and a set of letters for character recognition. Results have shown that the topologies optimized for MNIST perform well on EMNIST, proving that architectures can be reused across domains with similar characteristics. In summary, neuroevolution is an effective approach for automatically designing topologies for convolutional neural networks. However, it still remains as an unexplored field due to hardware limitations. Current advances, however, should constitute the fuel that empowers the emergence of this field, and further research should start as of today.This Ph.D. dissertation has been partially supported by the Spanish Ministry of Education, Culture and Sports under FPU fellowship with identifier FPU13/03917. This research stay has been partially co-funded by the Spanish Ministry of Education, Culture and Sports under FPU short stay grant with identifier EST15/00260.Programa Oficial de Doctorado en Ciencia y Tecnología InformáticaPresidente: María Araceli Sanchís de Miguel.- Secretario: Francisco Javier Segovia Pérez.- Vocal: Simon Luca

The Design and Implementation of a Notional Machine for teaching Introductory Programming

Comprehension of both programming and programs is a difficult task for novices to master, with many university courses that feature a programming component demonstrating significant failure and drop out rates. Many theories exist that attempt to explain why this is the case. One such theory, originally postulated by du Boulay, is that students do not understand the properties of the machine; they do not understand what they are or how they are controlling them by writing code. This idea formed the development of the notional machine, which exists solely as an abstraction of the physical machine to aid with its understanding and comprehension. This work contributes a design for a new notional machine and a graphical notation for its representation. The notional machine is designed to work with object-oriented languages (in particular Java). It provides several novel contributions over pre-existing models -- while existing similar models are generally constrained to line by line operation, the notional machine presented here can scale effectively across many program sizes, from few objects and lines to many. In addition, it is able to be used in a variety of formats (in both electronic and unplugged form). It also melds together three traditionally separate diagrams that had to be understood simultaneously (the stack trace, class diagram and object heap.) Novis, an implemented version of the notional machine, is also presented and evaluated. It is able to create automatic and animated versions of notional machine diagrams, and has been integrated into BlueJ's main interface. Novis can present static notional machine diagrams at selected stages of program execution, or animate ongoing execution in real time. The evaluation of Novis is presented in two parts. It is first tested alongside a selection of methodically chosen textbook examples to ensure it can visualise a range of useful programs, and it then undergoes usability testing with a group of first year computer science students

The Stained Glass of Knowledge: On Understanding Novice Mental Models of Computing

Learning to program can be a novel experience. The rigidity of programming can be at odds with beginning programmer\u27s existing perceptions, and the concepts can feel entirely unfamiliar. These observations motivated this research, which explores two major questions: What factors influence how novices learn programming? and How can analogy by more appropriately leveraged in programming education? This dissertation investigates the factors influencing novice programming through multiple methods. The CS1 classroom is observed as a whole system , with consideration to the factors present in it that can influence the learning process. Learning\u27s cognitive processes are elaborated to ground exploration into specifically learning programming. This includes extensive literature review spanning multiple disciplines. This allows positioning to guide the investigation. The literature survey also contributes to greater understanding of learning cognition within computing education research through its disciplinary depth. The focus on analogy with the second question is motivated through the factors observed in the first question. Analogy\u27s role in cognition and in education is observed, and the analogical inclinations of technology as a field are showcased. Stigma surrounds the use of analogy in computer science education in spite of these indications. This motivated investigation on how the use of analogy could be better addressed in programming education in order to utilize its value. This research presents a tool for the design of well-formed analogy in programming to answer this question. It also investigates additional forms analogy can take in the classroom setting, proposing relevant cultural forms such as memes can be analogical vehicles that promote learner engagement. This research presents a strong case for the value of analogy use in the CS1 classroom, and provides a tool to facilitate the design of well-formed analogies. In identifying ways to better leverage analogy in the programming classroom, presenting this research will hopefully contribute to dispelling analogy\u27s bad reputation in computing education. By exploring factors that contribute to the learning process in CS1, this research frames education design as experience design. This motivates methods and considerations from user experience design, and investigates aspects of the whole system that can promote or deter a learner\u27s experience. This dissertation presents findings on understanding the learner\u27s experience in the programming classroom, and how analogy can be used to benefit their learning process

Techniques for the reverse engineering of banking malware

Malware attacks are a significant and frequently reported problem, adversely affecting the productivity of organisations and governments worldwide. The well-documented consequences of malware attacks include financial loss, data loss, reputation damage, infrastructure damage, theft of intellectual property, compromise of commercial negotiations, and national security risks. Mitiga-tion activities involve a significant amount of manual analysis. Therefore, there is a need for automated techniques for malware analysis to identify malicious behaviours. Research into automated techniques for malware analysis covers a wide range of activities. This thesis consists of a series of studies: an anal-ysis of banking malware families and their common behaviours, an emulated command and control environment for dynamic malware analysis, a technique to identify similar malware functions, and a technique for the detection of ransomware. An analysis of the nature of banking malware, its major malware families, behaviours, variants, and inter-relationships are provided in this thesis. In doing this, this research takes a broad view of malware analysis, starting with the implementation of the malicious behaviours through to detailed analysis using machine learning. The broad approach taken in this thesis differs from some other studies that approach malware research in a more abstract sense. A disadvantage of approaching malware research without domain knowledge, is that important methodology questions may not be considered. Large datasets of historical malware samples are available for countermea-sures research. However, due to the age of these samples, the original malware infrastructure is no longer available, often restricting malware operations to initialisation functions only. To address this absence, an emulated command and control environment is provided. This emulated environment provides full control of the malware, enabling the capabilities of the original in-the-wild operation, while enabling feature extraction for research purposes. A major focus of this thesis has been the development of a machine learn-ing function similarity method with a novel feature encoding that increases feature strength. This research develops techniques to demonstrate that the machine learning model trained on similarity features from one program can find similar functions in another, unrelated program. This finding can lead to the development of generic similar function classifiers that can be packaged and distributed in reverse engineering tools such as IDA Pro and Ghidra. Further, this research examines the use of API call features for the identi-fication of ransomware and shows that a failure to consider malware analysis domain knowledge can lead to weaknesses in experimental design. In this case, we show that existing research has difficulty in discriminating between ransomware and benign cryptographic software. This thesis by publication, has developed techniques to advance the disci-pline of malware reverse engineering, in order to minimize harm due to cyber-attacks on critical infrastructure, government institutions, and industry.Doctor of Philosoph

Enhancing comprehension in open distance learning computer programming education with visualization

This thesis describes a research project aimed at improving the tracing skills of first-year programming students enrolled for an introductory C++ course at an open distance learning institution by means of a tutorial in the form of a program visualization tool to teach the students to draw variable diagrams. The research was based on the findings from the BRACElet project (Clear, Whalley, Robbins, Philpott, Eckerdal, Laakso & Lister, 2011). A design-based research methodology was followed. To guide the process of developing the tutorial, a framework of 26 guidelines for developing and using visualization tools to teach programming was synthesized from the literature on computing education research CER, educational psychology and computer graphics. Guidelines were supplemented with reasons or explanations for their recommendation and considerations to be taken into account when using a guideline. The framework was enhanced by lessons learnt during the development and testing of the tutorial. The tutorial was tested and refined during two implementation cycles. Both cycles included quantitative and qualitative investigations. All students registered for the introductory module received the tool with their study material. For the quantitative investigations, students completed a questionnaire after using the tutorial. Through the questionnaire biographical data was acquired, the manner in which students used the tutorial and how they experienced using it. The responses to the questionnaires were statistically analysed in combination with respondents’ final marks. The statistical modelling indicated that the students’ biographical properties (a combination of level of programming experience, marks obtained for Mathematics and English in matric and first-time registration for COS1511 or not), had the biggest impact on their final marks by far. During the qualitative investigations students were eye tracked in a Human-Computer Interaction laboratory. The gaze replays in both cycles revealed that students’ reading skills impacted largely on their success, connecting with the findings from the quantitative investigations. Reflections on why the tutorial did not achieve its purpose; and why poor reading skills may have such a strong effect on learning to program, contribute some theoretical understanding as to how novices learn to program.Computer ScienceD. Phil. (Computer Science

Transforming the Culture of the Stem Disciplines: A Multiple Case Study of Successful Strategies for Inclusive Excellence

In spite of the myriad initiatives promoting diversity and inclusion in science, technology, engineering, and math (STEM) disciplines at colleges and universities across the country and the increased funding for these types of initiatives, colleges and universities have experienced modest gains and, in some cases, slight declines in the graduation rates of women and underrepresented minorities (URMs). In addressing this issue, some institutions rely on traditional or “deficit” approaches that place primary responsibility on students for their success in the STEM fields. A growing number of institutions, however, embrace the concept of inclusive excellence, which is an assumption that underrepresented students can and will rise to high expectations provided institutions have a better understanding of and commitment to fostering the conditions in which these students thrive. Although there is much in the literature about successful programs and initiatives in the STEM disciplines that are aligned with the concept of inclusive excellence, there is little information about the organizational change processes that ensure that these initiatives are implemented on a scale large enough to significantly impact the enrollment and graduation rates of women and URMs. This study addressed the void in the literature through a multiple case study of three departments/schools at elite Predominantly White Institutions that have had success with enrolling and graduating women and underrepresented minorities (URMs) at rates above the national average – the computer science department at Harvey Mudd College, the physics department at the Massachusetts Institute of Technology, and the Cockrell School of Engineering at the University of Texas Austin. Employing qualitative content analysis as the methodology, 246 documents from these institutions were analyzed both deductively with the Transformational Change Model for Inclusive Excellence as a framework and inductively to determine the strategies that advanced initiatives in the model departments and school. Analysis revealed that each department/school utilized, to some degree, strategies from the framework, including understanding and articulating the rationale for change, securing senior administrator buy-in, collaborating across multiple departments and disciplines, providing professional development for those associated with initiatives, focusing attention on the environment for underrepresented students, developing a vision for the department/school that was flexible and modified when needed, engaging in high impact and visible actions that were staged over time, and conducting comprehensive assessment and evaluation. Beyond the framework, analysis revealed that campus culture, external funding, and selective admissions also played a role in advancing initiatives on a large scale, as did other factors that were specific to each institution

Pasifika Tertiary Students' Use of Information and Communication Technologies

Information and Communication Technologies (ICT) are pervasive in our daily lives. In New Zealand tertiary education institutions, the adoption of ICT is widespread. Meanwhile, Pasifika students’ participation in tertiary education has been increasing. Yet, for this group of students, academic success has been a concern to successive governments, tertiary institutions, and Pasifika peoples. ICT may offer an opportunity to improve Pasifika students’ academic achievement. The study is premised on the belief that positive learning experiences will lead to improved academic outcomes. Consequently, the study explored ways of using ICT to enhance Pasifika students’ learning experiences. Adopting an interpretivist approach, the case study investigated the ICT skills and the use of ICT for learning enhancements of a group of Pasifika students at a New Zealand institute of technology. Over a period of eighteen months, data was gathered through talanoa, participant observation, and the researcher’s reflective journal. The three method approach enabled data triangulation. Data analysis adhered to the theoretical propositions of the study. Among the key findings, is the disturbing realization that Pasifika students’ ICT skill levels were not at the level assumed by the institution. The main uses of ICT by the Pasifika students in this study were for personal communication and entertainment. The students’ use of ICT for educational purposes was limited due to the mismatch of their ICT skills and those required by the institution. Inadvertently, this has further disadvantaged the students’ learning experiences. The study concludes by offering an ICT skills development framework for use with Pasifika students. Moreover, the study proposes a number of recommendations for practice, policy, and further research