The concept of active learning and the measurement of learning outcomes: A review of research in engineering higher education

Active learning has gained growing political, instructional, and research interest. However, the definitions of active learning are wide. The learning outcomes related to it have been mostly positive but the measurement methods are not without problems. This review provides an overview of active learning, especially in the context of engineering higher education, by answering two research questions: (1) How is the concept of active learning defined and justified in engineering higher education research? (2) What are the learning outcomes connected to active learning and how is learning measured in engineering higher education research? Sixty-six empirical articles were analyzed inductively with qualitative content analysis. The analysis showed that active learning was defined in various ways, and in some articles, it was not defined at all. In addition, justification (theoretical or empirical) for the use of active learning was seldomly reported. Finally, the indicators used to measure the impact of active learning on students’ learning outcomes were mostly based on students’ self-report data and focused on course specific development in subject-related knowledge. More thorough descriptions and theoretical justifications, as well as the consideration of learning outcomes with appropriate research methods, could reinforce the transparency of empirical interventions and the application of active learning. © 2019 by the authors. Licensee MDPI, Basel, Switzerland.Peer reviewe

Experiment with Peer Instruction in Computer Science to Enhance Class Attendance

Class attendance of computer science courses in higher education is typically not overwhelming. Anecdotal reports and the authors’ experiences with a low-resource mode of peer instruction indicated increased class attendance after a lecture with such concept tests. This has been evaluated systematically with a 3rd-year computer science module using a medium-resource, software-based, Audience Response System (‘clickers’). Results show there is neither a positive nor a negative relation between lectures with peer instruction (PI) and class attendance. The student participation rate in software-based voting decreased and some decline in lecture attendance was observed. Thus, PI itself could not be shown to be a useful strategy to enhance class attendance. Notwithstanding, the students’ evaluation of the use of PI was a moderately positive

Review of Measurements Used in Computing Education Research and Suggestions for Increasing Standardization

The variables that researchers measure and how they measure them are central in any area of research. Which research questions can be asked and how they are answered depends on measurement. This paper describes a systematic review of the literature in computing education research to summarize the commonly used variables and measurements in 197 papers and to compare them to best practices in measurement for human-subjects research. Characteristics of the literature that are examined in the review include variables measured (including learner characteristics), measurements used, and type of data analysis. The review illuminates common practices related to each of these characteristics and their interactions with other characteristics. The paper lists standardized measurements that were used in the literature and highlights commonly used variables for which no standardized measures exist. To conclude, this review compares common practice in computing education to best practices in human-subjects research to make recommendations for increasing rigor

Assessment Of Two Pedagogical Tools For Cybersecurity Education

Cybersecurity is an important strategic areas of computer science, and a difficult discipline to teach effectively. To enhance and provide effective teaching and meaningful learning, we develop and assess two pedagogical tools: Peer instruction, and Concept Maps. Peer instruction teaching methodology has shown promising results in core computer science courses by reducing failure rates and improving student retention in computer science major. Concept maps are well-known technique for improving student-learning experience in class. This thesis document presents the results of implementing and evaluating the peer instruction in a semester-long cybersecurity course, i.e., introduction to computer security. Development and evaluation of concept maps for two cybersecurity courses: SCADA security systems, and digital forensics. We assess the quality of the concept maps using two well-defined techniques: Waterloo rubric, and topological scoring. Results clearly shows that overall concept maps are of high-quality and there is significant improvement in student learning gain during group-discussion

Development of Peer Instruction Material for a Cybersecurity Curriculum

Cybersecurity classes focus on building practical skills alongside the development of the open mindset that is essential to tackle the dynamic cybersecurity landscape. Unfortunately, traditional lecture-style teaching is insufficient for this task. Peer instruction is a non-traditional, active learning approach that has proven to be effective in computer science courses. The challenge in adopting peer instruction is the development of conceptual questions. This thesis presents a methodology for developing peer instruction questions for cybersecurity courses, consisting of four stages: concept identification, concept trigger, question presentation, and development. The thesis analyzes 279 questions developed over two years for three cybersecurity courses: introduction to computer security, network penetration testing, and introduction to computer forensics. Additionally, it discusses examples of peer instruction questions in terms of the methodology. Finally, it summarizes the usage of a workshop for testing a selection of peer instruction questions as well as gathering data outside of normal courses

Breakout group allocation schedules and the social golfer problem with adjacent group sizes

The current pandemic has led schools and universities to turn to online meeting software solutions such as Zoom and Microsoft Teams. The teaching experience can be enhanced via the use of breakout rooms for small group interaction. Over the course of a class (or over several classes), the class will be allocated to breakout groups multiple times over several rounds. It is desirable to mix the groups as much as possible, the ideal being that no two students appear in the same group in more than one round. In this paper, we discuss how the problem of scheduling balanced allocations of students to sequential breakout rooms directly corresponds to a novel variation of a well-known problem in combinatorics (the social golfer problem), which we call the social golfer problem with adjacent group sizes. We explain how solutions to this problem can be obtained using constructions from combinatorial design theory and how they can be used to obtain good, balanced breakout room allocation schedules. We present our solutions for up to 50 students and introduce an online resource that educators can access to immediately generate suitable allocation schedules

Metodologias interativas de ensino na formação de professores de física: um estudo de caso com o método instrução pelos colegas (Peer Instruction)

O presente estudo de caso, conduzido com dois licenciandos em Física que aplicaram o método ativo de ensino Instrução pelos Colegas (IpC) nos seus estágios de docência, visa investigar conflitos e dificuldades experimentadas, vantagens e desvantagens percebidas e como influenciaram suas atitudes em relação ao IpC. Os principais conflitos e dificuldades enfrentados pelos graduandos relacionaram-se ao comportamento dos alunos durante as votações inerentes ao IpC e à escolha dos testes conceituais. As respostas dos estudantes aos testes conceituais não foram efetivamente usadas como feedback pelos estagiários. Quanto às vantagens apontadas pelos estagiários, destacam-se a discussão entre os estudantes sobre tópicos de Física e o engajamento desses em sua própria aprendizagem; em relação às desvantagens, a necessidade de tempo para o aluno refletir sobre os testes conceituais e a limitação do tempo de aula para resolução de problemas. Apesar dos conflitos vivenciados, os estagiários desenvolveram atitudes positivas em relação ao IpC

Utilizing educational technology in computer science and programming courses : theory and practice

There is one thing the Computer Science Education researchers seem to agree: programming is a difficult skill to learn. Educational technology can potentially solve a number of difficulties associated with programming and computer science education by automating assessment, providing immediate feedback and by gamifying the learning process. Still, there are two very important issues to solve regarding the use of technology: what tools to use, and how to apply them? In this thesis, I present a model for successfully adapting educational technology to computer science and programming courses. The model is based on several years of studies conducted while developing and utilizing an exercise-based educational tool in various courses. The focus of the model is in improving student performance, measured by two easily quantifiable factors: the pass rate of the course and the average grade obtained from the course. The final model consists of five features that need to be considered in order to adapt technology effectively into a computer science course: active learning and continuous assessment, heterogeneous exercise types, electronic examination, tutorial-based learning, and continuous feedback cycle. Additionally, I recommend that student mentoring is provided and cognitive load of adapting the tools considered when applying the model. The features are classified as core components, supportive components or evaluation components based on their role in the complete model. Based on the results, it seems that adapting the complete model can increase the pass rate statistically significantly and provide higher grades when compared with a “traditional” programming course. The results also indicate that although adapting the model partially can create some improvements to the performance, all features are required for the full effect to take place. Naturally, there are some limits in the model. First, I do not consider it as the only possible model for adapting educational technology into programming or computer science courses. Second, there are various other factors in addition to students’ performance for creating a satisfying learning experience that need to be considered when refactoring courses. Still, the model presented can provide significantly better results, and as such, it works as a base for future improvements in computer science education.Ohjelmoinnin oppimisen vaikeus on yksi harvoja asioita, joista lähes kaikki tietojenkäsittelyn opetuksen tutkijat ovat jokseenkin yksimielisiä. Opetusteknologian avulla on mahdollista ratkaista useita ohjelmoinnin oppimiseen liittyviä ongelmia esimerkiksi hyödyntämällä automaattista arviointia, välitöntä palautetta ja pelillisyyttä. Teknologiaan liittyy kuitenkin kaksi olennaista kysymystä: mitä työkaluja käyttää ja miten ottaa ne kursseilla tehokkaasti käyttöön? Tässä väitöskirjassa esitellään malli opetusteknologian tehokkaaseen hyödyntämiseen tietojenkäsittelyn ja ohjelmoinnin kursseilla. Malli perustuu tehtäväpohjaisen oppimisjärjestelmän runsaan vuosikymmenen pituiseen kehitys- ja tutkimusprosessiin. Mallin painopiste on opiskelijoiden suoriutumisen parantamisessa. Tätä arvioidaan kahdella kvantitatiivisella mittarilla: kurssin läpäisyprosentilla ja arvosanojen keskiarvolla. Malli koostuu viidestä tekijästä, jotka on otettava huomioon tuotaessa opetusteknologiaa ohjelmoinnin kursseille. Näitä ovat aktiivinen oppiminen ja jatkuva arviointi, heterogeeniset tehtävätyypit, sähköinen tentti, tutoriaalipohjainen oppiminen sekä jatkuva palautesykli. Lisäksi opiskelijamentoroinnin järjestäminen kursseilla ja järjestelmän käyttöönottoon liittyvän kognitiivisen kuorman arviointi tukevat mallin käyttöä. Malliin liittyvät tekijät on tässä työssä lajiteltu kolmeen kategoriaan: ydinkomponentteihin, tukikomponentteihin ja arviontiin liittyviin komponentteihin. Tulosten perusteella vaikuttaa siltä, että mallin käyttöönotto parantaa kurssien läpäisyprosenttia tilastollisesti merkittävästi ja nostaa arvosanojen keskiarvoa ”perinteiseen” kurssimalliin verrattuna. Vaikka mallin yksittäistenkin ominaisuuksien käyttöönotto voi sinällään parantaa kurssin tuloksia, väitöskirjaan kuuluvien tutkimusten perusteella näyttää siltä, että parhaat tulokset saavutetaan ottamalla malli käyttöön kokonaisuudessaan. On selvää, että malli ei ratkaise kaikkia opetusteknologian käyttöönottoon liittyviä kysymyksiä. Ensinnäkään esitetyn mallin ei ole tarkoituskaan olla ainoa mahdollinen tapa hyödyntää opetusteknologiaa ohjelmoinnin ja tietojenkäsittelyn kursseilla. Toiseksi tyydyttävään oppimiskokemukseen liittyy opiskelijoiden suoriutumisen lisäksi paljon muitakin tekijöitä, jotka tulee huomioida kurssien uudelleensuunnittelussa. Esitetty malli mahdollistaa kuitenkin merkittävästi parempien tulosten saavuttamisen kursseilla ja tarjoaa sellaisena perustan entistä parempaan opetukseen