Predictive models as early warning systems for student academic performance in introductory programming

Computer programming is fundamental to Computer Science and IT curricula. At the novice level it covers programming concepts that are essential for subsequent advanced programming courses. However, introductory programming courses are among the most challenging courses for novices and high failure and attrition rates continue even as computer science education has seen improvements in pedagogy. Consequently, the quest to identify factors that affect student learning and academic performance in introductory computer programming courses has been a long-standing activity. Specifically, weak novice learners of programming need to be identified and assisted early in the semester in order to alleviate any potential risk of failing or withdrawing from their course. Hence, it is essential to identify at-risk programming students early, in order to plan (early) interventions. The goal of this thesis was to develop a validated, predictive model(s) with suitable predictors of student academic performance in introductory programming courses. The proposed model utilises the Naïve Bayes classification machine learning algorithm to analyse student performance data, based on the principle of parsimony. Furthermore, an additional objective was to propose this validated predictive model as an early warning system (EWS), to predict at-risk students early in the semester and, in turn, to potentially inform instructors (and students) for early interventions. We obtained data from two introductory programming courses in our study to develop and test the predictive models. The models were built with student presage and in progress-data for which instructors may easily collect or access despite the nature of pedagogy of educational settings. In addition, our work analysed the predictability of selected data sources and looked for the combination of predictors, which yields the highest prediction accuracy to predict student academic performance. The prediction accuracies of the models were computed by using confusion matrix data including overall model prediction accuracy, prediction accuracy sensitivity and specificity, balanced accuracy and the area under the ROC curve (AUC) score for generalisation. On average, the models developed with formative assessment tasks, which were partially assisted by the instructor in the classroom, returned higher at-risk prediction accuracies than the models developed with take-home assessment task only as predictors. The unknown data test results of this study showed that it is possible to predict 83% of students that need support as early as Week 3 in a 12-week introductory programming course. The ensemble method-based results suggest that it is possible to improve overall at-risk prediction performance with low false positives and to incorporate this in early warning systems to identify students that need support, in order to provide early intervention before they reach critical stages (at-risk of failing). The proposed model(s) of this study were developed on the basis of the principle of parsimony as well as previous research findings, which accounted for variations in academic settings, such as academic environment, and student demography. The predictive model could potentially provide early warning indicators to facilitate early warning intervention strategies for at-risk students in programming that allow for early interventions. The main contribution of this thesis is a model that may be applied to other programming and non-programming courses, which have both continuous formative and a final exam summative assessment, to predict final student performance early in the semester.Ohjelmointi on informaatioteknologian ja tietojenkäsittelytieteen opinto-ohjelmien olennainen osa. Aloittelijatasolla opetus kattaa jatkokurssien kannalta keskeisiä ohjelmoinnin käsitteitä. Tästä huolimatta ohjelmoinnin peruskurssit ovat eräitä haasteellisimmista kursseista aloittelijoille. Korkea keskeyttämisprosentti ja opiskelijoiden asteittainen pois jättäytyminen ovat vieläkin tunnusomaisia piirteitä näille kursseille, vaikka ohjelmoinnin opetuksen pedagogiikka onkin kehittynyt. Näin ollen vaikuttavia syitä opiskelijoiden heikkoon suoriutumiseen on etsitty jo pitkään. Erityisesti heikot, aloittelevat ohjelmoijat tulisi tunnistaa mahdollisimman pian, jotta heille voitaisiin tarjota tukea ja pienentää opiskelijan riskiä epäonnistua kurssin läpäimisessä ja riskiä jättää kurssi kesken. Heikkojen opiskelijoiden tunnistaminen on tärkeää, jotta voidaan suunnitella aikainen väliintulo. Tämän väitöskirjatyön tarkoituksena oli kehittää todennettu, ennustava malli tai malleja sopivilla ennnustusfunktioilla koskien opiskelijan akateemista suoriutumista ohjelmoinnin peruskursseilla. Kehitetty malli käyttää koneoppivaa naiivia bayesilaista luokittelualgoritmia analysoimaan opiskelijoiden suoriutumisesta kertynyttä aineistoa. Lähestymistapa perustuu yksinkertaisimpien mahdollisten selittävien mallien periaatteeseen. Lisäksi, tavoitteena oli ehdottaa tätä validoitua ennustavaa mallia varhaiseksi varoitusjärjestelmäksi, jolla ennustetaan putoamisvaarassa olevat opiskelijat opintojakson alkuvaiheessa sekä informoidaan ohjaajia (ja opiskelijaa) aikaisen väliintulon tarpeellisuudesta. Keräsimme aineistoa kahdelta ohjelmoinnin peruskurssilta, jonka pohjalta ennustavaa mallia kehitettiin ja testattiin. Mallit on rakennettu opiskelijoiden ennakkotietojen ja kurssin kestäessä kerättyjen suoriutumistietojen perusteella, joita ohjaajat voivat helposti kerätä tai joihin he voivat päästä käsiksi oppilaitoksesta tai muusta ympäristöstä huolimatta. Lisäksi väitöskirjatyö analysoi valittujen datalähteiden ennustettavuutta ja sitä, mitkä mallien muuttujista ja niiden kombinaatioista tuottivat kannaltamme korkeimman ennustetarkkuuden opiskelijoiden akateemisessa suoriutumisessa. Mallien ennustusten tarkkuuksia laskettiin käyttämällä sekaannusmatriisia, josta saadaan laskettua ennusteen tarkkuus, ennusteen spesifisyys, sensitiivisyys, tasapainotettu tarkkuus sekä luokitteluvastekäyriä (receiver operating characteristics (ROC)) ja näiden luokitteluvastepinta-ala (area under curve (AUC)) Mallit, jotka kehitettiin formatiivisilla tehtävillä, ja joissa ohjaaja saattoi osittain auttaa luokkahuonetilanteessa, antoivat keskimäärin tarkemman ennustuksen putoamisvaarassa olevista opiskelijoista kuin mallit, joissa käytettiin kotiin vietäviä tehtäviä ainoina ennusteina. Tuntemattomalla testiaineistolla tehdyt mallinnukset osoittavat, että voimme tunnistaa jo 3. viikon kohdalla 83% niistä opiskelijoista, jotka tarvitsevat lisätukea 12 viikkoa kestävällä ohjelmoinnin kurssilla. Tulosten perusteella vaikuttaisi, että yhdistämällä metodeja voidaan saavuttaa parempi yleinen ennustettavuus putoamisvaarassa olevien opiskelijoiden suhteen pienemmällä määrällä väärin luokiteltuja epätositapauksia. Tulokset viittaavat myös siihen, että on mahdollista sisällyttää yhdistelmämalli varoitusjärjestelmiin, jotta voidaan tunnistaa avuntarpeessa olevia opiskelijoita ja tarjota täten varhaisessa vaiheessa tukea ennen kuin on liian myöhäistä. Tässä tutkimuksessa esitellyt mallit on kehitetty nojautuen yksinkertaisimman selittävän mallin periaatteeseen ja myös aiempiin tutkimustuloksiin, joissa huomioidaan erilaiset akateemiset ympäristöt ja opiskelijoiden tausta. Ennustava malli voi tarjota indikaattoreita, jotka voivat mahdollisesti toimia pohjana väliintulostrategioihin kurssilta putoamisvaarassa olevien opiskelijoiden tukemiseksi. Tämän tutkimuksen keskeisin anti on malli, jolla opiskelijoiden suoriutumista voidaan arvioida muilla ohjelmointia ja muita aihepiirejä käsittelevillä kursseilla, jotka sisältävät sekä jatkuvaa arviointia että loppukokeen. Malli ennustaisi näillä kursseilla lopullisen opiskelijan suoritustason opetusjakson alkuvaiheessa

Trajectory-Oriented Approach to Managing Traffic Complexity: Trajectory Flexibility Metrics and Algorithms and Preliminary Complexity Impact Assessment

This document describes exploratory research on a distributed, trajectory oriented approach for traffic complexity management. The approach is to manage traffic complexity based on preserving trajectory flexibility and minimizing constraints. In particular, the document presents metrics for trajectory flexibility; a method for estimating these metrics based on discrete time and degree of freedom assumptions; a planning algorithm using these metrics to preserve flexibility; and preliminary experiments testing the impact of preserving trajectory flexibility on traffic complexity. The document also describes an early demonstration capability of the trajectory flexibility preservation function in the NASA Autonomous Operations Planner (AOP) platform

Initial Investigation of Operational Concept Elements for NASA's NextGen-Airportal Project Research

The NextGen-Airportal Project is organized into three research focus areas: Safe and Efficient Surface Operations, Coordinated Arrival/Departure Operations Management, and Airportal Transition and Integration Management. The content in this document was derived from an examination of constraints and problems at airports for accommodating future increases in air traffic, and from an examination of capabilities envisioned for NextGen. The concepts are organized around categories of constraints and problems and therefore do not precisely match, but generally reflect, the research focus areas. The concepts provide a framework for defining and coordinating research activities that are, and will be, conducted by the NextGen-Airportal Project. The concepts will help the research activities function as an integrated set focused on future needs for airport operations and will aid aligning the research activities with NextGen key capabilities. The concepts are presented as concept elements with more detailed sub-elements under each concept element. For each concept element, the following topics are discussed: constraints and problems being addressed, benefit descriptions, required technology and infrastructure, and an initial list of potential research topics. Concept content will be updated and more detail added as the research progresses. The concepts are focused on enhancing airportal capacity and efficiency in a timeframe 20 to 25 years in the future, which is similar to NextGen's timeframe

Evaluating Network Analysis and Agent Based Modeling for Investigating the Stability of Commercial Air Carrier Schedules

For a number of years, the United States Federal Government has been formulating the Next Generation Air Transportation System plans for National Airspace System improvement. These improvements attempt to address air transportation holistically, but often address individual improvements in one arena such as ground or in-flight equipment. In fact, air transportation system designers have had only limited success using traditional Operations Research and parametric modeling approaches in their analyses of innovative operations. They need a systemic methodology for modeling of safety-critical infrastructure that is comprehensive, objective, and sufficiently concrete, yet simple enough to be deployed with reasonable investment. The methodology must also be amenable to quantitative analysis so issues of system safety and stability can be rigorously addressed

Mathematical analysis of scheduling policies in peer-to-peer video streaming networks

Las redes de pares son comunidades virtuales autogestionadas, desarrolladas en la capa de aplicación sobre la infraestructura de Internet, donde los usuarios (denominados pares) comparten recursos (ancho de banda, memoria, procesamiento) para alcanzar un fin común. La distribución de video representa la aplicación más desafiante, dadas las limitaciones de ancho de banda. Existen básicamente tres servicios de video. El más simple es la descarga, donde un conjunto de servidores posee el contenido original, y los usuarios deben descargar completamente este contenido previo a su reproducción. Un segundo servicio se denomina video bajo demanda, donde los pares se unen a una red virtual siempre que inicien una solicitud de un contenido de video, e inician una descarga progresiva en línea. El último servicio es video en vivo, donde el contenido de video es generado, distribuido y visualizado simultáneamente. En esta tesis se estudian aspectos de diseño para la distribución de video en vivo y bajo demanda. Se presenta un análisis matemático de estabilidad y capacidad de arquitecturas de distribución bajo demanda híbridas, asistidas por pares. Los pares inician descargas concurrentes de múltiples contenidos, y se desconectan cuando lo desean. Se predice la evolución esperada del sistema asumiendo proceso Poisson de arribos y egresos exponenciales, mediante un modelo determinístico de fluidos. Un sub-modelo de descargas secuenciales (no simultáneas) es globalmente y estructuralmente estable, independientemente de los parámetros de la red. Mediante la Ley de Little se determina el tiempo medio de residencia de usuarios en un sistema bajo demanda secuencial estacionario. Se demuestra teóricamente que la filosofía híbrida de cooperación entre pares siempre desempeña mejor que la tecnología pura basada en cliente-servidor

Information-Theoretic Active Perception for Multi-Robot Teams

Multi-robot teams that intelligently gather information have the potential to transform industries as diverse as agriculture, space exploration, mining, environmental monitoring, search and rescue, and construction. Despite large amounts of research effort on active perception problems, there still remain significant challenges. In this thesis, we present a variety of information-theoretic control policies that enable teams of robots to efficiently estimate different quantities of interest. Although these policies are intractable in general, we develop a series of approximations that make them suitable for real time use. We begin by presenting a unified estimation and control scheme based on Shannon\u27s mutual information that lets small teams of robots equipped with range-only sensors track a single static target. By creating approximate representations, we substantially reduce the complexity of this approach, letting the team track a mobile target. We then scale this approach to larger teams that need to localize a large and unknown number of targets. We also examine information-theoretic control policies to autonomously construct 3D maps with ground and aerial robots. By using Cauchy-Schwarz quadratic mutual information, we show substantial computational improvements over similar information-theoretic measures. To map environments faster, we adopt a hierarchical planning approach which incorporates trajectory optimization so that robots can quickly determine feasible and locally optimal trajectories. Finally, we present a high-level planning algorithm that enables heterogeneous robots to cooperatively construct maps

Dissecting Drayage: An Examination of Structure, Information, and Control in Drayage Operations

The term dray dates back to the 14th century when it was used commonly to describe a type of very sturdy sideless cart . In the 1700s the word drayage came into use meaning “to transport by a sideless cart”. Today, drayage commonly refers to the transport of containerized cargo to and from port or rail terminals and inland locations. With the phenomenal growth of containerized freight since the container’s introduction in 1956, the drayage industry has also experienced significant growth. In fact, according to the Bureau for Transportation Statistics, the world saw total maritime container traffic grow to approximately 417 million twenty foot equivalent units (TEUs) in 2006. Unfortunately, the drayage portion of a door-to-door container move tends to be the most costly part of the move. There are a variety of reasons for this disproportionate assignment of costs, including a great deal of uncertainty at the interface of modes. For example, trucks moving containers to and from a port terminal are often uncertain as to how long it will take them to pick up a designated container coming from a ship, from the terminal stack, or from customs. This uncertainty leads to much difficulty and inefficiency in planning a profitable routing for multiple containers in one day. We study this problem from three perspectives using both empirical and theoretical techniques

Integration of evolutionary algorithm in an agent-oriented approach for an adaptive e-learning

This paper describes an agent- oriented approach that aims to create learning situations by solving problems. The proposed system is designed as a multi-agent that organizes interfaces, coordinators, sources of information and mobiles. The objective of this approach is to get learners to solve a problem that leads them to get engaged in several learning activities, chosen according to their level of knowledge and preferences in order to ensure adaptive learning and reduce the rate of learner abundance in an e-learning system. The search for learning activities procedure is based on evolutionary algorithms typically: genetic algorithm, to offer learners the optimal solution adapted to their profiles and ensuring a resolution of the proposed learning problem. In terms of results, we have adopted “immigration strategies” to improve the performance of the genetic algorithm. To show the effectiveness of the proposed approach we have made a comparative study with other artificial intelligence optimization methods. We conducted a real experiment with primary school learners in order to test the effectiveness of the proposed approach and to set up its functioning. The experiment results showed a high rate of success and engagement among the learners who followed the proposed adaptive learning scenario