Faculty Publications and Creative Works 2003

Faculty Publications & Creative Works is an annual compendium of scholarly and creative activities of University of New Mexico faculty during the noted calendar year. It serves to illustrate the robust and active intellectual pursuits conducted by the faculty in support of teaching and research at UNM

Motivational and metacognitive feedback in an ITS: linking past states and experiences to current problems

Feedback is an important element in learning as it can provide learners with both information about progress as well as external motivational stimuli, providing them with an opportunity for reflection. Motivation and metacognition are strongly intertwined, with learners high in self-efficacy more likely to use a variety of self-regulatory learning strategies, as well as to persist longer on challenging tasks. Learning from past experience involves metacognitive processes as an act of reflecting upon one’s own experience and, coupled with existing knowledge, aids the acquisition and construction of further knowledge. The aim of the research was to improve the learner’s focus on the process and experience of problem solving while using an Intelligent Tutoring System (ITS), by addressing the primary question: what are the effects of including motivational and metacognitive feedback based on the learner’s past states and experiences? An existing ITS, SQL-Tutor, was used in a study with participants from first year undergraduate degrees studying a database module. The study used two versions of SQL-Tutor: the Control group used a base version providing domain feedback and the Study group used an extended version that also provided motivational and metacognitive feedback. Three sources of data collection were used: module summative assessments, ITS log files and a post-study questionnaire. The analysis included both pre-post comparisons and how the participants interacted with the system, for example their persistence in problem-solving and the degree to which they referred to past learning. Comparisons between groups showed some differing trends both in learning and behaviour in favour of the Study group, though these trends were not significantly different. The study findings showed promise for the use of motivational and metacognitive feedback based on the learners’ past states and experiences that could be used as a basis for future research work and refinement

Empirical modelling as a new paradigm for educational technology

Educational technology has yet to deliver the benefits or successes that were expected in educational practice, especially in relation to issues other than the communication and delivery of teaching materials. Evidence suggests that these difficulties stem from the mismatch between formalised virtual learning environments and everyday sensemaking and between the rich potential for enhanced learning afforded by new technology and the constraints of old-style educational practice. In addressing this mismatch, some commentators suggest that the primary need is for a new culture of educational practice-and even that such a culture is already emerging, and others identify the need for a new paradigm for educational technology. The aim of this thesis is to explore the potential for a new paradigm for educational technology based on the principles and tools of Empirical Modelling (see http://dcs.warwick.ac.uk/modelling). The thesis builds upon previous research on Empirical Modelling as a constructionist approach to learning, and in particular Roe's doctoral thesis 'Computers for learning: an Empirical Modelling perspective'. Roe's treatment of Empirical Modelling can be viewed as generalising the use of spreadsheets for learning through applying 'programming by dependency' within the framework of existing educational practice. In contrast, this thesis is concerned at a more fundamental level with the contribution that Empirical Modelling can make to technology enhanced learning that may lead to new educational practices. In particular, it identifies eight significant characteristics of learning that are well-matched to Empirical Modelling activity, and associates these with experimental, flexible and meaningful strands in learning. The credentials of Empirical Modelling as a potential new foundation for educational technology are enhanced by demonstrating that Empirical Modelling is radically different from traditional software development and use. It provides a methodology for modelling with dependency that is more closely related to the use of spreadsheets for learning. The thesis elaborates on the relationship between Empirical Modelling and learning in a variety of different contexts, ways and applications. Three examples drawn from computer science higher education are explored to emphasise the experimental, flexible and meaningful characteristics of Empirical Modelling. This discussion of Empirical Modelling in a specific educational context is complemented by an investigation of its relevance to learning in a wider context, with reference to a broad range of subjects, to specific issues in language learning, and to the topics of lifelong learning and collaborative learning. Although the application of Empirical Modelling for learning is as yet too immature for large scale empirical studies, its potential is evaluated using informal empirical evidence arising from Empirical Modelling practice at Warwick. The sources for this evaluation are well-established teaching activities relating to Empirical Modelling in Computer Science at the University of 'Warwick, comprising an introductory module and a number of final year undergraduate projects. The thesis concludes by considering the extent to which Empirical Modelling can go beyond the support for constructionism envisaged by Roe, to address the broader agenda of supporting constructivist learning using computers. To this end, a close relationship between Empirical Modelling and a vision of constructivism recently set out by Bruno Latour in his paper 'The Promises of Constructivism' is demonstrated

A generic approach to the evolution of interaction in ubiquitous systems

This dissertation addresses the challenge of the configuration of modern (ubiquitous, context-sensitive, mobile et al.) interactive systems where it is difficult or impossible to predict (i) the resources available for evolution, (ii) the criteria for judging the success of the evolution, and (iii) the degree to which human judgements must be involved in the evaluation process used to determine the configuration. In this thesis a conceptual model of interactive system configuration over time (known as interaction evolution) is presented which relies upon the follow steps; (i) identification of opportunities for change in a system, (ii) reflection on the available configuration alternatives, (iii) decision-making and (iv) implementation, and finally iteration of the process. This conceptual model underpins the development of a dynamic evolution environment based on a notion of configuration evaluation functions (hereafter referred to as evaluation functions) that provides greater flexibility than current solutions and, when supported by appropriate tools, can provide a richer set of evaluation techniques and features that are difficult or impossible to implement in current systems. Specifically this approach has support for changes to the approach, style or mode of use used for configuration - these features may result in more effective systems, less effort involved to configure them and a greater degree of control may be offered to the user. The contributions of this work include; (i) establishing the the need for configuration evolution through a literature review and a motivating case study experiment, (ii) development of a conceptual process model supporting interaction evolution, (iii) development of a model based on the notion of evaluation functions which is shown to support a wide range of interaction configuration approaches, (iv) a characterisation of the configuration evaluation space, followed by (v) an implementation of these ideas used in (vi) a series of longitudinal technology probes and investigations into the approaches

An Exploration of Traditional and Data Driven Predictors of Programming Performance

This thesis investigates factors that can be used to predict the success or failure of students taking an introductory programming course. Four studies were performed to explore how aspects of the teaching context, static factors based upon traditional learning theories, and data-driven metrics derived from aspects of programming behaviour were related to programming performance. In the first study, a systematic review into the worldwide outcomes of programming courses revealed an average pass rate of 67.7\%. This was found to have not significantly changed over time, or to have differed based upon aspects of the teaching context, such as the programming language taught to students. The second study showed that many of the factors based upon traditional learning theories, such as learning styles, are context dependent, and fail to consistently predict programming performance when they are applied across different teaching contexts. The third study explored data-driven metrics derived from the programming behaviour of students. Analysing data logged from students using the BlueJ IDE, 10 new data-driven metrics were identified and validated on three independently gathered datasets. Weaker students were found to make a greater percentage of successive errors, and spend a greater percentage of their lab time resolving errors than stronger students. The Robust Relative algorithm was developed to hybridize four of the strongest data-driven metrics into a performance predictor. The novel relative scoring of students based upon how their resolve times for different types of errors compared to the resolve times of their peers, resulted in a predictor which could explain a large proportion of the variance in the performance of three independent cohorts, $R^2$ = 42.19\%, 43.65\% and 44.17\% - almost double the variance which could be explained by Jadud's Error Quotient metric. The fourth study situated the findings of this thesis within the wider literature, by applying meta-analysis techniques to statistically synthesise fifty years of conflicting research, such that the most important factors for learning programming could be identified. 482 results describing the effects of 116 factors on programming performance were synthesised and consolidated to form a six class theoretical framework. The results showed that the strongest predictors identified over the past fifty years are data-driven metrics based upon programming behaviour. Several of the traditional predictors were also found to be influential, suggesting that both a certain level of scientific maturity and self-concept are necessary for programming. Two thirds of the weakest predictors were based upon demographic and psychological factors, suggesting that age, gender, self-perceived abilities, learning styles, and personality traits have no relevance for programming performance. This thesis argues that factors based upon traditional learning theories struggle to consistently predict programming performance across different teaching contexts because they were not intended to be applied for this purpose. In contrast, the main advantage of using data-driven approaches to derive metrics based upon students' programming processes, is that these metrics are directly based upon the programming behaviours of students, and therefore can encapsulate such changes in their programming knowledge over time. Researchers should continue to explore data-driven predictors in the future

Empirical modelling as a new paradigm for educational technology

Educational technology has yet to deliver the benefits or successes that were expected in educational practice, especially in relation to issues other than the communication and delivery of teaching materials. Evidence suggests that these difficulties stem from the mismatch between formalised virtual learning environments and everyday sensemaking and between the rich potential for enhanced learning afforded by new technology and the constraints of old-style educational practice. In addressing this mismatch, some commentators suggest that the primary need is for a new culture of educational practice-and even that such a culture is already emerging, and others identify the need for a new paradigm for educational technology. The aim of this thesis is to explore the potential for a new paradigm for educational technology based on the principles and tools of Empirical Modelling (see http://dcs.warwick.ac.uk/modelling). The thesis builds upon previous research on Empirical Modelling as a constructionist approach to learning, and in particular Roe's doctoral thesis 'Computers for learning: an Empirical Modelling perspective'. Roe's treatment of Empirical Modelling can be viewed as generalising the use of spreadsheets for learning through applying 'programming by dependency' within the framework of existing educational practice. In contrast, this thesis is concerned at a more fundamental level with the contribution that Empirical Modelling can make to technology enhanced learning that may lead to new educational practices. In particular, it identifies eight significant characteristics of learning that are well-matched to Empirical Modelling activity, and associates these with experimental, flexible and meaningful strands in learning. The credentials of Empirical Modelling as a potential new foundation for educational technology are enhanced by demonstrating that Empirical Modelling is radically different from traditional software development and use. It provides a methodology for modelling with dependency that is more closely related to the use of spreadsheets for learning. The thesis elaborates on the relationship between Empirical Modelling and learning in a variety of different contexts, ways and applications. Three examples drawn from computer science higher education are explored to emphasise the experimental, flexible and meaningful characteristics of Empirical Modelling. This discussion of Empirical Modelling in a specific educational context is complemented by an investigation of its relevance to learning in a wider context, with reference to a broad range of subjects, to specific issues in language learning, and to the topics of lifelong learning and collaborative learning. Although the application of Empirical Modelling for learning is as yet too immature for large scale empirical studies, its potential is evaluated using informal empirical evidence arising from Empirical Modelling practice at Warwick. The sources for this evaluation are well-established teaching activities relating to Empirical Modelling in Computer Science at the University of 'Warwick, comprising an introductory module and a number of final year undergraduate projects. The thesis concludes by considering the extent to which Empirical Modelling can go beyond the support for constructionism envisaged by Roe, to address the broader agenda of supporting constructivist learning using computers. To this end, a close relationship between Empirical Modelling and a vision of constructivism recently set out by Bruno Latour in his paper 'The Promises of Constructivism' is demonstrated.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Challenges of rapid migration to fully virtual education in the age of the Corona virus pandemic: experiences from across the world

The social disruption caused by the sudden eruption of the Corona Virus pandemic has shaken the whole world, influencing all levels of education immensely. Notwithstanding there was a lack of preparedness for this global public health emergency which continues to affect all aspects of work and life. The problem is, naturally, multifaceted, fast evolving and complex, affecting everyone, threatening our well-being, the global economy, the environment and all societal and cultural norms and our everyday activities. In a recent UNESCO report it is noted that nearly a billion and a quarter (which is 67,7 % of the total number) of learners have been affected by the Corona Virus pandemic worldwide. The education sector at all levels has been one of the hardest hit sectors particularly as the academic/school year was in full swing. The impact of the pandemic is widespread, representing a health hazard worldwide. Being such, it profoundly affects society as a whole, and its members that are, in particular, i) individuals (the learners, their parents, educators, support staff), ii) schools, training organisations, pedagogical institutions and education systems, iii) quickly transformed policies, methods and pedagogies to serve the newly appeared needs of the latter. Lengthy developments of such scale usually take years of consultation, strategic planning and implementation. In addition to raising awareness across the population of the dangers of the virus transmission and instigating total lockdown, it has been necessary to develop mechanisms for continuing the delivery of education as well as demanding mechanisms for assuring the quality of the educational experience and educational results. There is often scepticism about securing quality standards in such a fast moving situation. Often in the recent past, the perception was that courses and degrees leading to an award are inferior if the course modules (and sometimes its assessment components) were wholly online. Over the last three decades most Higher Education institutions developed both considerable infrastructure and knowhow enabling distance mode delivery schools (Primary and Secondary) had hardly any necessary infrastructure nor adequate knowhow for enabling virtual education. In addition, community education and various training providers were mainly delivered face-to-face and that had to either stop altogether or rapidly convert materials, exercises and tests for online delivery and testing. A high degree of flexibility and commitment was demanded of all involved and particularly from the educators, who undertook to produce new educational materials in order to provide online support to pupils and students. Apart from the delivery mode of education, which is serving for certificated programmes, it is essential to ensure that learners’ needs are thoroughly and continuously addressed and are efficiently supported throughout the Coronavirus or any other future lockdown. The latter can be originated by various causes and reasons that vary in nature, such as natural or socioeconomical. Readiness, thus, in addition to preparedness, is the primary key question and solution when it comes to quality education for any lockdown. In most countries, the compulsory primary and secondary education sectors have been facing a more difficult challenge than that faced by Higher Education. The poor or in many cases non-existent technological infrastructure and low technological expertise of the teachers, instructors and parents, make the delivery of virtual education difficult or even impossible. The latter, coupled with phenomena such as social exclusion and digital divide where thousands of households do not have adequate access to broadband Internet, Wi-Fi infrastructure and personal computers hamper the promising and strenuous virtual solutions. The shockwaves of the sudden demands on all sectors of society and on individuals required rapid decisions and actions. We will not attempt to answer the question “Why was the world unprepared for the onslaught of the Coronavirus pandemic” but need to ascertain the level of preparedness and readiness particularly of the education sector, to effect the required rapid transition. We aimed to identify the challenges, and problems faced by the educators and their institutions. Through first-hand experiences we also identify best practices and solutions reached. Thus we constructed a questionnaire to gather our own responses but also experiences from colleagues and members of our environment, family, friends, and colleagues. This paper reports the first-hand experiences and knowledge of 33 co-authors from 27 institutions and from 13 different countries from Europe, Asia, and Africa. The communication technologies and development platforms used are identified; the challenges faced as well as solutions and best practices are reported. The findings are consolidated into the four areas explored i.e. Development Platforms, Communications Technologies, Challenges/Problems and Solutions/Best Practices. The conclusion summarises the findings into emerging themes and similarities. Reflections on the lasting impact of the effect of Coronavirus on education, limitations of study, and indications of future work complete the paper

Exploring higher education engagement in computer programming within a blended learning environment : an action research approach

Ph. D. University of KwaZulu-Natal, Durban 2014.Many novice programmers in higher education find computer programming particularly difficult due to its problem solving nature. High dropout rates have been observed both internationally and locally, but in South Africa, the circumstances of students coming from disadvantaged schools where they struggle in subjects like Mathematics and Science, especially compounds their challenges in computer programming when they enrol at a tertiary institute. In this study, I explore the engagement of computer programming at a higher education institution using an innovative approach of incorporating tools in the form of online learning and support structures to supplement the existing face-to-face and practical lessons thereby creating a blended learning environment (BLE). This study, which is a qualitative one, used an interpretivist paradigm to explore the engagement of sixty, first year students in an introductory computer-programming course at a selected university in South Africa, using an action research approach within the context of a BLE. Action research refers to an evaluation of one’s own practice with a view to improving one’s effectiveness, in this case, analysing my own efficacy as a teacher, and the learning that occurred by my students (McNiff, 2013; Whitehead, 1989). This study used two lenses: The first lens was my own as a lecturer/researcher who developed a variety of support structures in the form of notes, videos, animations, and blogging, to support student engagement in computer programming, and the second lens was the students’ engagement with these tools. The study explored this dual engagement and asked two critical questions: 1) How does engagement of computer programming take place within a BL context using an action research approach, and, 2) Why does engagement of computer programming take place within a BL context using an action research approach, in the way it does? A dual form of engagement occurred creating a dynamic BLE. In the study, students were exposed to one theory classroom lesson, and three practical lessons. As the lecturer, I received feedback from the students which informed my attempts to improve the environment. Observations, a personal diary, electronic questionnaires, and focus group meetings were used to gather feedback on how students engaged in the BLE. The action research methodology was based on planning, acting, observing and reflecting. The analysis of the reflections was used in the re-planning phase of the next cycle and a total of three cycles were used. Although there were three main action research cycles, each tool was transformed resulting in smaller cycles emanating within the main action research cycle. Activity Theory was used as a theoretical framework to describe and analyse the actions and engagement that transpired within the BLE. The results from this study highlight positive student engagement in learning through the use of examples and visual tools although the use of language was found to be a barrier under certain circumstance. Support and planning were also identified as important factors for both student and lecture engagement. Other aspects concerning feedback and reflection were established as important during the dual engagement employed resulting in the creation of a dynamic action research model of engagement

The Derivation of a pragmatic requirements framework for web development.

Web-based development is a relatively immature area of Software Engineering, producing often complex applications to many different types of end user and stakeholders. Web Engineering as a research area, was created to introduce processes that enable web based development to be repeatable and to avoid potential failure in the fast changing landscape that is the current ubiquitous Internet. A survey of existing perspectives from the literature highlights a number of points. Firstly, that web development has a number of subtle differences to Software Engineering and that many web development methods are not used. Further, that there has been little work done on what should be in a web development method. A full survey of 50 web development methods finds that they do not give enough detail to be used in their entirety; they are difficult for a non-computer scientist to understand in the techniques they use and most do not cover the lifecycle, particularly in the area of requirements, implementation and testing. This thesis introduces a requirements framework for novice web developers. It is created following an in-depth case study carried out over two years that investigates the use of web development methods by novice developers. The study finds that web development methods are not easy to understand, there is a lack of explanation as to how to use the techniques within the method and the language used is too complex. A high level method is derived with an iterative process and with the requirements phase in the form of a framework; it addresses the problems that are discussed and provides excellent support for a novice web developer in the requirements phase of the lifecycle. An evaluation of the method using a group of novice developers who reflect on the method and a group who use it for development finds that the method is both easy to understand and use