Student profiling in a dispositional learning analytics application using formative assessment

How learning disposition data can help us translating learning feedback from a learning analytics application into actionable learning interventions, is the main focus of this empirical study. It extends previous work where the focus was on deriving timely prediction models in a data rich context, encompassing trace data from learning management systems, formative assessment data, e-tutorial trace data as well as learning dispositions. In this same educational context, the current study investigates how the application of cluster analysis based on e-tutorial trace data allows student profiling into different at-risk groups, and how these at-risk groups can be characterized with the help of learning disposition data. It is our conjecture that establishing a chain of antecedent-consequence relationships starting from learning disposition, through student activity in e-tutorials and formative assessment performance, to course performance, adds a crucial dimension to current learning analytics studies: that of profiling students with descriptors that easily lend themselves to the design of educational interventions

Stability and sensitivity of Learning Analytics based prediction models

Learning analytics seek to enhance the learning processes through systematic measurements of learning related data and to provide informative feedback to learners and educators. Track data from Learning Management Systems (LMS) constitute a main data source for learning analytics. This empirical contribution provides an application of Buckingham Shum and Deakin Crick’s theoretical framework of dispositional learning analytics: an infrastructure that combines learning dispositions data with data extracted from computer-assisted, formative assessments and LMSs. In two cohorts of a large introductory quantitative methods module, 2049 students were enrolled in a module based on principles of blended learning, combining face-to-face Problem-Based Learning sessions with e-tutorials. We investigated the predictive power of learning dispositions, outcomes of continuous formative assessments and other system generated data in modelling student performance and their potential to generate informative feedback. Using a dynamic, longitudinal perspective, computer-assisted formative assessments seem to be the best predictor for detecting underperforming students and academic performance, while basic LMS data did not substantially predict learning. If timely feedback is crucial, both use-intensity related track data from e-tutorial systems, and learning dispositions, are valuable sources for feedback generation

Data mining technology for the evaluation of learning content interaction

Interactivity is central for the success of learning. In e-learning and other educational multimedia environments, the evaluation of interaction and behaviour is particularly crucial. Data mining – a non-intrusive, objective analysis technology – shall be proposed as the central evaluation technology for the analysis of the usage of computer-based educational environments and in particular of the interaction with educational content. Basic mining techniques are reviewed and their application in a Web-based third-level course environment is illustrated. Analytic models capturing interaction aspects from the application domain (learning) and the software infrastructure (interactive multimedia) are required for the meaningful interpretation of mining results

Layered evaluation of interactive adaptive systems : framework and formative methods

Peer reviewedPostprin

Implementation of computer assisted assessment: lessons from the literature

This paper draws attention to literature surrounding the subject of computer-assisted assessment (CAA). A brief overview of traditional methods of assessment is presented, highlighting areas of concern in existing techniques. CAA is then defined, and instances of its introduction in various educational spheres are identified, with the main focus of the paper concerning the implementation of CAA. Through referenced articles, evidence is offered to inform practitioners, and direct further research into CAA from a technological and pedagogical perspective. This includes issues relating to interoperability of questions, security, test construction and testing higher cognitive skills. The paper concludes by suggesting that an institutional strategy for CAA coupled with staff development in test construction for a CAA environment can increase the chances of successful implementation

The role of unit evaluation, learning and culture dimensions related to student cognitive style in hypermedia learning

Recent developments in learning technologies such as hypermedia are\ud becoming widespread and offer significant contributions to improving the delivery\ud of learning and teaching materials. A key factor in the development of hypermedia\ud learning systems is cognitive style (CS) as it relates to users‟ information\ud processing habits, representing individual users‟ typical modes of perceiving,\ud thinking, remembering and problem solving.\ud \ud \ud \ud \ud A total of 97 students from Australian (45) and Malaysian (52) universities\ud participated in a survey. Five types of predictor variables were investigated with\ud the CS: (i) three learning dimensions; (ii) five culture dimensions; (iii) evaluation\ud of units; (iv) demographics of students; and (v) country in which students studied.\ud Both multiple regression models and tree-based regression were used to analyse\ud the direct effect of the five types of predictor variables, and the interactions within\ud each type of predictor variable. When comparing both models, tree-based\ud regression outperformed the generalized linear model in this study. The research\ud findings indicate that unit evaluation is the primary variable to determine students‟\ud CS. A secondary variable is learning dimension and, among the three dimensions,\ud only nonlinear learning and learner control dimensions have an effect on students‟\ud CS. The last variable is culture and, among the five culture dimensions, only\ud power distance, long term orientation, and individualism have effects on students‟\ud CS. Neither demographics nor country have an effect on students‟ CS.\ud These overall findings suggest that traditional unit evaluation, students‟\ud preference for learning dimensions (such as linear vs non-linear), level of learner\ud control and culture orientation must be taken into consideration in order to enrich\ud students‟ quality of education. This enrichment includes motivating students to\ud acquire subject matter through individualized instruction when designing,\ud developing and delivering educational resources

Towards an Integrative Formative Approach of Data-Driven Decision Making, Assessment for Learning, and Diagnostic Testing

This study concerns the comparison of three approaches to assessment: Data-Driven Decision Making, Assessment for Learning, and Diagnostic Testing. Although the three approaches claim to be beneficial with regard to student learning, no clear study into the relationships and distinctions between these approaches exists to date. The goal of this study was to investigate the extent to which the three approaches can be shaped into an integrative formative approach towards assessment. The three approaches were compared on nine characteristics of assessment. The results suggest that although the approaches seem to be contradictory with respect to some characteristics, it is argued that they could complement each other despite these differences. The researchers discuss how the three approaches can be shaped into an integrative formative approach towards assessmen

Responsible research and innovation in science education: insights from evaluating the impact of using digital media and arts-based methods on RRI values

The European Commission policy approach of Responsible Research and Innovation (RRI) is gaining momentum in European research planning and development as a strategy to align scientific and technological progress with socially desirable and acceptable ends. One of the RRI agendas is science education, aiming to foster future generations' acquisition of skills and values needed to engage in society responsibly. To this end, it is argued that RRI-based science education can benefit from more interdisciplinary methods such as those based on arts and digital technologies. However, the evidence existing on the impact of science education activities using digital media and arts-based methods on RRI values remains underexplored. This article comparatively reviews previous evidence on the evaluation of these activities, from primary to higher education, to examine whether and how RRI-related learning outcomes are evaluated and how these activities impact on students' learning. Forty academic publications were selected and its content analysed according to five RRI values: creative and critical thinking, engagement, inclusiveness, gender equality and integration of ethical issues. When evaluating the impact of digital and arts-based methods in science education activities, creative and critical thinking, engagement and partly inclusiveness are the RRI values mainly addressed. In contrast, gender equality and ethics integration are neglected. Digital-based methods seem to be more focused on students' questioning and inquiry skills, whereas those using arts often examine imagination, curiosity and autonomy. Differences in the evaluation focus between studies on digital media and those on arts partly explain differences in their impact on RRI values, but also result in non-documented outcomes and undermine their potential. Further developments in interdisciplinary approaches to science education following the RRI policy agenda should reinforce the design of the activities as well as procedural aspects of the evaluation research

Students’ Perceptions of Their Teachers’ Performance in Teaching Engineering Drawing in Nigerian Tertiary Institutions

There have been concerns about the performance of Nigerian school teachers’ in delivering occupational related courses. However, there are currently limited empirical data on this phenomenon – in particular with respect to the teaching of engineering drawing – to justify further actions from educational managers and policy makers. The aim of this study was to assess teachers’ performance in teaching engineering drawing using students’ perception as indicator of teachers’ performance. The study utilized a cross-sectional research design method with the target population of technical education students drawn from four (4) Federal Colleges of education (Technical) in Northern Nigeria. Stratified proportionate sampling technique was used to arrive at the study sample of 253 technical education students. A specifically designed instrument, the Students’ Perceptions of Teachers’ Performance Scales (SPTPS) was used to gather data on the three performance dimensions namely contextual, task and adaptability performance. The exploratory factor analysis and confirmatory factor analysis methods were conducted to validate the performance constructs. The instrument has a high reliability of 0.90 based on the Cronbach Alpha method. The result of the analysis using estimation method indicates that students perceive their teachers’ performance to be at a slightly above average level (M= 3.51 ± 0.05 at the 95% confidence level). The teachers’ task performance, in particular, is found to be the least developed among the three dimension of performance while their adaptability performance is the highest while still being less than excellent. The data support the conclusion that there are aspects of teachers’ performance in teaching engineering drawing that is less than excellent and in need of further enhancements

Students’ Perceptions of Their Teachers’ Performance in Teaching Engineering Drawing in Nigerian Tertiary Institutions

There have been concerns about the performance of Nigerian school teachers’ in delivering occupational related courses. However, there are currently limited empirical data on this phenomenon – in particular with respect to the teaching of engineering drawing – to justify further actions from educational managers and policy makers. The aim of this study was to assess teachers’ performance in teaching engineering drawing using students’ perception as indicator of teachers’ performance. The study utilized a cross-sectional research design method with the target population of technical education students drawn from four (4) Federal Colleges of education (Technical) in Northern Nigeria. Stratified proportionate sampling technique was used to arrive at the study sample of 253 technical education students. A specifically designed instrument, the Students’ Perceptions of Teachers’ Performance Scales (SPTPS) was used to gather data on the three performance dimensions namely contextual, task and adaptability performance. The exploratory factor analysis and confirmatory factor analysis methods were conducted to validate the performance constructs. The instrument has a high reliability of 0.90 based on the Cronbach Alpha method. The result of the analysis using estimation method indicates that students perceive their teachers’ performance to be at a slightly above average level (M= 3.51 ± 0.05 at the 95% confidence level). The teachers’ task performance, in particular, is found to be the least developed among the three dimension of performance while their adaptability performance is the highest while still being less than excellent. The data support the conclusion that there are aspects of teachers’ performance in teaching engineering drawing that is less than excellent and in need of further enhancements