Predicting Academic Success Based on Learning Material Usage

In this work, we explore students' usage of online learning material as a predictor of academic success. In the context of an introductory programming course, we recorded the amount of time that each element such as a text paragraph or an image was visible on the students' screen. Then, we applied machine learning methods to study to what extent material usage predicts course outcomes. Our results show that the time spent with each paragraph of the online learning material is a moderate predictor of student success even when corrected for student time-on-task, and that the information can be used to identify at-risk students. The predictive performance of the models is dependent on the quantity of data, and the predictions become more accurate as the course progresses. In a broader context, our results indicate that course material usage can be used to predict academic success, and that such data can be collected in-situ with minimal interference to the students' learning process.Peer reviewe

The Potential for Student Performance Prediction in Small Cohorts with Minimal Available Attributes

The measurement of student performance during their progress through university study provides academic leadership with critical information on each student’s likelihood of success. Academics have traditionally used their interactions with individual students through class activities and interim assessments to identify those “at risk” of failure/withdrawal. However, modern university environments, offering easy on-line availability of course material, may see reduced lecture/tutorial attendance, making such identification more challenging. Modern data mining and machine learning techniques provide increasingly accurate predictions of student examination assessment marks, although these approaches have focussed upon large student populations and wide ranges of data attributes per student. However, many university modules comprise relatively small student cohorts, with institutional protocols limiting the student attributes available for analysis. It appears that very little research attention has been devoted to this area of analysis and prediction. We describe an experiment conducted on a final-year university module student cohort of 23, where individual student data are limited to lecture/tutorial attendance, virtual learning environment accesses and intermediate assessments. We found potential for predicting individual student interim and final assessment marks in small student cohorts with very limited attributes and that these predictions could be useful to support module leaders in identifying students potentially “at risk.”.Peer reviewe

Academic Performance and Behavioral Patterns

Identifying the factors that influence academic performance is an essential part of educational research. Previous studies have documented the importance of personality traits, class attendance, and social network structure. Because most of these analyses were based on a single behavioral aspect and/or small sample sizes, there is currently no quantification of the interplay of these factors. Here, we study the academic performance among a cohort of 538 undergraduate students forming a single, densely connected social network. Our work is based on data collected using smartphones, which the students used as their primary phones for two years. The availability of multi-channel data from a single population allows us to directly compare the explanatory power of individual and social characteristics. We find that the most informative indicators of performance are based on social ties and that network indicators result in better model performance than individual characteristics (including both personality and class attendance). We confirm earlier findings that class attendance is the most important predictor among individual characteristics. Finally, our results suggest the presence of strong homophily and/or peer effects among university students

The Knowledge Application and Utilization Framework Applied to Defense COTS: A Research Synthesis for Outsourced Innovation

Purpose -- Militaries of developing nations face increasing budget pressures, high operations tempo, a blitzing pace of technology, and adversaries that often meet or beat government capabilities using commercial off-the-shelf (COTS) technologies. The adoption of COTS products into defense acquisitions has been offered to help meet these challenges by essentially outsourcing new product development and innovation. This research summarizes extant research to develop a framework for managing the innovative and knowledge flows. Design/Methodology/Approach – A literature review of 62 sources was conducted with the objectives of identifying antecedents (barriers and facilitators) and consequences of COTS adoption. Findings – The DoD COTS literature predominantly consists of industry case studies, and there’s a strong need for further academically rigorous study. Extant rigorous research implicates the importance of the role of knowledge management to government innovative thinking that relies heavily on commercial suppliers. Research Limitations/Implications – Extant academically rigorous studies tend to depend on measures derived from work in information systems research, relying on user satisfaction as the outcome. Our findings indicate that user satisfaction has no relationship to COTS success; technically complex governmental purchases may be too distant from users or may have socio-economic goals that supersede user satisfaction. The knowledge acquisition and utilization framework worked well to explain the innovative process in COTS. Practical Implications – Where past research in the commercial context found technological knowledge to outweigh market knowledge in terms of importance, our research found the opposite. Managers either in government or marketing to government should be aware of the importance of market knowledge for defense COTS innovation, especially for commercial companies that work as system integrators. Originality/Value – From the literature emerged a framework of COTS product usage and a scale to measure COTS product appropriateness that should help to guide COTS product adoption decisions and to help manage COTS product implementations ex post

Scholarly insight Autumn 2017:a Data wrangler perspective

As the OU is going through several fundamental changes, it is important that strategic decisions made by Faculties and senior management are informed by evidence-based research and insights. One way how Data Wranglers provide insights of longitudinal development and performance of OU modules is the Key Metric Report 2017. A particular new element is that data can now also be unpacked and visualised on a Nation-level. As evidenced by the Nation-level reporting, there are substantial variations of success across the four Nations, and we hope that our interactive dashboards allow OU staff to unpack the underlying data. The second way Data Wranglers provide insight to Faculties and Units is through the Scholarly insight report series. Building on the previous two reports whereby we reported on substantial variation and inconsistencies in learning designs and assessment practices within qualifications across the OU, in this Scholarly insight Autumn 2017 report we address four big pedagogical questions that were framed and co-constructed together with the Faculties and LTI units. Many Faculties and colleagues have reacted positively on our Scholarly insight Spring 2017 report, whereby for the first time we were able to show empirically that students experienced substantial variations in success within 12 large OU qualifications. As evidenced in our previous report, 55% of variation in students’ success over time was explained by OU institutional factors (i.e., how students were assessed within their respective module; how students were able to effectively transition from one learning design of one module to the next one), rather than students’ characteristics, engagement and behaviour. We have received several queries and questions from Faculties and Units about how to better understand these students’ journeys, and how qualifications and module designs could be better aligned within their respective qualification(s). As these are complex conceptual and Big Pedagogy questions, in Chapter 1 we continued these complex analyses by looking at the transitional processes of the first two modules that OU students take, and how well aligned these modules and qualification paths are. In Chapter 2, we explored the more fine-grained, qualitative, and lived experiences of 19 students across a range of qualifications to understand how OU grading practices and (in)consistencies of assessment and feedback influenced their affect, behaviour, and cognition. In addition to building on previous topics, we introduced two new Scholarly insights in Chapter 3 and Chapter 4. As the OU is increasingly using learning analytics to support our staff and students, in Chapter 3 we analysed the impact of giving Predictive Learning Analytics to over 500 Associate Lecturers across 31 modules on student retention. Finally, in Chapter 4 we explored the impact of first presentations of new modules on pass rates and satisfaction, whereby we were able to bust another myth that may have profound implications for Student First Transformation. Working organically in various Faculty sub-group meetings and LTI Units and in a google doc with various key stakeholders in the Faculties , we hope that our Scholarly insights can help to inform our staff, but also spark some ideas how to further improve our module designs and qualification pathways. Of course we are keen to hear what other topics require Scholarly insight

Using Shared Workspaces in Higher Education

We evaluate the use of BSCW shared workspaces in higher education by means of a comparison of seven courses in which this environment was used. We identify a number of different functions for which the BSCW environment has been used and discuss the relative success of these functions across the cases. In addition, we evaluate the cases with the 4E model of Collis et al. (2000) which predicts the chances of acceptance of ICT in an educational setting. Effectiveness for the given task appears to be a prime success factor for using ICT. But an effective tool may fail due to other factors like ease of use and organisational, socialcultural or technological obstacles. The particular strength of a shared workspace, for which BSCW is most effective and efficient, is providing a repository for objects of collaborative work. Other types of usage showed mixed results. In the future we expect that learning takes place in an integrated, open ICT environment in which different kinds of tools are available for different purposes and users can switch between tools as appropriate. We could observe this in several of the case studies, where non-use of BSCW did not mean that a particular task was not performed, but, on the contrary, a more efficient solution for the same function was available. Shared workspaces have proven to be highly useful, but it seems advisable that their purpose be limited to what they were originally designed for

The Effects of Motivation, Technology and Satisfaction on Student Achievement in Face-to-Face and Online College Algebra Classes

Demand for online learning has increased in recent years due to the convenience of class delivery. However, some students appear to have difficulties with online education resulting in lack of completion. The study utilized a quantitative approach with archival data and survey design. The factors of demographics, motivation, technology, and satisfaction were compared for face-to-face and online students. MANCOVA tests were performed to analyze the data while controlling age and gender to uncover significant differences between the two groups. The sample and population for this study were predominantly Hispanic students. Motivation and Technology were non-significant, but satisfaction was proven to be significant. In face-to-face courses, females were more satisfied than males. While in online courses, males were more satisfied than females. There was an interaction effect between the methods of instruction and the grade levels of A, B, C, D, and F/W on the dependent variables; Motivation, Technology, and Satisfaction

From Patient to Student Activation: Development of the Student Activation Measure

The Patient Activation Measure (PAM) was constructed to measure a person’s knowledge, skill, and confidence for self-managing one’s healthcare, or “activation” (Hibbard, Stockard, Mahoney, & Tusler, 2004). The Student Activation Measure (SAM) extends this definition to secondary education. The SAM is a short, positively worded measure that is intended to guide intervention planning. Six hundred three students from two disparate high schools located in the Pacific Northwest completed the measure and an accompanying demographic questionnaire. The respective schools provided the students’ GPAs and attendance records. Using Rasch modeling, the SAM evidenced excellent reliability and construct validity. One-way ANOVAs with post hoc Scheffe’s tests showed that higher SAM scores had significantly higher GPAs, fewer absences, increased time spent on homework, and less time spent on social media or playing video games. Overall, the SAM showed promise as both a research and intervention tool. In addition, the concept of activation has the added benefits of ease of measurement and bridges the gap between evidence-based practices in medicine and secondary education. Further research is needed to understand the properties of the SAM when used with students diagnosed with learning impairing disorders such as ADHD