The Usability of E-learning Platforms in Higher Education: A Systematic Mapping Study

The use of e-learning in higher education has increased significantly in recent years, which has led to several studies being conducted to investigate the usability of the platforms that support it. A variety of different usability evaluation methods and attributes have been used, and it has therefore become important to start reviewing this work in a systematic way to determine how the field has developed in the last 15 years. This paper describes a systematic mapping study that performed searches on five electronic libraries to identify usability issues and methods that have been used to evaluate e-learning platforms. Sixty-one papers were selected and analysed, with the majority of studies using a simple research design reliant on questionnaires. The usability attributes measured were mostly related to effectiveness, satisfaction, efficiency, and perceived ease of use. Furthermore, several research gaps have been identified and recommendations have been made for further work in the area of the usability of online learning

Assessing the usability of virtual learning environments in higher education

Context: E-learning is an integral part of the modern higher education system, and therefore it is essential that students and staff are able to use systems that support E-learning, such as Virtual Learning Environments (VLEs), effectively. Usability is essential to ensure effective use of these systems and is often assessed by means of subjective questions. Although developed mainly for industry use, the Technology Acceptance Model (TAM) and System Usability Scale (SUS) questionnaire are often used to assess E-learning systems. Goal: The main goal of this thesis is to assess the usability of a VLE platform currently used in universities (Blackboard) and identify the most common and appropriate methods used to assess a VLE platform’s usability. Another aim is to investigate whether there are extensions to common usability models and methods (such as the SUS and TAM) that could improve their accuracy, including the potential of combining them with more objective measures such as number of clicks, time taken and open-ended questions. Method: The literature on VLE usability evaluation was reviewed using a mapping study methodology to identify the usability methods and factors that have been used previously. Informed by the findings of this study, a set of usability questionnaires have been developed, used and evaluated, with 101 student respondents recruited from all the Schools at Keele University participating in the first study (Chapter Four) and 162 in the second study (Chapter Five). A standard usability questionnaire and a novel form of observation were then combined to record 25 participants’ interactions with the VLE (Chapter Six) while they completed a set of representative tasks in two sessions that were held eight weeks apart. These interactions were then compared. Results: The results indicate that the VLE performed below the average usability expectation score (SUS score of 62.52) but is still considered as ‘acceptable’. Twenty-seven free text responses were also obtained in the first study and a thematic analysis of comments revealed very negative views of the VLE as well as areas for improvement. In the second study, it was found that perceived enjoyment (PE) and usability were jointly related to the perceived usefulness (PU), although the association was relatively weak. Perceived enjoyment and learnability were jointly associated with perceived ease of use (PEOU), with the association accounting for 39% of the variation in PEOU. Usability was related to PE but learnability was not. Overall, the original TAM can be improved by the addition of learnability, PE and usability as they have a positive effect on TAM. In the final study, the task success rate was relatively high (i.e. 82.3% in session 1); however, an average participant took 3.6 times longer to complete the set of tasks than a competent user. Furthermore, task time, clicks and success rate improved only marginally in the second session (which was at the end of the semester). However, when compared with the analysis of the results from the standard usability questionnaires (subjective measures), participants stated that they were satisfied with the usability of the system, contradicting the objective measures (number of clicks, task time and success rate). Conclusions: Using subjective measures alone, in the form of standard usability questionnaires, to assess the usability of a complex system can conceal significant issues. Usability assessment should therefore be based on actual performance against a defined baseline and combined with forms of qualitative feedback such as free text responses. Evaluating the effect of usability on E-learning is complicated. The studies conducted in this thesis have provided valuable guidance on how to measure the usability of VLEs. Suggestions for future work on the usability of VLEs as well as appropriate recommendations are provided

User experience evaluation of electronic moderation systems : a case study at a private higher education institution in South Africa

The transformation of a manual paper-based moderation process into an electronic moderation (eModeration) process poses unique challenges. These challenges concern academic processes, people and the user experience of interactive systems. eModeration can improve the user experience of assessment processes while lowering the risk of delaying the process or losing scripts. Despite the benefits associated with optimising assessment procedures, particularly examination procedures, the use of eModeration in South Africa is limited. There are several possible reasons for a lack of eModeration adoption ranging from infrastructure and technical issues through to organisational and human factors. The focus of this study is on the human factors involved in eModeration. Since no User Experience Evaluation Framework for eModeration existed at the time of this research, an in-depth study was conducted based on the experiences of eModeration users in the context of private higher education institutions. The study focused on identifying the most important user experience constructs for the evaluation of an eModerate system within the context of private higher education institutions in South Africa towards proposing a framework. The study was based in the fields of Information Systems and Human-Computer Interaction with eModeration being the application domain. The research used a Design Science Research methodology, which involved the development and testing of a User Experience Evaluation Framework for eModeration. The data generation methods included interviews with deans, eModerators and management, as well as a survey that included responses from both moderators and deans. The research was conducted at Midrand Graduate Institute and evaluated at Monash University. The study makes a validated contribution towards identifying the most important user experience constructs. The identified constructs were utilised in the design and development of the User Experience Evaluation Framework for eModeration, which can be used along with the evaluation criteria tool to evaluate eModerate systems.Information ScienceD. Litt. et Phil. (Information Systems