Assessing the Usability of a Visual Tool for the definition of E-learning Processes

In this paper, we present a usability study aiming at assessing a visual language-based tool for developing adaptive e-learning processes. The tool implements the adaptive self-consistent learning object SET (ASCLO-S) visual language, a special case of flow diagrams, to be used by instructional designers to define classes of learners through stereotypes and to specify the more suited adaptive learning process for each class of learners. The usability study is based on the combined use of two techniques: a questionnaire-based survey and an empirical analysis. The survey has been used to achieve feedbacks from the subjects' point of view. In particular, it has been useful to capture the perceived usability of the subjects. The outcomes show that both the proposed visual notation and the system prototype are suitable for instructional designers with or without experience on the computer usage and on tools for defining e-learning processes. This result is further confirmed by the empirical analysis we carried out by analysing the correlation between the effort to develop adaptive e-learning processes and some measures suitable defined for those processes. Indeed, the empirical analysis revealed that the effort required to model e-learning processes is not influenced by the experience of the instructional designer with the use of e-learning tools, but it only depends on the size of the developed process

Cognitive Effectiveness of Visual Instructional Design Languages

The introduction of learning technologies into education is making the design of courses and instructional materials an increasingly complex task. Instructional design languages are identified as conceptual tools for achieving more standardized and, at the same time, more creative design solutions, as well as enhancing communication and transparency in the design process. In this article we discuss differences in cognitive aspects of three visual instructional design languages (E²ML, PoEML, coUML), based on user evaluation. Cognitive aspects are of relevance for learning a design language, creating models with it, and understanding models created using it. The findings should enable language constructors to improve the usability of visual instructional design languages in the future. The paper concludes with directions with regard to how future research on visual instructional design languages could strengthen their value and enhance their actual use by educators and designers by synthesizing existing efforts into a unified modeling approach for VIDLs

Modelling of pedagogical patterns through e-learning objects

Програмні платформи для електронного навчання підтримують різні варіанти подання навчального контенту. Одним із способів для його організації та структурування є так звані педагогічні моделі. Вони є методом для відображення і розповсюдження отриманих знань та практичного досвіду. Педагогічні моделі використовуються для опису педагогічних ситуацій, які неодноразово виникають під час навчання. У контексті систем електронного навчання існують різні підходи до цифровізації педагогічних моделей. Мета роботи – показати, як побудувати педагогічні моделі, використовуючи педагогічні об’єкти електронного навчання, які можна легко та зручно впровадити як моделі в адаптивному середовищі електронного навчання. Педагогічний об’єкт електронного навчання – це абстрактне поняття, яке може бути представлено в конкретній формі об’єкта електронного навчання, методичного об’єкта електронного навчання, об’єкта електронного навчання для моніторингу та діагностики чи об’єкта електронного навчання з результатами навчання. Ці об'єкти є базовими блоками для побудови педагогічних моделей. У даній роботі детально розглянуто питання створення педагогічних моделей з використанням чотирьох типів педагогічних об’єктів електронного навчання. Представлений приклад педагогічної моделі призначений для досягнення певних освітніх цілей. Приклади моделей, створених з використанням певних об’єктів електронного навчання, представляють навчальні блоки, які використовуються залежно від контексту певної педагогічної ситуації. Педагогічні об’єкти електронного навчання та педагогічні моделі, призначені для їх застосування як засоби навчання в адаптивному середовищі електронного навчання, застосовуються в Moodle LMS відповідно до тенденції програмного забезпечення з метою допомоги викладачу або заміни деяких його функцій, а роль викладача піднімається на більш високий організаційний, педагогічний та методичний рівень. За допомогою педагогічних об’єктів електронного навчання створено три приклади педагогічних моделей: «Ранній зворотний зв’язок», «Сендвіч - метод зворотного зв’язку» та «Послідовна метафора» в LMS Moodle, які були апробовані в навчальному курсі «Моделювання навчальних курсів у Moodle» під час осіннього триместру 2021/2022 навчального року на факультеті математики та інформатики Пловдивського університету “Паїсій Хілендарський”, Болгарія.Software platforms for e-learning support various options for presenting educational content. One of the ways to organize and structure it is via so-called pedagogical patterns. They are a method for describing and sharing knowledge and practical experience. Pedagogical patterns are used to describe pedagogical situations that occur repeatedly in the learning process. In the context of e-learning systems, there are various approaches to digitalization of pedagogical patterns. The purpose of the paper is to show how to build instances of pedagogical patterns using e-learning pedagogical objects, which can be easily and conveniently used as models in an adaptive e-learning environment. An e-learning pedagogical object is an abstract concept that can be presented in the concrete form of an e-learning object, an e-learning methodological object, an e-learning object for monitoring and diagnostics or an e-learning object with learning outcomes. These objects are building blocks for constructing instances of pedagogical patterns. This paper thoroughly discusses the issue of creating instances of pedagogical patterns of the four types of e-learning pedagogical objects. The instance of a pedagogical pattern is meant to serve to create subsections of educational topics. The instances of the patterns built of e-learning objects are learning units that are used depending on the context of a particular pedagogical situation. The e-learning pedagogical objects and the pedagogical pattern instances are intended to be applied in an adaptive e-learning environment as teaching aids. Their theoretical models are applied in Moodle LMS in line with the tendency for software to assist and replace some of the teacher functions, while the teacher’s role is raised to a higher organizational, pedagogical and methodological level. Three instances of pedagogical patterns have been created through e-learning pedagogical objects: “Early Feedback”, “Feedback Sandwich” and “Consistent Metaphor” in LMS Moodle, which have been tested in the training course “Modeling of training courses in Moodle” during the autumn trimester of the academic year 2021/2022 at the Faculty of Mathematics and Informatics of Paisii Hilendarski University of Plovdiv, Bulgaria

Assessing the effectiveness of sequence diagrams in the comprehension of functional requirements: results from a family of five experiments

Modeling is a fundamental activity within the requirements engineering process and concerns the construction of abstract descriptions of requirements that are amenable to interpretation and validation. The choice of a modeling technique is critical whenever it is necessary to discuss the interpretation and validation of requirements. This is particularly true in the case of functional requirements and stakeholders with divergent goals and different backgrounds and experience. This paper presents the results of a family of experiments conducted with students and professionals to investigate whether the comprehension of functional requirements is influenced by the use of dynamic models that are represented by means of the UML sequence diagrams. The family contains five experiments performed in different locations and with 112 participants of different abilities and levels of experience with UML. The results show that sequence diagrams improve the comprehension of the modeled functional requirements in the case of high ability and more experienced participants.The authors wish to thank all the participants in the experiments. This research was partially supported by the MULTIPLE project (with ref. TIN2009-13838).Abrahao Gonzales, SM.; Gravino, .C.; Insfrán Pelozo, CE.; Scaniello, .G.; Tortora, .G. (2013). Assessing the effectiveness of sequence diagrams in the comprehension of functional requirements: results from a family of five experiments. IEEE Transactions on Software Engineering. 39(3):327-342. https://doi.org/10.1109/TSE.2012.27S32734239

The Usability of E-learning Platforms in Higher Education: A Systematic Mapping Study

The use of e-learning in higher education has increased significantly in recent years, which has led to several studies being conducted to investigate the usability of the platforms that support it. A variety of different usability evaluation methods and attributes have been used, and it has therefore become important to start reviewing this work in a systematic way to determine how the field has developed in the last 15 years. This paper describes a systematic mapping study that performed searches on five electronic libraries to identify usability issues and methods that have been used to evaluate e-learning platforms. Sixty-one papers were selected and analysed, with the majority of studies using a simple research design reliant on questionnaires. The usability attributes measured were mostly related to effectiveness, satisfaction, efficiency, and perceived ease of use. Furthermore, several research gaps have been identified and recommendations have been made for further work in the area of the usability of online learning

Assessing the usability of virtual learning environments in higher education

Context: E-learning is an integral part of the modern higher education system, and therefore it is essential that students and staff are able to use systems that support E-learning, such as Virtual Learning Environments (VLEs), effectively. Usability is essential to ensure effective use of these systems and is often assessed by means of subjective questions. Although developed mainly for industry use, the Technology Acceptance Model (TAM) and System Usability Scale (SUS) questionnaire are often used to assess E-learning systems. Goal: The main goal of this thesis is to assess the usability of a VLE platform currently used in universities (Blackboard) and identify the most common and appropriate methods used to assess a VLE platform’s usability. Another aim is to investigate whether there are extensions to common usability models and methods (such as the SUS and TAM) that could improve their accuracy, including the potential of combining them with more objective measures such as number of clicks, time taken and open-ended questions. Method: The literature on VLE usability evaluation was reviewed using a mapping study methodology to identify the usability methods and factors that have been used previously. Informed by the findings of this study, a set of usability questionnaires have been developed, used and evaluated, with 101 student respondents recruited from all the Schools at Keele University participating in the first study (Chapter Four) and 162 in the second study (Chapter Five). A standard usability questionnaire and a novel form of observation were then combined to record 25 participants’ interactions with the VLE (Chapter Six) while they completed a set of representative tasks in two sessions that were held eight weeks apart. These interactions were then compared. Results: The results indicate that the VLE performed below the average usability expectation score (SUS score of 62.52) but is still considered as ‘acceptable’. Twenty-seven free text responses were also obtained in the first study and a thematic analysis of comments revealed very negative views of the VLE as well as areas for improvement. In the second study, it was found that perceived enjoyment (PE) and usability were jointly related to the perceived usefulness (PU), although the association was relatively weak. Perceived enjoyment and learnability were jointly associated with perceived ease of use (PEOU), with the association accounting for 39% of the variation in PEOU. Usability was related to PE but learnability was not. Overall, the original TAM can be improved by the addition of learnability, PE and usability as they have a positive effect on TAM. In the final study, the task success rate was relatively high (i.e. 82.3% in session 1); however, an average participant took 3.6 times longer to complete the set of tasks than a competent user. Furthermore, task time, clicks and success rate improved only marginally in the second session (which was at the end of the semester). However, when compared with the analysis of the results from the standard usability questionnaires (subjective measures), participants stated that they were satisfied with the usability of the system, contradicting the objective measures (number of clicks, task time and success rate). Conclusions: Using subjective measures alone, in the form of standard usability questionnaires, to assess the usability of a complex system can conceal significant issues. Usability assessment should therefore be based on actual performance against a defined baseline and combined with forms of qualitative feedback such as free text responses. Evaluating the effect of usability on E-learning is complicated. The studies conducted in this thesis have provided valuable guidance on how to measure the usability of VLEs. Suggestions for future work on the usability of VLEs as well as appropriate recommendations are provided