Development and validation of an integrated model for evaluating e-service quality, usability and user experience (e-SQUUX) of Web-based applications in the context of a University web portal

Text in EnglishDevelopments in Internet technology and pervasive computing over the past two and half decades have resulted in a variety of Web-based applications (WBAs) that provide products and services to online users or customers. The Internet is used not only to transfer information via the web but is increasingly used to provide electronic services including business transactions, information-delivery and social networking, as well as e-government, e-health and e-learning. For such organisations, e-service quality, usability and user experience are considered to be critical determinants of their products’ or services’ success. Many studies to model these three concepts separately have been undertaken as part of broader studies of software quality or service quality modelling. However, to the current researcher’s knowledge, none of the studies have focussed on proposing an evaluation model that integrates and combines the three of them. This research is an effort to fill that gap. The primary purpose of this mixed-methods research was to develop a conceptual integrated model for evaluating e-service quality, usability and user experience (e-SQUUX) of WBAs and then contextualise it to evaluation of a University web portal (UWP). This was undertaken using an exploratory sequential research design. During a qualitative phase, an extensive extensive systematic literature review of 264 relevant sources relating to dimensions of e-service quality, usability and user experience, was undertaken to derive an integrated conceptual e-service quality, usability and user experience (e-SQUUX) Model for evaluating WBAs. The model was then empirically refined through a sequential series of validations, thus developing various versions of the e-SQUUX Model. First, it was content validated by a set of four expert reviewers. Second, during the quantitative phase, in the context of a University web portal, a questionnaire survey was conducted that included a comprehensive pilot study with 29 partipants, prior to the main survey. The main survey data from 174 particiapants was used to determine a validated model, using Exploratory factor analysis (EFA), followed by producing a structural model, using partial least square – structural equation modelling (PLS-SEM). This version consisted of the components of the final e-SQUUX Model. Consequently, the research enriches the body of knowledge on IS and HCI by providing the e-SQUUX Model as an evaluation tool. For designers, developers and managers of UWPs, the model serves as a customisable set of evaluation criteria and also provides specific recommendations for design. In line with the Exploratory sequential design of mixed methods research, the findings of the qualitative work in this research influenced the subsequent quantitative study, since the potential Likert-scale questionnaire items were derived from the definitions and meanings of the components that emanated from the qualitative phase of the study. Consequently, this research is an exemplar for developing an integrated evaluation model for specific facets or domains, and of its application in a particular context, in this case, a University web portal. Keywords: e-service quality, usability, user experience, evaluation model, integrated model, exploratory factor analysis, partial least square – structural equation modelling (PLS-SEM), mixed methods research, Exploratory sequential design, quantitative study, qualitative study, validation, Web-based applications, University web portalInformation SystemPh D. (Information Systems

Usability evaluation of a web-based e-learning application: a study of two evaluation methods

Despite widespread use of web-based e-learning applications, insufficient attention is paid to their usability. There is a need to conduct evaluation using one or more of the various usability evaluation methods. Given that heuristic evaluation is known to be easy to use and cost effective, this study investigates the extent to which it can identify usability problems in a web-based e-learning application at a tertiary institution. In a comparative case study, heuristic evaluation by experts and survey evaluation among end users (learners) are conducted and the results of the two compared. Following literature studies in e-learning - particularly web-based learning - and usability, the researcher generates an extensive set of criteria/heuristics and uses it in the two evaluations. The object of evaluation is a website for a 3rd year Information Systems course. The findings indicate a high correspondence between the results of the two evaluations, demonstrating that heuristic evaluation is an appropriate, effective and sufficient usability evaluation method, as well as relatively easy to conduct. It identified a high percentage of usability problems.ComputingM.Sc. (Information Systems

Empirical validation of a usability inspection method for model-driven Web development

Web applications should be usable in order to be accepted by users and to improve their success probability. Despite the fact that this requirement has promoted the emergence of several usability evaluation methods, there is a need for empirically validated methods that provide evidence about their effectiveness and that can be properly integrated into early stages of Web development processes. Model-driven Web development processes have grown in popularity over the last few years, and offer a suitable context in which to perform early usability evaluations due to their intrinsic traceability mechanisms. These issues have motivated us to propose a Web Usability Evaluation Process (WUEP) which can be integrated into model-driven Web development processes. This paper presents a family of experiments that we have carried out to empirically validate WUEP. The family of experiments was carried out by 64 participants, including PhD and Master¿s computer science students. The objective of the experiments was to evaluate the participants¿ effectiveness, efficiency, perceived ease of use and perceived satisfaction when using WUEP in comparison to an industrial widely used inspection method: Heuristic Evaluation (HE). The statistical analysis and meta-analysis of the data obtained separately from each experiment indicated that WUEP is more effective and efficient than HE in the detection of usability problems. The evaluators were also more satisfied when applying WUEP, and found it easier to use than HE. Although further experiments must be carried out to strengthen these results, WUEP has proved to be a promising usability inspection method for Web applications which have been developed by using model-driven development processes.The authors would like to thank all the participants in the experiments, along with the usability experts that supported certain tasks of the evaluation design stage, and of which the control group was composed. This research work is funded by the MULTIPLE project (TIN2009-13838) and the FPU program (AP2007-03731) from the Spanish Ministry of Science and Education.Fernández Martínez, A.; Abrahao Gonzales, SM.; Insfrán Pelozo, CE. (2013). Empirical validation of a usability inspection method for model-driven Web development. Journal of Systems and Software. 86(1):161-186. https://doi.org/10.1016/j.jss.2012.07.043S16118686

The Usability of E-learning Platforms in Higher Education: A Systematic Mapping Study

The use of e-learning in higher education has increased significantly in recent years, which has led to several studies being conducted to investigate the usability of the platforms that support it. A variety of different usability evaluation methods and attributes have been used, and it has therefore become important to start reviewing this work in a systematic way to determine how the field has developed in the last 15 years. This paper describes a systematic mapping study that performed searches on five electronic libraries to identify usability issues and methods that have been used to evaluate e-learning platforms. Sixty-one papers were selected and analysed, with the majority of studies using a simple research design reliant on questionnaires. The usability attributes measured were mostly related to effectiveness, satisfaction, efficiency, and perceived ease of use. Furthermore, several research gaps have been identified and recommendations have been made for further work in the area of the usability of online learning

Effectiveness of heuristic evaluation in usability evaluation of elearning applications in higher educ

The Internet, World Wide Web (WWW) and e-learning are contributing to new forms of teaching and learning. Such environments should be designed and evaluated in effective ways, considering both usability- and pedagogical issues. The selection of usability evaluation methods (UEMs) is influenced by the cost of a methods and its effectiveness in addressing users’ issues. The issue of usability is vital in e-learning, where students cannot begin to learn unless they can first use the application. Heuristic evaluation (HE) remains the most widely-used usability evaluation method. This paper describes meta-evaluation research that investigated an HE of a web-based learning (WBL) application. The evaluations were based on a synthesised framework of criteria, related to usability and learning within WBL environments. HE was found to be effective in terms of the number and nature of problems identified in the target application by a complementary team of experienced experts. The findings correspond closely with those of a survey among learners

Knowledge Transfer in Science Education: The Case for Usability-Based Knowledge Visualization Guidelines

There is growing evidence that visualization aids knowledge transfer. However, the cases where learners have been actively involved as cocreators of knowledge visualization aids are limited. Furthermore, employing knowledge visualization for teaching and learning in high-school science have been proposed but empirical evidence of the effect on knowledge transfer is limited. The purpose of this study is to report on the knowledge transfer effect of applying usability-based knowledge visualization guidelines. A design-based research methodology guided by pragmatism was applied. The data capturing methods include a questionnaire-based survey, interviews and observations. The results suggest that the use of knowledge visualization can support knowledge transfer and the students’ learning experience in secondary school education, but more research is required to confirm this. The contribution of this paper is to add to the emerging discourse on the use of knowledge visualization for teaching and learning, and to report on how knowledge visualization guidelines can be used in practice.School of Computin