Empirical validation of a usability inspection method for model-driven Web development

Web applications should be usable in order to be accepted by users and to improve their success probability. Despite the fact that this requirement has promoted the emergence of several usability evaluation methods, there is a need for empirically validated methods that provide evidence about their effectiveness and that can be properly integrated into early stages of Web development processes. Model-driven Web development processes have grown in popularity over the last few years, and offer a suitable context in which to perform early usability evaluations due to their intrinsic traceability mechanisms. These issues have motivated us to propose a Web Usability Evaluation Process (WUEP) which can be integrated into model-driven Web development processes. This paper presents a family of experiments that we have carried out to empirically validate WUEP. The family of experiments was carried out by 64 participants, including PhD and Master¿s computer science students. The objective of the experiments was to evaluate the participants¿ effectiveness, efficiency, perceived ease of use and perceived satisfaction when using WUEP in comparison to an industrial widely used inspection method: Heuristic Evaluation (HE). The statistical analysis and meta-analysis of the data obtained separately from each experiment indicated that WUEP is more effective and efficient than HE in the detection of usability problems. The evaluators were also more satisfied when applying WUEP, and found it easier to use than HE. Although further experiments must be carried out to strengthen these results, WUEP has proved to be a promising usability inspection method for Web applications which have been developed by using model-driven development processes.The authors would like to thank all the participants in the experiments, along with the usability experts that supported certain tasks of the evaluation design stage, and of which the control group was composed. This research work is funded by the MULTIPLE project (TIN2009-13838) and the FPU program (AP2007-03731) from the Spanish Ministry of Science and Education.Fernández Martínez, A.; Abrahao Gonzales, SM.; Insfrán Pelozo, CE. (2013). Empirical validation of a usability inspection method for model-driven Web development. Journal of Systems and Software. 86(1):161-186. https://doi.org/10.1016/j.jss.2012.07.043S16118686

Effects of Relationship Quality on Citizen Intention Use of E-government Services: An Empirical Study of E-government System

This study aims to explain the concept of relationship quality on citizen intention use of e-Government service. We have built 5 hypotheses. The result of 5 hypotheses is positive and significant impact. In building citizen relationship, we build two antecedent they are perceived e-Government usability and perceived e-Government credibility. The method we use in this research is empirical, with 366 valid respondents

Usability Inspection in Model-Driven Web Development: Empirical Validation in WebML

There is a lack of empirically validated usability evaluation methods that can be applied to models in model-driven Web development. Evaluation of these models allows an early detection of usability problems perceived by the end-user. This motivated us to propose WUEP, a usability inspection method which can be integrated into different model-driven Web development processes. We previously demonstrated how WUEP can effectively be used when following the Object-Oriented Hypermedia method. In order to provide evidences about WUEP’s generalizability, this paper presents the operationalization and empirical validation of WUEP into another well-known method: WebML. The effectiveness, efficiency, perceived ease of use, and satisfaction of WUEP were evaluated in comparison to Heuristic Evaluation (HE) from the viewpoint of novice inspectors. The results show that WUEP was more effective and efficient than HE when detecting usability problems on models. Also, inspectors were satisfied when applying WUEP, and found it easier to use than HE.Fernández Martínez, A.; Abrahao Gonzales, SM.; Insfrán Pelozo, CE.; Matera, M. (2013). Usability Inspection in Model-Driven Web Development: Empirical Validation in WebML. Lecture Notes in Computer Science. 8107:740-756. doi:10.1007/978-3-642-41533-3_457407568107Abrahão, S., Iborra, E., Vanderdonckt, J.: Usability Evaluation of User Interfaces Generated with a Model-Driven Architecture Tool. In: Maturing Usability: Quality in Software, Interaction and Value, pp. 3–32. Springer (2007)Atterer, R., Schmidt, A.: Adding Usability to Web Engineering Models and Tools. In: Lowe, D.G., Gaedke, M. (eds.) ICWE 2005. LNCS, vol. 3579, pp. 36–41. Springer, Heidelberg (2005)Basili, V., Rombach, H.: The TAME Project: Towards Improvement-Oriented Software Environments. IEEE Transactions on Software Engineering 14(6), 758–773 (1988)Briand, L., Labiche, Y., Di Penta, M., Yan-Bondoc, H.: An experimental investigation of formality in UML-based development. IEEE TSE 31(10), 833–849 (2005)Carifio, J., Perla, R.: Ten Common Misunderstandings, Misconceptions, Persistent Myths and Urban Legends about Likert Scales and Likert Response Formats and their Antidotes. Journal of Social Sciences 3(3), 106–116 (2007)Ceri, S., Fraternali, P., Bongio, A.: Web modeling language (WebML): a modeling language for designing Web sites. In: 9th International World Wide Web Conference, pp. 137–157 (2000)Ceri, S., Fraternali, P., Acerbis, R., Bongio, A., Butti, S., Ciapessoni, F., Conserva, C., Elli, R., Greppi, C., Tagliasacchi, M., Toffetti, G.: Architectural issues and solutions in the development of data-intensive Web applications. In: Proceedings of the 1st Biennial Conference on Innovative Data Systems Research, Asilomar, CA (2003)Conte, T., Massollar, J., Mendes, E., Travassos, G.H.: Usability Evaluation Based on Web Design Perspectives. In: Proceedings of the International Symposium on Empirical Software Engineering and Measurement (ESEM 2007), pp. 146–155 (2007)Fernandez, A., Insfran, E., Abrahão, S.: Usability evaluation methods for the Web: a systematic mapping study. Information and Software Technology 53, 789–817 (2011)Fernandez, A., Abrahão, S., Insfran, E.: A Web usability evaluation process for model-driven Web development. In: Mouratidis, H., Rolland, C. (eds.) CAiSE 2011. LNCS, vol. 6741, pp. 108–122. Springer, Heidelberg (2011)Fernandez, A., Abrahão, S., Insfran, E., Matera, M.: Further Analysis on the Validation of a Usability Inspection Method for Model-Driven Web Development. In: 6th International Symposium on Empirical Software Engineering and Measurement (ESEM 2012), pp. 153–156 (2012)Fernandez, A., Abrahão, S., Insfran, E.: Empirical Validation of a Usability Inspection Method for Model-Driven Web Development. Journal of Systems and Software 86, 161–186 (2013)Fraternali, P., Matera, M., Maurino, A.: WQA: an XSL Framework for Analyzing the Quality of Web Applications. In: Proceedings of IWWOST 2002 - ECOOP 2002 Workshop, Malaga, Spain (2002)Hornbæk, K.: Dogmas in the assessment of usability evaluation methods. Behaviour & Information Technology 29(1), 97–111 (2010)Hwang, W., Salvendy, G.: Number of people required for usability evaluation: the 10±2 rule. Communications of the ACM 53(5), 130–113 (2010)International Organization for Standardization: ISO/IEC 25000, Software Engineering – Software Product Quality Requirements and Evaluation (SQuaRE) – Guide to SQuaRE (2005)Juristo, N., Moreno, A.M.: Basics of Software Engineering Experimentation. Kluwer Academic Publishers (2001)Juristo, N., Moreno, A., Sanchez-Segura, M.I.: Guidelines for eliciting usability functionalities. IEEE Transactions on Software Engineering 33(11), 744–758 (2007)Matera, M., Costabile, M.F., Garzotto, F., Paolini, P.: SUE inspection: an effective method for systematic usability evaluation of hypermedia. IEEE Transactions on Systems, Man, and Cybernetics, Part A 32(1), 93–103 (2002)Matera, M., Rizzo, F., Carughi, G.: Web Usability: Principles and Evaluation Methods. In: Web Engineering, pp. 143–180. Springer (2006)Maxwell, K.: Applied Statistics for Software Managers. Software Quality Institute Series. Prentice Hall (2002)Molina, F., Toval, A.: Integrating usability requirements that can be evaluated in design time into Model Driven Engineering of Web Information Systems. Advances in Engineering Software 40(12), 1306–1317 (2009)Moreno, N., Vallecillo, A.: Towards interoperable Web engineering methods. Journal of the American Society for Information Science and Technolog 59(7), 1073–1092 (2008)Neuwirth, C.M., Regli, S.H.: IEEE Internet Computing Special Issue on Usability and the Web 6(2) (2002)Nielsen, J.: Heuristic evaluation. In: Usability Inspection Methods. John Wiley & Sons, NY (1994)Offutt, J.: Quality attributes of Web software applications. IEEE Software: Special Issue on Software Engineering of Internet Software, 25–32 (2002)Panach, I., Condori, N., Valverde, F., Aquino, N., Pastor, O.: Understandability measurement in an early usability evaluation for MDD. In: International Symposium on Empirical Software Engineering (ESEM 2008), pp. 354–356 (2008)Webratio. Success stories, Online article, http://www.webratio.com/portal/content/en/success-storiesWohlin, C., Runeson, P., Host, M., Ohlsson, M.C., Regnell, B., Weslen, A.: Experimentation in Software Engineering - An Introduction. Kluwer (2000

Cognitive Analysis of Intrusion Detection System

Usability evaluation methods have gained a substantial attention in networks particularly in Intrusion Detection System (IDS) as these evaluation methods are envisioned to achieve usability and define usability defects for a large number of practical software’s. Despite a good number of available survey and methods on usability evaluation, we feel that there is a gap in existing literature in terms of usability evaluation methods, IDS interfaces and following usability guidelines in IDS development. This paper reviews the state of the art for improving usability of networks that illustrates the issues and challenges in the context of design matters. Further, we propose the taxonomy of key issues in evaluation methods and usability problems. We also define design heuristics for IDS users and interfaces that improves detection of usability defects and interface usability compared to conventional evaluation heuristics. The similarities and differences of usability evaluation methods and usability problems are summarized on the basis of usability factors, current evaluation methods and interfaces loopholes

From GenderMag to InclusiveMag: An Inclusive Design Meta-Method

How can software practitioners assess whether their software supports diverse users? Although there are empirical processes that can be used to find “inclusivity bugs” piecemeal, what is often needed is a systematic inspection method to assess software’s support for diverse populations. To help fill this gap, this paper introduces InclusiveMag, a generalization of GenderMag that can be used to generate systematic inclusiveness methods for a particular dimension of diversity. We then present a multicase study covering eight diversity dimensions, of eight teams’ experiences applying InclusiveMag to eight under-served populations and their “mainstream” counterparts

A Proposal for Modelling Usability in a Holistic MDD Method

Holistic methods for Model-Driven Development (MDD) aim to model all the system features in a conceptual model. This conceptual model is the input for a model compiler that can generate software systems by means of automatic transformations. However, in general, MDD methods focus on modelling the structure and functionality of systems, relegating the interaction and usability features to manual implementations at the last steps of the software development process. Some usability features are strongly related to the functionality of the system and their inclusion is not so easy. In order to facilitate the inclusion of functional usability features from the first steps of the development process and bring closer MDD methods to the holistic perspective, we propose a Usability Model. The Usability Model gathers conceptual primitives that represent functional usability features in a sufficiently abstract way so that the model can be used with different holistic MDD methods. This paper defines all the primitives that can be used to represent functional usability features. Moreover, we have defined a process to include the Usability Model in any MDD method without affecting its existing conceptual model. The proposal is based on model-to-model and model-to-code transformations. As proof of concept, we have applied our proposal to an existing MDD method called the OO-method and we have measured its efficiency. (C) 2013 Elsevier B.V. All rights reserved.This work has been developed with the support of MICINN (PROS-Req TIN2010-19130-C02-02), UV (UV-INV-PRECOMP13-115032), GVA (ORCA PROMETEO/2009/015), and co-financed with ERDF. We acknowledge the support of the ITEA2 Call 3 UsiXML (20080026) and funding by the MITYC (TSI-020400-2011-20).Panach Navarrete, JI.; Aquino Salvioni, N.; Pastor López, O. (2014). A Proposal for Modelling Usability in a Holistic MDD Method. Science of Computer Programming. 86:74-88. https://doi.org/10.1016/j.scico.2013.06.008S74888

Adaptive model for web engineering methods to develop multi web applications in agile environment

Model Driven Web Engineering (MDWE) is an application of a model driven paradigm in the domain of web software development. MDWE is helpful because technologies and platforms of web applications constantly evolve into Web Engineering Methods (WEMs). The evolution of web applications has consequently introduced new features and challenges, therefore existing WEMs need to be improved. These WEMs have failed to develop modern web applications’ features. Furthermore, no single WEM is capable of covering the whole lifecycle phases. These issues decrease the usability. In addition, the Interaction Flow Modeling Language (IFML) as a recent method has also not been able to address them. This thesis developed a new WEM, Useable Adaptive Agile IFML (UAA-IFML) to solve these issues in several steps. In this research, mixed methods used were qualitative and quantitative methodologies. In the first step, a new adaptive model was defined for supporting the features of multi-web applications. The new model was developed via an adaptive model into the IFML metamodels known as Adaptive IFML (AIFML). In the second step, IFML was enriched through MockupDD for covering lifecycle, known as Agile IFML (A-IFML). This is because MockupDD provides an agile environment, hence agile lifecycle can solve the lifecycle issue. In the third step, a new adaptive model and agile process were combined as Adaptive Agile IFML (AAIFML). This integration increased the usability of the IFML method. In presenting the usability of AA-IFML, experimentation of the framework was extended to evaluate the usability of WEMs. Besides, feedbacks on the usability of AA-IFML were obtained from developers around the world using three instruments, namely performing tasks, answering questionnaires, and interviewing experts. Analysis on the feedback indicated a 20% improvement usability of the AA-IFML compared with current IFML. The findings have shown that the UAA-IFML is beneficial for developers, as they would only need to use one method to design modern web application features in the whole lifecycle phases

Usability Improvement Through A/B Testing and Refactoring

Usability evaluation is an essential task in web application development. There have been several attempts to integrate user-centered design with agile methods, but it is hard to synchronize their practices. User testing is very valuable to learn from feedback of actual use, but it remains expensive to find and solve usability problems. Furthermore, the high cost of usability evaluation forces small/medium-sized companies to trust the first solution applied, without actually testing the success of the solution or considering a possible regression in usability, as could be highlighted by an iterative testing method. In this article we advocate for a usability improvement cycle oriented by user feedback, and compatible with an agile development process. We propose an iterative method supported by a toolkit that allows usability experts to design user tests, run them remotely, analyze results, and assess alternative solutions to usability problems similarly to A/B testing. Each solution is created by applying client-side web refactorings, i.e., changes to the web pages in the client which are meant to improve usability. The main benefit of our approach is that it reduces the overall cost of user testing and particularly, A/B testing, by applying refactorings to create alternative solutions without modifying the application’s server code. By making it affordable for usability experts to apply the method in parallel with the development cycle, we aim to encourage them to incorporate user feedback and try different ideas to discover the best performing solution in terms of the metrics of interest.Laboratorio de Investigación y Formación en Informática Avanzad

Meta-analysis for families of experiments in software engineering: a systematic review and reproducibility and validity assessment

ContextPrevious studies have raised concerns about the analysis and meta-analysis of crossover experiments and we were aware of several families of experiments that used crossover designs and meta-analysis.ObjectiveTo identify families of experiments that used meta-analysis, to investigate their methods for effect size construction and aggregation, and to assess the reproducibility and validity of their results.MethodWe performed a systematic review (SR) of papers reporting families of experiments in high quality software engineering journals, that attempted to apply meta-analysis. We attempted to reproduce the reported meta-analysis results using the descriptive statistics and also investigated the validity of the meta-analysis process.ResultsOut of 13 identified primary studies, we reproduced only five. Seven studies could not be reproduced. One study which was correctly analyzed could not be reproduced due to rounding errors. When we were unable to reproduce results, we provide revised meta-analysis results. To support reproducibility of analyses presented in our paper, it is complemented by the reproducer R package.ConclusionsMeta-analysis is not well understood by software engineering researchers. To support novice researchers, we present recommendations for reporting and meta-analyzing families of experiments and a detailed example of how to analyze a family of 4-group crossover experiments