A Comparison of Quantitative and Qualitative Data from a Formative Usability Evaluation of an Augmented Reality Learning Scenario

The proliferation of augmented reality (AR) technologies creates opportunities for the devel-opment of new learning scenarios. More recently, the advances in the design and implementation of desktop AR systems make it possible the deployment of such scenarios in primary and secondary schools. Usability evaluation is a precondition for the pedagogical effectiveness of these new technologies and requires a systematic approach for finding and fixing usability problems. In this paper we present an approach to a formative usability evaluation based on heuristic evaluation and user testing. The basic idea is to compare and integrate quantitative and qualitative measures in order to increase confidence in results and enhance the descriptive power of the usability evaluation report.augmented reality, multimodal interaction, e-learning, formative usability evaluation, user testing, heuristic evaluation

Specification and Validation of a Formative Index to Evaluate the Ergonomic Quality of an AR-based Educational Platform

The ergonomic quality of educational systems is a key feature influencing both the usefulness and motivation for the learner. Desktop Augmented Reality (AR) systems are featuring specific interaction techniques that may create additional usability issues affecting the perceived ease of use. Measuring key usability aspects and understanding the causal relationships between them is a challenge that requires formative measurement models specification and validation. In this paper we present an evaluation instrument based on two main formative indexes that are capturing specific usability measures for two AR-based applications. The formative indexes are forming a second order formative construct that acts as predictor for both the general ease of use and ease of learning how to operate with the application

25 Years of Model-Driven Web Engineering : What we achieved, what is missing

Model-Driven Web Engineering (MDWE) approaches aim to improve the Web applications development process by focusing on modeling instead of coding, and deriving the running application by transformations from conceptual models to code. The emergence of the Interaction Flow Modeling Language (IFML) has been an important milestone in the evolution of Web modeling languages, indicating not only the maturity of the field but also a final convergence of languages. In this paper we explain the evolution of modeling and design approaches since the early years (the 90’s) detailing the forces which drove that evolution and discussing the strengths and weaknesses of some of those approaches. A brief presentation of IFML is accompanied with a thorough analysis of the most important achievements of the MDWE community as well as the problems and obstacles that hinder the dissemination of model-driven techniques in the Web engineering field.Laboratorio de Investigación y Formación en Informática Avanzada (LIFIA

Providing end-user facilities to simplify ontology-driven web application authoring

This is the author’s version of a work that was accepted for publication in Interacting with Computers. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Interacting with Computers, Interacting with Computers 17, 4 (2007) DOI: 10.1016/j.intcom.2007.01.006Generally speaking, emerging web-based technologies are mostly intended for professional developers. They pay poor attention to users who have no programming abilities but need to customize software applications. At some point, such needs force end-users to act as designers in various aspects of software authoring and development. Every day, more new computing-related professionals attempt to create and modify existing applications in order to customize web-based artifacts that will help them carry out their daily tasks. In general they are domain experts rather than skilled software designers, and new authoring mechanisms are needed in order that they can accomplish their tasks properly. The work we present is an effort to supply end-users with easy mechanisms for authoring web-based applications. To complement this effort, we present a user study showing that it is possible to carry out a trade-off between expressiveness and ease of use in order to provide end-users with authoring facilities.The work reported in this paper is being partially supported by the Spanish Ministry of Science and Technology (MCyT), projects TIN2005-06885 and TSI2005-08225-C07-06

25 Years of Model-Driven Web Engineering : What we achieved, what is missing

Model-Driven Web Engineering (MDWE) approaches aim to improve the Web applications development process by focusing on modeling instead of coding, and deriving the running application by transformations from conceptual models to code. The emergence of the Interaction Flow Modeling Language (IFML) has been an important milestone in the evolution of Web modeling languages, indicating not only the maturity of the field but also a final convergence of languages. In this paper we explain the evolution of modeling and design approaches since the early years (the 90’s) detailing the forces which drove that evolution and discussing the strengths and weaknesses of some of those approaches. A brief presentation of IFML is accompanied with a thorough analysis of the most important achievements of the MDWE community as well as the problems and obstacles that hinder the dissemination of model-driven techniques in the Web engineering field.Laboratorio de Investigación y Formación en Informática Avanzada (LIFIA

Model-Driven Development of Interactive Multimedia Applications

The development of highly interactive multimedia applications is still a challenging and complex task. In addition to the application logic, multimedia applications typically provide a sophisticated user interface with integrated media objects. As a consequence, the development process involves different experts for software design, user interface design, and media design. There is still a lack of concepts for a systematic development which integrates these aspects. This thesis provides a model-driven development approach addressing this problem. Therefore it introduces the Multimedia Modeling Language (MML), a visual modeling language supporting a design phase in multimedia application development. The language is oriented on well-established software engineering concepts, like UML 2, and integrates concepts from the areas of multimedia development and model-based user interface development. MML allows the generation of code skeletons from the models. Thereby, the core idea is to generate code skeletons which can be directly processed in multimedia authoring tools. In this way, the strengths of both are combined: Authoring tools are used to perform the creative development tasks while models are used to design the overall application structure and to enable a well-coordinated development process. This is demonstrated using the professional authoring tool Adobe Flash. MML is supported by modeling and code generation tools which have been used to validate the approach over several years in various student projects and teaching courses. Additional prototypes have been developed to demonstrate, e.g., the ability to generate code for different target platforms. Finally, it is discussed how models can contribute in general to a better integration of well-structured software development and creative visual design

Une méthode d'inspection automatique de recommandations ergonomiques tout au long du processus de conception des applications Web

Les applications Web actuelles offrent de plus en plus de services. Pour éviter les difficultés d'usage de ces applications, l'utilisabilité doit être assurée. L'évaluation de l'utilisabilité est une tâche qui requiert une expertise en ergonomie logicielle. Cette expertise peut être capitalisée sous forme de recommandations qui sont l'expression d'une connaissance en ergonomie et qui vont aider à l'évaluation. Toutefois, puisqu'il est nécessaire d'appliquer celles-ci de manière systématique, leur inspection manuelle peut s'avérer laborieuse. Pour ne pas être limité par l'inspection manuelle, des outils ont été développés pour guider et supporter l'inspection automatique. Un des avantages de ces outils est que les connaissances en ergonomie y sont directement intégrées. De plus, le manque d'experts et le coût élevé des autres méthodes d'évaluation font que l'inspection automatique est une méthode adaptée. Cependant, ces outils ne peuvent évaluer que l'application finale et si des erreurs sont détectées, des modifications importantes de l'application peuvent avoir lieu. Dans cette thèse nous proposons une méthode d'évaluation basée sur modèles permettant de s'assurer tout au long du cycle de vie de l'utilisabilité des applications Web développées. Nous avons établi une ontologie qui organise les recommandations autour des éléments de l'interface pour identifier précisément quels éléments évaluer à chaque étape du cycle de vie. Cette ontologie est exploitée pour vérifier l'utilisabilité sur les différents artefacts produits dans le cycle de vie. Ces travaux ont été appliqués et validés à l'échelle industrielle sur une plateforme de développement de téléservices.The increasing use of the Web as a software platform together with the advance of technology has promoted Web applications as a start point for delivering information and services. Facing to the ever growing number of users, usability became a major requirement for the universal access of Web applications. In the last years, a number of evaluation methods have been developed by researchers, practitioners and Information Technology companies to help organizations to identify and to fix usability problems. However, usability evaluation of Web sites is not a straightforward process. On one hand, usability evaluation requires some knowledge and expertise in software ergonomics. One the other hand, due to constant evolution of Web application, it is required frequent evaluations to make sure that content updates do not introduce new usability problems. In order to overcome these limitations, much effort has been devoted in the development of tools for automating the inspection of usability and accessibility guidelines. However, tools currently available can only evaluate the final applications. In this thesis we propose a model-based evaluation method that allows ensuring the ergonomic quality of Web applications throughout the lifecycle. We developed an ontology that organizes recommendations around interface elements of a Web application and that allows identifying precisely what elements to evaluate at each phase of the lifecycle. We exploit this ontology to assess the various artifacts produced throughout the lifecycle. This work has been applied and validated at an industrial scale on an e-service development platform

A pattern-oriented and model-driven architecture for interactive systems

Day-to-day experiences suggest that it is not enough to approach a complex design equipped with design tips, guidelines, and hints. Developers must also be able to use proven solutions emerging from the best design practices to solve new design challenges. Without these, the designer is unable to properly apply guidelines or take full advantage of the power of technology, resulting therefore in poor performance, poor scalability, and poor usability. Furthermore, the designer might "reinvent the wheel" when attempting to implement a design solution. A number of design problems continue to arise, such as: (1) decoupling the various aspects of interactive systems (for example, business logic, the UI, navigation, and information architecture) and (2) isolating platform specifics from the concerns common to all interactive systems. In the context of a proposal for a Pattern-Oriented and Model-driven Architecture (POMA) for interactive systems, this thesis identifies an extensive list of pattern categories and types of models aimed at providing a pool of proven solutions to these problems. The models of patterns span several levels of abstraction, such as domain, task, dialog, presentation and layout. The proposed POMA architecture illustrates how several individual models can be combined at different levels of abstraction into heterogeneous structures which can then be used as building blocks in the development of interactive systems. This document is divided into six chapters: the first chapter presents a background and related work on "Patterns" in general and on various architectures for interactive systems development such as "N-tiers architectures", "Pattern-Oriented Design" (POD), "Pattern- Supported Approach" (PSA), and "Model-Driven Architecture" (MDA). The second chapter introduces the research topic with its objectives, its limits, the research methodology, and research steps. The third chapter describes primarily the most important parts of the research which is the development of a new architecture called Pattern-Oriented and Model-Driven Architecture, facilitating the development of interactive systems including fundamentals and key concepts, an overview, justifications versus N-tiers, POD, PSA, and MDA architectures and specifications. The fourth chapter describes architectural levels and categories of patterns used in POMA. The fifth chapter describes the categories of models used in POMA. The sixth chapter presents an exploratory case study applied to the architecture proposed in this research. The last chapter presents a conclusion on this research work and its expected evolution in the future