Involving users in Automotive HMI design: Design evaluation of an interactive simulation based on participatory design

Abstract: User-centered design (UCD) methods for human-machine interfaces (HMI) have been a key to develop safe and user-friendly interaction for years. Especially in safety-critical domains like transportation, humans need to have clear instructions and feedback loops to safely interact with the vehicle. With the shift towards more automation on the streets, human-machine interaction needs to be predictable to ensure safe road interaction. Understanding human behavior and prior user needs in crucial situation can be significant in a multitude of complex interactions for in-vehicle passengers, pedestrians and other traffic participants.While research mostly focused on addressing user behavior and user needs, the inclusion of users has often been limited to study participants with behavioral inputs or interviewees prompted for opinions. Although users do not have the knowledge and experience as professional designers and experts to create a product for others alone, unbiased insights into the future target groups’ mental models are a valuable and necessary asset. Hence, with stronger user participation and appropriate tools for users to design prototypes, the design process may deeper involve all type of stakeholders helping to provide insights into their mental models to understand user need and expectation.To extend current UCD practices in the development of automotive HMIs, our work introduces a user-interactive approach, based on the principles of participatory design (PD), to enable users to actively create and work within design process. A within-subject study was conducted based on evaluating users’ trust within an interaction with an AV and subsequently configuring the corresponding HMI. The scenario focuses on the interaction between a pedestrian (user’s point of view) deciding to cross path with an automated vehicle (AV, SAE L4). The AV would show its intention via a 360-degree light band HMI on its roof. The interactive simulation offered users hands-on options to iteratively experience, evaluate and improve HMI elements within changeable environmental settings (i.e., weather, daytime) until they were satisfied with the result. The addition of participation was provided by an interface using common visual user interface elements, i.e. sliders and buttons, giving users a range of variety for real-time HMI configuring.A first prototype of this interactive simulation was tested for the safety-critical use-case in a usability study (N=29). Results from questionnaires and interviews show high usability acceptance of the interactive simulation among participants as assessed by the system usability scale. Overall usability was rated high (System Usability Scale) and frustration low (NASA-TLX raw). Moreover, the interactive simulation was rated to have above average user experience (User Experience Questionnaire). Appended feedback interviews gave valuable insights on improving the simulation user interface, offering different design opportunities within the simulation and a wider parameter space. The short design session time shows the limit of customizability options within this study but needs to be further investigated to determine optimal range for longer evaluation and design sessions. Based on the study results, further requirements for PD simulative environments to assess limits for parameter spaces in virtual environments are derived

Towards a kansei-based user modeling methodology for eco-design

We propose here to highlight the benefits of building a framework linking Kansei Design (KD), User Centered Design (UCD) and Eco-design, as the correlation between these fields is barely explored in research at the current time. Therefore, we believe Kansei Design could serve the goal of achieving more sustainable products by setting up an accurate understanding of the user in terms of ecological awareness, and consequently enhancing performance in the Eco-design process. In the same way, we will consider the means-end chain approach inspired from marketing research, as it is useful for identifying ecological values, mapping associated functions and defining suitable design solutions. Information gathered will serve as entry data for conducting scenario-based design, and supporting the development of an Eco-friendly User Centered Design methodology (EcoUCD).ANR-ECOUS

Usability evaluation of a virtual museum interface

The Augmented Representation of Cultural Objects (ARCO) system provides software and interface tools to museum curators to develop virtual museum exhibitions, as well as a virtual environment for museum visitors over the World Wide Web or in informative kiosks. The main purpose of the system is to offer an enhanced educative and entertaining experience to virtual museum visitors. In order to assess the usability of the system, two approaches have been employed: a questionnaire based survey and a Cognitive Walkthrough session. Both approaches employed expert evaluators, such as domain experts and usability experts. The result of this study shows a fair performance of the followed approach, as regards the consumed time, financial and other resources, as a great deal of usability problems has been uncovered and many aspects of the system have been investigated. The knowledge gathered aims at creating a conceptual framework for diagnose usability problems in systems in the area of Virtual Cultural Heritage

Teaching and learning in information retrieval

A literature review of pedagogical methods for teaching and learning information retrieval is presented. From the analysis of the literature a taxonomy was built and it is used to structure the paper. Information Retrieval (IR) is presented from different points of view: technical levels, educational goals, teaching and learning methods, assessment and curricula. The review is organized around two levels of abstraction which form a taxonomy that deals with the different aspects of pedagogy as applied to information retrieval. The first level looks at the technical level of delivering information retrieval concepts, and at the educational goals as articulated by the two main subject domains where IR is delivered: computer science (CS) and library and information science (LIS). The second level focuses on pedagogical issues, such as teaching and learning methods, delivery modes (classroom, online or e-learning), use of IR systems for teaching, assessment and feedback, and curricula design. The survey, and its bibliography, provides an overview of the pedagogical research carried out in the field of IR. It also provides a guide for educators on approaches that can be applied to improving the student learning experiences

Evaluating system utility and conceptual fit using CASSM

There is a wealth of user-centred evaluation methods (UEMs) to support the analyst in assessing interactive systems. Many of these support detailed aspects of use – for example: Is the feedback helpful? Are labels appropriate? Is the task structure optimal? Few UEMs encourage the analyst to step back and consider how well a system supports users’ conceptual understandings and system utility. In this paper, we present CASSM, a method which focuses on the quality of ‘fit’ between users and an interactive system. We describe the methodology of conducting a CASSM analysis and illustrate the approach with three contrasting worked examples (a robotic arm, a digital library system and a drawing tool) that demonstrate different depths of analysis. We show how CASSM can help identify re-design possibilities to improve system utility. CASSM complements established evaluation methods by focusing on conceptual structures rather than procedures. Prototype tool support for completing a CASSM analysis is provided by Cassata, an open source development

User-Based Data Collection Techniques and Strategies for Evaluating Networked Information Services

published or submitted for publicatio

Towards an evaluation framework for medical web applications

Copyright @ 2013 EMCIS.The main aim of this study is to review and analyse various evaluation frameworks used to assess the operational effectiveness of various Information Technology (IT) processes/applications and identify their strengths in order to form a new holistic framework for economic evaluation of web applications. This research aims to address the need for a new holistic evaluation framework for the purpose of the evaluation of the medical web applications. Over the last decade more and more, companies used accountancy techniques such as the frameworks analysed in this research. This new holistic framework that was developed will include also steps regarding the indirect and intangible costs and benefits identification and their incorporation in the evaluation process. Moreover the new emerging market of the medical websites and the embedded on them web applications requires also a new evaluation framework that will provide accurate results in the estimation of the efficiency of an investment on them. The paper first presents an introduction about why economic evaluation is important when evaluating the Information Technology in organizations. Various studies are reviewed, which highlight the ever increasing importance of integrating economic evaluation processes, such as Cost Benefit Analysis (CBA) and Return on Investment (ROI), into systems and processes of organizations and economic organizations, and analyse the factors that govern their role. Finally, the evaluation frameworks and methods that are found in these studies should be used as a part of a proactive systematic action plan that the organizations could use to avoid budget reduction due to incorrect planning. The next part of this study includes a comprehensive presentation and review of past frameworks used to evaluate Information Technology. The frameworks that are reviewed are the Framework for evaluation of information systems, the Information Technology Adoption Model (ITAM), the Total Evaluation and Acceptance Methodology (TEAM) framework, the ROI Process Model and HOT-fit evaluation framework for Health Information Systems