Spiraling Toward Usability: An Integrated Design Environment and Management System

Decades of innovation in designing usable (and unusable) interfaces have resulted in a plethora of guidelines, usability engineering methods, and other design tools. However, novice developers often have difficulty selecting and utilizing theory-based design tools in a coherent design process. This work introduces an integrated design environment and knowledge management system, LINK-UP. The central design record (CDR) module, provides tools to enable a guided, coherent development process. The CDR aims to prevent breakdowns occurring between design and evaluation phases both within the development team and during design knowledge reuse processes. We report on results from three case studies illustrating novice designers use of LINK-UP. A design knowledge IDE incorporating a CDR can help novice developers craft interfaces in a methodical fashion, while applying, verifying, and producing reusable design knowledge. Although LINK-UP supports a specific design domain, our IDE approach can transfer to other domains

Designing Attention-Centric Notification Systems: Five HCI Challenges

Through an examination of the emerging domain of cognitive systems, with a focus on attention-centric cognitive systems used for notification, this document explores the human-computer interaction challenges that must be addressed for successful interface design. This document asserts that with compatible tools and methods, user notification requirements and interface usability can be abstracted, expressed, and compared with critical parameter ratings; that is, even novice designers can assess attention cost factors to determine target parameter levels for new system development. With a general understanding of the user tasks supported by the notification system, a designer can access the repository of design knowledge for appropriate information and interaction design techniques (e.g., use of color, audio features, animation, screen size, transition of states, etc), which have analytically and empirically derived ratings. Furthermore, usability evaluation methods, provided to designers as part of the integrated system, are adaptable to specific combinations of targeted parameter levels. User testing results can be conveniently added back into the design knowledge repository and compared to target parameter levels to determine design success and build reusable HCI knowledge. This approach is discussed in greater detail as we describe five HCI challenges relating to cognitive system development: (1) convenient access to basic research and guidelines, (2) requirements engineering methods for notification interfaces, (3) better and more usable predictive modeling for pre-attentive and dual-task interfaces, (4) standard empirical evaluation procedures for notification systems, and (5) conceptual frameworks for organizing reusable design and software components. This document also describes our initial work toward building infrastructure to overcome these five challenges, focused on notification system development. We described LINK-UP, a design environment grounded on years of theory and method development within HCI, providing a mechanism to integrate interdisciplinary expertise from the cognitive systems research community. Claims allow convenient access to basic research and guidelines, while modules parallel a lifecycle development iteration and provide a process for requirements engineering guided by this basic research. The activities carried out through LINK-UP provide access to and interaction with reusable design components organized based on our framework. We think that this approach may provide the scientific basis necessary for exciting interdisciplinary advancement through many fields of design, with notification systems serving as an initial model. A version of this document will appear as chapter 3 in the book Cognitive Systems: Human Cognitive Models in Systems Design edited by Chris Forsythe, Michael Bernard, and Timothy Goldsmith resulting from a workshop led by the editors in summer 2003. The authors are grateful for the input of the workshop organizers and conference attendees in the preparation of this document

Human-computer interaction as science

Human-computer interaction (HCI) has had a long and troublesome relationship to the role of 'science'. HCI's status as an academic object in terms of coherence and adequacy is often in question---leading to desires for establishing a true scientific discipline. In this paper I explore formative cognitive science influences on HCI, through the impact of early work on the design of input devices. The paper discusses a core idea that I argue has animated much HCI research since: the notion of scientific design spaces. In evaluating this concept, I disassemble the broader 'picture of science' in HCI and its role in constructing a disciplinary order for the increasingly diverse and overlapping research communities that contribute in some way to what we call 'HCI'. In concluding I explore notions of rigour and debates around how we might reassess HCI's disciplinarity

Enhancing user experience and safety in the context of automated driving through uncertainty communication

Operators of highly automated driving systems may exhibit behaviour characteristic of overtrust issues due to an insufficient awareness of automation fallibility. Consequently, situation awareness in critical situations is reduced and safe driving performance following emergency takeovers is impeded. Previous research has indicated that conveying system uncertainties may alleviate these issues. However, existing approaches require drivers to attend the uncertainty information with focal attention, likely resulting in missed changes when engaged in non-driving-related tasks. This research project expands on existing work regarding uncertainty communication in the context of automated driving. Specifically, it aims to investigate the implications of conveying uncertainties under consideration of non-driving-related tasks and, based on the outcomes, develop and evaluate an uncertainty display that enhances both user experience and driving safety. In a first step, the impact of visually conveying uncertainties was investigated under consideration of workload, trust, monitoring behaviour, non-driving-related tasks, takeover performance, and situation awareness. For this, an anthropomorphic visual uncertainty display located in the instrument cluster was developed. While the hypothesised benefits for trust calibration and situation awareness were confirmed, the results indicate that visually conveying uncertainties leads to an increased perceived effort due to a higher frequency of monitoring glances. Building on these findings, peripheral awareness displays were explored as a means for conveying uncertainties without the need for focused attention to reduce monitoring glances. As a prerequisite for developing such a display, a systematic literature review was conducted to identify evaluation methods and criteria, which were then coerced into a comprehensive framework. Grounded in this framework, a peripheral awareness display for uncertainty communication was developed and subsequently compared with the initially proposed visual anthropomorphic uncertainty display in a driving simulator study. Eye tracking and subjective workload data indicate that the peripheral awareness display reduces the monitoring effort relative to the visual display, while driving performance and trust data highlight that the benefits of uncertainty communication are maintained. Further, this research project addresses the implications of increasing the functional detail of uncertainty information. Results of a driving simulator study indicate that particularly workload should be considered when increasing the functional detail of uncertainty information. Expanding upon this approach, an augmented reality display concept was developed and a set of visual variables was explored in a forced choice sorting task to assess their ordinal characteristics. Particularly changes in colour hue and animation-based variables received high preference ratings and were ordered consistently from low to high uncertainty. This research project has contributed a series of novel insights and ideas to the field of human factors in automated driving. It confirmed that conveying uncertainties improves trust calibration and situation awareness, but highlighted that using a visual display lessens the positive effects. Addressing this shortcoming, a peripheral awareness display was designed applying a dedicated evaluation framework. Compared with the previously employed visual display, it decreased monitoring glances and, consequentially, perceived effort. Further, an augmented reality-based uncertainty display concept was developed to minimise the workload increments associated with increases in the functional detail of uncertainty information.</div

Predictive statistical user models under the collaborative approach

Mención Internacional en el título de doctorUser models and recommender systems due to their similarity can be considered the same thing except from the use that we make of them. Both have their root in multiple disciplines such as information retrieval or machine learning among others. The impact has grown rapidly with the importance of data on systems and applications. Most of the big companies employ one of the other for different reasons such as: gathering more customers, boost sales or increase revenue. Thus very well-known companies like Amazon, EBay or Google use models to improve their businesses. In fact, as data becomes more and more important for companies, universities and people, user models are crucial to make decisions over large amounts of data. Although user models can provide accurate predictions on large populations their use and application is not restricted to predictions but can be extended to selection of dialogue strategies or detection of communities within complex domains. After a deep review of the existing literature, it was found that there is a lack of statistical user models based on experience plus the existing models in the area are content-based models that suffer from major problems as scalability, cold-start or new user problem. Furthermore, researchers in the area of user modelling usually develop their own models and then perform ad-hoc evaluations that are not replicable and therefore not comparable. The lack of a complete framework for evaluation makes very difficult to compare results across models and domains. There are two main approaches to build a user model or recommender system: the content based approach, where predictions are based on the same user past behaviours; and the collaborative approach where predictions rely on like-minded people. Both approaches have advantages but also downsides that have to be considered before building a model. The main goal of this thesis is to develop a hybrid user model that takes the strengths of both approaches and mitigates the downsides by combining both methods. The proposed hybrid model is based on an R-Tree structure. The selection of this structure to support the models is backed from the fact that the rectangle tree is specifically designed to effectively store and manipulate multidimensional data. This data structure introduced by Guttman in 1984 is a height balanced tree that only requires visiting a few nodes to perform a tree search. As a result, it can manage large populations of data efficiently as only a few nodes are visited during the inference. R-Tree has two different typologies of nodes: the leaf-node and the non-leaf node. Leaf nodes contain the whole universe of users while non leaf nodes are somehow redundant and contain summaries of child nodes. Along this thesis two statistical user models based on experience have been proposed. The first one is a knowledge base user mode (KLUM), is a classical approach that summarizes and remove data in order to keep performance level within reasonable margins. The second one, an R-Tree user model (RTUM), is an innovative model based on an R-Tree structure. This new model not only solves the problem of removing data but also the scalability problem which turns out to be one of the major problems in the area of user modelling. Both models have been developed and tested with equivalent formulations to make comparisons relevant. Both models are prepared to create their own knowledge base from scratch but also they can be fed with expert knowledge. Thus alleviating another major problem in the area of user modelling as it is the start-up problem. Regarding the proposal of this thesis, two statistical user models are proposed (KLUM and RTUM). In addition, a refinement of RTUM user model is proposed, while RTUM performs node partitions based on the centroids of the users in that node, the new refinement implements a new partition based on privileged features. Hence, the new approach takes advantage of most discriminatory features of the domain to perform the partition. This new approach not only provides accurate inferences, but also an excellent clustering that can be useful in many different scenarios. For instance, this clustering can be employed in the area of social networks to detect communities within the social network. This is a tough task that has been one of the goals of many researchers during the last few years. This thesis also provides a complete evaluation of the models with a great diversity of parameterizations and domains. The models are tested in four different domains and as a result of the evaluation, it is proved that RTUM user model provides a massive gain against classical user models as KLUM. During the evaluation, RTUM reached success rates of 85% while the analogous KLUM could only reach a 65% thus leaving a 20% gain for the proposed model. The evaluation provided not only compares models and success rates, but also provides a broad analysis of how every parameter of the models impact the performance plus a complete study of the databases sizes and inference times for the models. The main conclusion to the evaluation is that after a complete evaluation with a wide diversity of parameters and domains RTUM outperforms KLUM on every scenario tested. As previously mentioned, after the literature review it was also found a lack of evaluation frameworks for user modelling. This thesis also provides a complete evaluation framework for user modelling. This fills a gap in the literature as well as makes the evaluation replicable and therefore comparable. Along years researchers and developers had found difficulties to compare evaluations and measure the quality of their models in different domains due to the lack of an evaluation standard. The evaluation framework presented in this thesis covers data samples including training set and test set plus different sets of experiments alongside with a statistical analysis of the domain, confidence intervals and confidence levels to guarantee that each experiment is statistically significant. The evaluation framework can be downloaded and then used to complete evaluations and cross-validate results across different models.This thesis would not have been possible without the financial support of the following research projects Cadooh (TSI-020302-2011-21), Thuban (TIN2008-02711) that funded part of this research.Programa Oficial de Doctorado en Ciencia y Tecnología InformáticaPresidente: Antonio de Amescua Seco.- Secretario: Ruth Cobos Pérez.- Vocal: Dominikus Heckman

HCI engineering design principles: Acquisition of class-level knowledge.

This research addresses a general problem, which is characterised as a lack of validated HCI design knowledge, supported by guarantees of application. Long & Dowell's (1989) conception of HCI as an engineering discipline characterises such knowledge as HCI Engineering Design Principles (EDPs), which offer prescriptive design support to solve the general design problem of HCI, and can be validated, such that guarantees of application may be ascribed. EDPs may thus be considered a solution to the general problem identified. The technical aim, addressed by this research, is the specification of initial EDPs. The EDPs are considered 4initial', as they have not yet been validated, and as such are not supported by guarantees of application. However, their specification makes progress towards the general solution of EDPs supported by guarantees of application. In order to support development of EDPs, this research extends Long & Dowell's conception of EDPs, by specifying a conception of the general EDP as class HCI design knowledge, that is, EDPs have a class of design problems as their scope, and support specification of a corresponding class of design solutions and a conception of the general design solution delivered by application of the general EDP. The thesis then identifies a strategy and a method for the development of EDPs, which involve the construction of class design problems and corresponding class design solutions, and the identification of EDP components from commonalities and non-commonalities between these classes. The method is operationalised in two cycles of research, to construct two initial EDPs for business-to-consumer electronic commerce transaction systems. The two initial EDPs achieve the technical aim of the research, and as such, make progress towards the general solution of acquiring EDPs supported by guarantees of application. The requirement for additional research, sufficient to develop guarantees of application, is then discussed