Quality Evaluation of Requirements Models: The Case of Goal Models and Scenarios

Context: Requirements Engineering approaches provide expressive model techniques for requirements elicitation and analysis. Yet, these approaches struggle to manage the quality of their models, causing difficulties in understanding requirements, and increase development costs. The models’ quality should be a permanent concern. Objectives: We propose a mixed-method process for the quantitative evaluation of the quality of requirements models and their modelling activities. We applied the process to goal-oriented (i* 1.0 and iStar 2.0) and scenario-based (ARNE and ALCO use case templates) models, to evaluate their usability in terms of appropriateness recognisability and learnability. We defined (bio)metrics about the models and the way stakeholders interact with them, with the GQM approach. Methods: The (bio)metrics were evaluated through a family of 16 quasi-experiments with a total of 660 participants. They performed creation, modification, understanding, and review tasks on the models. We measured their accuracy, speed, and ease, using metrics of task success, time, and effort, collected with eye-tracking, electroencephalography and electro-dermal activity, and participants’ opinion, through NASA-TLX. We characterised the participants with GenderMag, a method for evaluating usability with a focus on gender-inclusiveness. Results: For i*, participants had better performance and lower effort when using iStar 2.0, and produced models with lower accidental complexity. For use cases, participants had better performance and lower effort when using ALCO. Participants using a textual representation of requirements had higher performance and lower effort. The results were better for ALCO, followed by ARNE, iStar 2.0, and i* 1.0. Participants with a comprehensive information processing and a conservative attitude towards risk (characteristics that are frequently seen in females) took longer to start the tasks but had a higher accuracy. The visual and mental effort was also higher for these participants. Conclusions: A mixed-method process, with (bio)metric measurements, can provide reliable quantitative information about the success and effort of a stakeholder while working on different requirements models’ tasks

Augmented Reality and Context Awareness for Mobile Learning Systems

Learning is one of the most interactive processes that humans practice. The level of interaction between the instructor and his or her audience has the greatest effect on the output of the learning process. Recent years have witnessed the introduction of e-learning (electronic learning), which was then followed by m-learning (mobile learning). While researchers have studied e-learning and m-learning to devise a framework that can be followed to provide the best possible output of the learning process, m-learning is still being studied in the shadow of e-learning. Such an approach might be valid to a limited extent, since both aims to provide educational material over electronic channels. However, m-learning has more space for user interaction because of the nature of the devices and their capabilities. The objective of this work is to devise a framework that utilises augmented reality and context awareness in m-learning systems to increase their level of interaction and, hence, their usability. The proposed framework was implemented and deployed over an iPhone device. The implementation focused on a specific course. Its material represented the use of augmented reality and the flow of the material utilised context awareness. Furthermore, a software prototype application for smart phones, to assess usability issues of m-learning applications, was designed and implemented. This prototype application was developed using the Java language and the Android software development kit, so that the recommended guidelines of the proposed framework were maintained. A questionnaire survey was conducted at the University, with approximately twenty-four undergraduate computer science students. Twenty-four identical smart phones were used to evaluate the developed prototype, in terms of ease of use, ease of navigating the application content, user satisfaction, attractiveness and learnability. Several validation tests were conducted on the proposed augmented reality m-learning verses m-learning. Generally, the respondents rated m-learning with augmented reality as superior to m-learning alone

CLiFF Notes: Research In Natural Language Processing at the University of Pennsylvania

The Computational Linguistics Feedback Forum (CLIFF) is a group of students and faculty who gather once a week to discuss the members\u27 current research. As the word feedback suggests, the group\u27s purpose is the sharing of ideas. The group also promotes interdisciplinary contacts between researchers who share an interest in Cognitive Science. There is no single theme describing the research in Natural Language Processing at Penn. There is work done in CCG, Tree adjoining grammars, intonation, statistical methods, plan inference, instruction understanding, incremental interpretation, language acquisition, syntactic parsing, causal reasoning, free word order languages, ... and many other areas. With this in mind, rather than trying to summarize the varied work currently underway here at Penn, we suggest reading the following abstracts to see how the students and faculty themselves describe their work. Their abstracts illustrate the diversity of interests among the researchers, explain the areas of common interest, and describe some very interesting work in Cognitive Science. This report is a collection of abstracts from both faculty and graduate students in Computer Science, Psychology and Linguistics. We pride ourselves on the close working relations between these groups, as we believe that the communication among the different departments and the ongoing inter-departmental research not only improves the quality of our work, but makes much of that work possible

Cognitive approaches to SLA.

Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/139742/1/CognitiveApproachestoSLA.pd

Enhancing user experience and safety in the context of automated driving through uncertainty communication

Operators of highly automated driving systems may exhibit behaviour characteristic of overtrust issues due to an insufficient awareness of automation fallibility. Consequently, situation awareness in critical situations is reduced and safe driving performance following emergency takeovers is impeded. Previous research has indicated that conveying system uncertainties may alleviate these issues. However, existing approaches require drivers to attend the uncertainty information with focal attention, likely resulting in missed changes when engaged in non-driving-related tasks. This research project expands on existing work regarding uncertainty communication in the context of automated driving. Specifically, it aims to investigate the implications of conveying uncertainties under consideration of non-driving-related tasks and, based on the outcomes, develop and evaluate an uncertainty display that enhances both user experience and driving safety. In a first step, the impact of visually conveying uncertainties was investigated under consideration of workload, trust, monitoring behaviour, non-driving-related tasks, takeover performance, and situation awareness. For this, an anthropomorphic visual uncertainty display located in the instrument cluster was developed. While the hypothesised benefits for trust calibration and situation awareness were confirmed, the results indicate that visually conveying uncertainties leads to an increased perceived effort due to a higher frequency of monitoring glances. Building on these findings, peripheral awareness displays were explored as a means for conveying uncertainties without the need for focused attention to reduce monitoring glances. As a prerequisite for developing such a display, a systematic literature review was conducted to identify evaluation methods and criteria, which were then coerced into a comprehensive framework. Grounded in this framework, a peripheral awareness display for uncertainty communication was developed and subsequently compared with the initially proposed visual anthropomorphic uncertainty display in a driving simulator study. Eye tracking and subjective workload data indicate that the peripheral awareness display reduces the monitoring effort relative to the visual display, while driving performance and trust data highlight that the benefits of uncertainty communication are maintained. Further, this research project addresses the implications of increasing the functional detail of uncertainty information. Results of a driving simulator study indicate that particularly workload should be considered when increasing the functional detail of uncertainty information. Expanding upon this approach, an augmented reality display concept was developed and a set of visual variables was explored in a forced choice sorting task to assess their ordinal characteristics. Particularly changes in colour hue and animation-based variables received high preference ratings and were ordered consistently from low to high uncertainty. This research project has contributed a series of novel insights and ideas to the field of human factors in automated driving. It confirmed that conveying uncertainties improves trust calibration and situation awareness, but highlighted that using a visual display lessens the positive effects. Addressing this shortcoming, a peripheral awareness display was designed applying a dedicated evaluation framework. Compared with the previously employed visual display, it decreased monitoring glances and, consequentially, perceived effort. Further, an augmented reality-based uncertainty display concept was developed to minimise the workload increments associated with increases in the functional detail of uncertainty information.</div

Developing educational games for children with ASC. VENECA- Virtual Environment for Navigational Education for Children with ASC

This thesis describes the full design process of VENECA, a research tool for experts in the area of cognitive science who specialize in children with ASC. The initial sections of this work concentrate on analyses of open questions and underexplored hypotheses in the area that can potentially be interesting to future expert users, and specifications of interconnected flexible features that could assist experts in their research studies. A review of different deficits of children with ASC, research directions and previous work, done in the area, and critical analysis of analogous computational tools in the area suggested potential problems and doubtful design decisions of the currently existing computer games that had to be avoided VENECA. After that, by means of pre-design analysis, all the constraints and goals of the project were met, and user requirements were specified. As a result, VENECA was developed as a research tool with a game as an internal cognitive element. The design process, which was organised in an iterative fashion, allowed precise specification of all design decisions and adjustment of the system to the needs of both children and research users. Prototype testings with experts in the area provided valuable feedback about not only the experts’ requirements, but also those of the child user’s. The fact that this feedback was gained during the whole design process allowed for the quick and low-cost fixing of all potential usability problems. At the end of this project, a full scope of initial goals was achieved, and the first version of VENECA was implemented as a complete software package with all necessary manuals and documentation. It provides necessary support for experts in order to answer a set of initial questions. Moreover, flexible combinations of features suggest that experts will potentially be able to specify their own research questions in the future. However, there still exist many directions for future extensions of VENECA, which were also analysed, described and justified in this work

Automotive UX design and data-driven development: Narrowing the gap to support practitioners

The development and evaluation of In-Vehicle Information Systems (IVISs) is strongly based on insights from qualitative studies conducted in artificial contexts (e.g., driving simulators or lab experiments). However, the growing complexity of the systems and the uncertainty about the context in which they are used, create a need to augment qualitative data with quantitative data, collected during real-world driving. In contrast to many digital companies that are already successfully using data-driven methods, Original Equipment Manufacturers (OEMs) are not yet succeeding in releasing the potentials such methods offer. We aim to understand what prevents automotive OEMs from applying data-driven methods, what needs practitioners formulate, and how collecting and analyzing usage data from vehicles can enhance UX activities. We adopted a Multiphase Mixed Methods approach comprising two interview studies with more than 15 UX practitioners and two action research studies conducted with two different OEMs. From the four studies, we synthesize the needs of UX designers, extract limitations within the domain that hinder the application of data-driven methods, elaborate on unleveraged potentials, and formulate recommendations to improve the usage of vehicle data. We conclude that, in addition to modernizing the legal, technical, and organizational infrastructure, UX and Data Science must be brought closer together by reducing silo mentality and increasing interdisciplinary collaboration. New tools and methods need to be developed and UX experts must be empowered to make data-based evidence an integral part of the UX design process

Measuring Learnability in Human-Computer Interaction

It is well accepted that learnability is a crucial attribute of usability that should be considered in almost every software system. A good learnability leads within a short time and with minimal effort to a high level of proficiency of the user. Therefore, expensive training time of complex systems is reduced. However, there is only few consensus on how to define and evaluate learnability. In addition, gathering detailed information on learnability is quite difficult. In todays books on usability evaluation, learnability gets only few attention, research publications are spread to several other fields and the term learnability is also used in other context. The objective of this thesis is to give an structured overview of learnability and methods for evaluation and additionally assist in the evaluator’s individual choice of an appropriate method. First of all, several definitions of learnability are discussed. For a deeper understanding psychological background knowledge is provided. Afterwards, methods to asses learnability are presented. This comprises nine methods that seem particularly appropriate to measure learnability. As this methods are very diverse, a framework based on analytical hierarchy process is provided. This framework aims to classify presented methods with respect to certain criteria and assess practitioners in selecting an appropriate method to measure learnability