An Experiential comparative analysis of two remote usability testing methods

Remote usability testing is a key tool for usability professionals. Several remote methods exist and it is often difficult to choose the appropriate method. Testing lower-fidelity prototypes often present unique problems because they provide minimally aesthetic and minimally interactive partial representations of a final product. This qualitative and experiential pilot study is an attempt to compare a remote synchronous (RS) usability testing method where the moderator and participant are displaced by space, and a remote asynchronous (RA) usability testing method where the moderator and participant are displaced by both time and space. An important byproduct of the comparison is the creation of a low-cost, online asynchronous testing mechanism. The results show that the usability issues and participant experience vary between remote synchronous and asynchronous usability testing. While the remote asynchronous method does not require a test moderator and participants find it more convenient to complete the test whenever he or she chooses, participants may require clarification on tasks and usability issues discovered during the test. Participants are also critical of the remote asynchronous method and sometimes find it difficult to complete an entire session on his or her own time without anyone to guide them. Further research is needed to validate the results using a more controlled methodology

Asynchronous Testing of Synchronous Components in GALS Systems

International audienceGALS (Globally Asynchronous Locally Synchronous) systems, such as the Internet of Things or autonomous cars, integrate reactive synchronous components that interact asynchronously. The complexity induced by combining synchronous and asynchronous aspects makes GALS systems difficult to develop and debug. Ensuring their functional correctness and reliability requires rigorous design methodologies, based on formal methods and assisted by validation tools. In this paper we propose a testing methodology for GALS systems integrating: (1) synchronous and asynchronous concurrent models; (2) functional unit testing and behavioral conformance testing; and (3) various formal methods and their tool equipments. We leverage the conformance test generation for asynchronous systems to automatically derive realistic scenarios (input constraints and oracle), which are necessary ingredients for the unit testing of individual synchronous components, and are difficult and error-prone to design manually. We illustrate our approach on a simple, but relevant example inspired by autonomous cars

How to Implement an Asynchronous Test Wrapper for Network-on-Chip Nodes

International audienceThe Network-on-Chip (NoC) paradigm is currently known as an alternative solution for the on chip communication in the next SoC generation, especially, asynchronous NoCs. One of the challenges for asynchronous NoC-based systems design is testing asynchronous network architectures for manufacturing defects. To improve the testability of asynchronous NoCs, we have developed a scalable and configurable asynchronous Design-for-Test (DfT) architecture. In this architecture, each asynchronous network node is surrounded by an asynchronous test wrapper and the network communication channels are reused as a high-speed Test Access Mechanism (TAM). This architecture is designed to test all network elements (routers, communication channels), but it can also be used to test computational resources. In this paper, we introduce how to realize and implement the test wrapper in Quasi Delay Insensitive (QDI) asynchronous logic style. The validation and experimental results are also presented

HTTP Mailbox - Asynchronous RESTful Communication

We describe HTTP Mailbox, a mechanism to enable RESTful HTTP communication in an asynchronous mode with a full range of HTTP methods otherwise unavailable to standard clients and servers. HTTP Mailbox allows for broadcast and multicast semantics via HTTP. We evaluate a reference implementation using ApacheBench (a server stress testing tool) demonstrating high throughput (on 1,000 concurrent requests) and a systemic error rate of 0.01%. Finally, we demonstrate our HTTP Mailbox implementation in a human assisted web preservation application called "Preserve Me".Comment: 13 pages, 6 figures, 8 code blocks, 3 equations, and 3 table

The complexity of asynchronous model based testing

This is the post-print version of the final paper published in Theoretical Computer Science. The published article is available from the link below. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. Copyright @ 2012 Elsevier B.V.In model based testing (MBT), testing is based on a model MM that typically is expressed using a state-based language such as an input output transition system (IOTS). Most approaches to MBT assume that communications between the system under test (SUT) and its environment are synchronous. However, many systems interact with their environment through asynchronous channels and the presence of such channels changes the nature of testing. In this paper we investigate the situation in which the SUT interacts with its environment through asynchronous channels and the problems of producing test cases to reach a state, execute a transition, or to distinguish two states. In addition, we investigate the Oracle Problem. All four problems are explored for both FIFO and non-FIFO channels. It is known that the Oracle Problem can be solved in polynomial time for FIFO channels but we also show that the three test case generation problems can also be solved in polynomial time in the case where the IOTS is observable but the general test generation problems are EXPTIME-hard. For non-FIFO channels we prove that all of the test case generation problems are EXPTIME-hard and the Oracle Problem in NP-hard, even if we restrict attention to deterministic IOTSs

Cognitive Load in Asynchronous Discussions of an Online Undergraduate Stem Course

Purpose – As online course enrollments increase; it is important to understand how common course features influence students’ behaviors and performance. Asynchronous online courses often include a discussion forum to promote community through interaction between students and instructors. Students interact both socially and cognitively; instructors’ engagement often demonstrates social or teaching presence. Students’ engagement in the discussions introduces both intrinsic and extraneous cognitive load. The purpose of this study is to validate an instrument for measuring cognitive load in asynchronous online discussions. Design/methodology/approach – This study presents the validation of the NASA-TLX instrument for measuring cognitive load in asynchronous online discussions in an introductory physics course. Findings – The instrument demonstrated reliability for a model with four subscales for all five discrete tasks. This study is foundational for future work that aims at testing the efficacy of interventions and reducing extraneous cognitive load in asynchronous online discussions. Research limitations/implications – Nonresponse error due to the unincentivized, voluntary nature of the survey introduces a sample-related limitation. Practical implications – This study provides a strong foundation for future research focused on testing the effects of interventions aimed at reducing extraneous cognitive load in asynchronous online discussions. Originality/value – This is a novel application of the NASA-TLX instrument for measuring cognitive load in asynchronous online discussions

WEBPAGE HIDING FOR ANTI-FLICKERING SYNCHRONOUS SCRIPT LOADING

This document describes a technique for eliminating webpage flickering during A/B testing, also known as split or bucket testing, without delaying webpage load times. An asynchronous-hide snippet embedded into a webpage hides the whole webpage during the asynchronous loading of an A/B testing tools’ optimization scripts. A snippet includes an end-of-hiding function to unhide the webpage, a timer to remove the function, and a globally accessible data structure to identify other scripts or clients. A separate endHide function invoked on start-up implements an un-hiding protocol, ensures snippet availability and script registration, marks the data structure to indicate the script hiding status, and removes the end-of-hiding function