Reviewing and extending the five-user assumption: A grounded procedure for interaction evaluation

" © ACM, 2013. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in ACM Transactions on Computer-Human Interaction (TOCHI), {VOL 20, ISS 5, (November 2013)} http://doi.acm.org/10.1145/2506210 "The debate concerning how many participants represents a sufficient number for interaction testing is well-established and long-running, with prominent contributions arguing that five users provide a good benchmark when seeking to discover interaction problems. We argue that adoption of five users in this context is often done with little understanding of the basis for, or implications of, the decision. We present an analysis of relevant research to clarify the meaning of the five-user assumption and to examine the way in which the original research that suggested it has been applied. This includes its blind adoption and application in some studies, and complaints about its inadequacies in others. We argue that the five-user assumption is often misunderstood, not only in the field of Human-Computer Interaction, but also in fields such as medical device design, or in business and information applications. The analysis that we present allows us to define a systematic approach for monitoring the sample discovery likelihood, in formative and summative evaluations, and for gathering information in order to make critical decisions during the interaction testing, while respecting the aim of the evaluation and allotted budget. This approach – which we call the ‘Grounded Procedure’ – is introduced and its value argued.The MATCH programme (EPSRC Grants: EP/F063822/1 EP/G012393/1

Snoring and Its Management (Part 2/2): Preliminary Design and Prototyping of Anti-Snoring Chin Strap Device

The-aim of this-research was to-design and prototype a-chin-strap-device, which can-be-used to-manage largely-untapped local-population of open-mouth-habitual-snorers. Document-analysis was utilized as one of the-study-instruments (including review of: (i) selected-International-patents on the-designs of chin-strap device; (ii) published-research on head-gear, head-products, and the-use of anthropometric-data in their-design; and (iii) prior-art on chin-strap-devices and their-respective-limitations). Target-specifications/ objectives of chin-strap-device were formulated from the-document-analysis, while Pair-wise-Comparison Charts were-used, to-rank the-importance of the-objectives, in the-different-levels.  This-study also used such-basic-tools-as pencil and paper, for sketching, of the-alternative-designs; and a-database, as a-tool for information-storage and retrieval.  Besides, the-study applied fundamental-Engineering-principles of product design, and was-carried-out in-compliance-with both; ISO8559: 1989 (Garment-construction and anthropometric-surveys-body-dimensions), and ISO7250: 1996 (Basic-human-body-measurements for technological-design). The-best-ranked-design (out of the-3 alternatives made) was chosen, via Engineering numerical-weighted-decision-Matrix and ‘Drop and Re-vote’ (D&R) method. 13 head-dimensions, and a-head of a-medium-size, and of normal-shape, for 50th percentile, black-African-male, of over 40 years of age, was selected, as a-design-target. The-values, for these-head-dimensions (one-dimensional measurements), were obtained from IOM and Anthrotech anthropometric-data-tables, for civilian-population. 2D-drawings, of the-selected-alternative, were created via computer-aided-design (CAD) AutoCAD-software. The-fibre, from which the-final-fabric, to-be-made, for a-chin-strap-device, was designed discretely of the-product-design-process, and then was-incorporated, into-the-design, as a-finished input/fabric-structure. This-study also adopted ‘analysis’ method of materials-selection. The-main objectives, of the-intended-device, was used as a-guide, in-materials-selection. Acrylic man-made textile-fibre was selected, via computerized-materials-databases/libraries; and afterwards the-woven-fabric of plain-weave was-chosen as main-material, for the-device. The-fabrication and assembly, of the-prototype, was achieved via stitching-joining-method. Traditional-testing/usability-inspection (by dynamic- verification) was conducted on a-volunteer (open-mouth-snorer); observers reported, that some-snoring reduction was noticeable, in-comparison with the-observations, of the-volunteer, sleeping, without the-device, moreover, the-device stayed-in-position (without sliding-off), during the-five-observation-nights. Overall, the-result of this-preliminary-design is somewhat-optimistic, further-improvement(s) and trials, however, are necessary. The-study, hence, further-recommended to: (1) carry-out a-detailed-design; (2) fabricate the-next/refined-prototype(s); (3) conduct explorative-use-ability-trials, in-collaboration-with the-department of Medical-Engineering, School of Medicine, MU; and (4) analyze the-marketing-aspect of the-design. Keywords: materials-selection, Acrylic, textile, testing

Exploratory sequential data analysis of user interaction in contemporary BIM applications

Creation oriented software allows the user to work according to their own vision and rules. From the perspective of software analysis, this is challenging because there is no certainty as to how the users are using the software and what kinds of workflows emerge among different users. The aim of this thesis was to study and identify the potential of sequential event pattern data extraction analysis from expert field creation oriented software in the field of Building Information Modeling (BIM). The thesis additionally introduces a concept evaluation model for detecting repetition based usability disruption. Finally, the work presents an implementation of sequential pattern mining based user behaviour analysis and machine learning predictive application using state of the art algorithms. The thesis introduces a data analysis implementation that is built upon inspections of Sequential or Exploratory Sequential Data Analysis (SDA or ESDA) based theory in usability studies. The study implements a test application specific workflow sequence detection and database transfer approach. The paper uses comparative modern mining algorithms known as BIDE and TKS for sequential pattern discovery. Finally, the thesis utilizes the created sequence database to create user detailing workflow predictions using a CPT+ algorithm. The main contribution of the thesis outcome is to open scalable options for both software usability and product development to automatically recognize and predict usability and workflow related information, deficiencies and repetitive workflow. By doing this, more quantifiable metrics can be revealed in relation to software user interface behavior analytics.Luomiseen perustuva ohjelmisto mahdollistaa käyttäjän työskentelyn oman visionsa ja sääntöjensä mukaisesti. Ohjelmien analysoinnin kannalta tämä on haastavaa, koska ei ole varmuutta siitä, kuinka ohjelmistoa tarkalleen käytetään ja millaisia työskentelytapoja ohjelmiston eri käyttäjäryhmille voi syntyä. Opinnäytetyön tavoitteena oli tutkia ja identifioida toistuvien käyttäjätapahtumasekvenssien analyysipotentiaalia tietomallinnukseen keskittyvässä luomispoh jaisessa ohjelmistossa. Opinnäyte esittelee myös evaluointimallikonseptin, jonka avulla on mahdollista tunnistaa toistuvasta käyttäytymisestä aiheutuvia käytettävyysongelmia. Lopuksi työssä esitellään sekvenssianalyysiin perustuva ohjelmiston käyttäjän toiminta-analyysi sekä ennustava koneoppimisen sovellus. Opinnäytetyössä esitellään data-analyysisovellus, joka perustuu käytettävyystutkimuksessa esiintyvien toistuvien sekvenssien tai kokeellisesti toistuvien sekvenssien analyysiteorian tarkasteluun. Sovelluksen toteutus on tehty eritoten työssä käytetylle ohjelmistolle, jossa käyttäjän detaljointitapahtumista muodostetaan sekvenssejä sekvenssitietokannan luomiseksi. Työssä käytetään sekvenssien toistuvuusanalyysiin moderneja louhintamenetelmiä nimeltään BIDE ja TKS. Lopuksi työssä hyödynnetään luotua sekvenssitietokantaa myös käyttäjän detaljointityön ennustamista varten käyttämällä CPT+ algoritmia. Opinnäytetyön tulosten pohjalta pyritään löytämään vaihtoehtoja käytettävyyden ja tuotekehityksen päätöksenteon tietopohjaiseksi tueksi tunnistamalla ja ennusta malla käyttäjien toimintaa ohjelmistossa. Löydetyn informaation avulla on mahdollista ilmaista käytettävyyteen liittyviä ongelmia kvantitatiivisen tiedon valossa

Balancing automation and user control in a home video editing system

The context of this PhD project is the area of multimedia content management, in particular interaction with home videos. Nowadays, more and more home videos are produced, shared and edited. Home videos are captured by amateur users, mainly to document their lives. People frequently edit home videos, to select and keep the best parts of their visual memories and to add to them a touch of personal creativity. However, most users find the current products for video editing timeconsuming and sometimes too technical and difficult. One reason of the large amount of time required for editing is the slow accessibility caused by the temporal dimension of videos: a video needs to be played back in order to be watched or edited. Another reason of the limitation of current video editing tools is that they are modelled too much on professional video editing systems, including technical details like frame-by-frame browsing. This thesis aims at making home video editing more efficient and easier for the non-technical, amateur user. To accomplish this goal, an approach was taken characterized by two main guidelines. We designed a semi-automatic tool, and we adopted a user-centered approach. To gain insights on user behaviours and needs related to home video editing, we designed an Internet-based survey, which was answered by 180 home video users. The results of the survey revealed the facts that video editing is done frequently and is seen as a very time-consuming activity. We also found that users with low experience with PCs often consider video editing programs too complex. Although nearly all commercial editing tools are designed for a PC, many of our respondents said to be interested in doing video editing on a TV. We created a novel concept, Edit While Watching, designed to be user-friendly. It requires only a TV set and a remote control, instead of a PC. The video that the user inputs to the system is automatically analyzed and structured in small video segments. The editing operations happen on the basis of these video segments: the user is not aware anymore of the single video frames. After the input video has been analyzed and structured, a first edited version is automatically prepared. Successively, Edit While Watching allows the user to modify and enrich the automatically edited video while watching it. When the user is satisfied, the video can be saved to a DVD or to another storage medium. We performed two iterations of system implementation and use testing to refine our concept. After the first iteration, we discovered that two requirements were insufficiently addressed: to have an overview of the video and to precisely control which video content to keep or to discard. The second version of EditWhileWatching was designed to address these points. It allows the user to visualize the video at three levels of detail: the different chapters (or scenes) of the video, the shots inside one chapter, and the timeline representation of a single shot. Also, the second version allows the users to edit the video at different levels of automation. For example, the user can choose an event in the video (e.g. a child playing with a toy) and just ask the system to automatically include more content related to it. Alternatively, if the user wants more control, he or she can precisely select which content to add to the video. We evaluated the second version of our tool by inviting nine users to edit their own home videos with it. The users judged Edit While Watching as an easy to use and fast application. However, some of them missed the possibility of enriching the video with transitions, music, text and pictures. Our test showed that the requirements of overview on the video and control in the selection of the edited material are better addressed than in the first version. Moreover, the participants were able to select which video portions to keep or to discard in a time close to the playback time of the video. The second version of Edit While Watching exploits different levels of automation. In some editing functions the user only gives an indication about editing a clip, and the system automatically decides the start and end points of the part of the video to be cut. However, there are also editing functions in which the user has complete control on the start and end points of a cut. We wanted to investigate how to balance automation and user control to optimize the perceived ease of use, the perceived control, the objective editing efficiency and the mental effort. To this aim, we implemented three types of editing functions, each type representing a different balance between automation and user control. To compare these three levels, we invited 25 users to perform pre-defined tasks with the three function types. The results showed that the type of functions with the highest level of automation performed worse than the two other types, according to both subjective and objective measurements. The other two types of functions were equally liked. However, some users clearly preferred the functions that allowed faster editing while others preferred the functions that gave full control and a more complete overview. In conclusion, on the basis of this research some design guidelines can be offered for building an easy and efficient video editing application. Such application should automatically structure the video, eliminate the detail about single frames, support a scalable video overview, implement a rich set of editing functionalities, and should be preferably TV-based

A Participative Approach for Objective and Subjective Evaluation of Seat Discomfort and User Interface Usability

The main aim of the work is the objective and subjective evaluation of two aspects of the interaction user-product, the seating discomfort and the user interface usability, relevant to industrial design, by using innovative methodologies for generating interpretative and predictive models that allowed the development of analysis strategies useful to improve the satisfaction of use of the types of industrial products considered. On the first aspect investigated, research in the field of medicine and epidemiology has shown that, over the past decades, the incidence of work-related musculoskeletal disorders (WMSDs) has considerably increased due to sedentary modern lifestyle, closely related to prolonged period of sitting. The importance of good office seating design in improving human wellness, greatly motivates the interest of specialized literature in topics related to the investigation of the biomechanical aspects of sitting and their effect on perceived discomfort. Typically discomfort assessment is realized on the basis of subjective evaluations and/or postural analysis by the interface pressures. In such context, the experimental sessions and the related data analysis were aimed to investigating on three critical aspects of seat discomfort assessment: 1) the relationship between subjective and objective measures of seat discomfort; 2) gender-based differences in seat interface pressure distribution; 3) discriminant effectiveness of indexes based on seat interface pressure. On the second aspect investigated, it's helpful to recognize that, today, design team can speed up the process of managing information related to design process by adopting digital pattern tools. These tools, as Knowledge Based Engineering (KBE) systems, can assist engineers in capture and re-use the multidisciplinary knowledge in an integrated way, in order to reduce time and cost of designing, to automate repetitive tasks and to support activities in conceptual design. The KBE analyzed in this study is a new digital pattern tool that supports the designers of automotive gearboxes. In such context, the research has focused on the evaluation of interface usability that represents a critical point in the development of a KBE system to demonstrate an effective reduction in the time and cost of designing and increased satisfaction in its use. The methods used for the two aspects studied are both theoretical and experimental and can be summarized in four main steps: 1) developmentofparticipativeprotocolsandexecutionofexperimentalsessionswith collecting of objective measures related to the interaction user-product and subjective measures related to user perceptions; 2) organization, classification and synthesis of experimental data collected by using techniques of descriptive statistics; 3) definition of interpretative and predictive models of phenomena investigated including by developing synthetic indexes, by using techniques of multivariate and multicriteria analysis; 4) statisticalvalidationofthesemodelsandindexes. The main results achieved concern the assessment of user-product interaction for different types of industrial products where such evaluation is essential. The outcomes are originals because they allowed to find the factors that had most influence on case studies and to develop synthetic indexes useful for identify some critical issues related use. Statistical data analysis provided new information relating to phenomena examined. Furthermore, the proposed data analysis strategies can be easily adapted to other experimental contexts, involving different target populations, and could have important effects in the industrial field, because they allow the reduction of design time (with obvious consequences on cost) and improvement of products in terms of end-user satisfaction

Understanding and Evaluating User Interface Visibility

Technology dominates our lives, mobile technology in particular. In 2016 Apple sold their billionth iPhone. By 2018 they had sold their 2 billionth device based on the same underlying operating system. We access such technology through the user interface (UI) and concerns have been raised about the usability of such devices. The situation has been described by some as a “usability crisis”. One of the key issues raised is the lack of visibility of user interface elements, which is deemed to be a critical component of an effective UI. An initial investigation highlighted that UI visibility can be broken down into three key aspects: Firstly; some user interface elements are effectively ‘missing’; Secondly, they are ‘missed’ because they are not seen by the user; and thirdly, they are seen but ‘misunderstood’. Further analysis of the home screen of an iPhone revealed that only 8% of the available functions were visible at the top level, in other words, 92% were effectively ‘missing’. This raises key questions about how UI visibility can be evaluated, and such evaluation adopted into design practice. This research took a psychophysical perspective to better understand UI visibility. This led to the development of an evaluation framework and associated tool called vis-UI-lise. The tool represents UI visibility as a series of 5 hurdles between the user and the interface that have to be overcome for a successful interaction. This tool was applied to an everyday task on a mobile phone which resulted in highlighting a range of possible usability problems. Comparison of the predicted versus observed problems showed that the vis-UI-lise tool had predicted 74% of them, a score that compares well with other usability evaluation tools. A training and support package was also developed for the vis-UI-lise tool and evaluated with four different organisations. This provided key insights into how the tool could be improved to fit in with typical design practice. This thesis brings a new perspective to the understanding and evaluation of UI visibility that could have a real impact on the design of everyday user interfaces