Usability Evaluation in Virtual Environments: Classification and Comparison of Methods

Virtual environments (VEs) are a relatively new type of human-computer interface in which users perceive and act in a three-dimensional world. The designers of such systems cannot rely solely on design guidelines for traditional two-dimensional interfaces, so usability evaluation is crucial for VEs. We present an overview of VE usability evaluation. First, we discuss some of the issues that differentiate VE usability evaluation from evaluation of traditional user interfaces such as GUIs. We also present a review of VE evaluation methods currently in use, and discuss a simple classification space for VE usability evaluation methods. This classification space provides a structured means for comparing evaluation methods according to three key characteristics: involvement of representative users, context of evaluation, and types of results produced. To illustrate these concepts, we compare two existing evaluation approaches: testbed evaluation [Bowman, Johnson, & Hodges, 1999], and sequential evaluation [Gabbard, Hix, & Swan, 1999]. We conclude by presenting novel ways to effectively link these two approaches to VE usability evaluation

Exploratory sequential data analysis of user interaction in contemporary BIM applications

Creation oriented software allows the user to work according to their own vision and rules. From the perspective of software analysis, this is challenging because there is no certainty as to how the users are using the software and what kinds of workflows emerge among different users. The aim of this thesis was to study and identify the potential of sequential event pattern data extraction analysis from expert field creation oriented software in the field of Building Information Modeling (BIM). The thesis additionally introduces a concept evaluation model for detecting repetition based usability disruption. Finally, the work presents an implementation of sequential pattern mining based user behaviour analysis and machine learning predictive application using state of the art algorithms. The thesis introduces a data analysis implementation that is built upon inspections of Sequential or Exploratory Sequential Data Analysis (SDA or ESDA) based theory in usability studies. The study implements a test application specific workflow sequence detection and database transfer approach. The paper uses comparative modern mining algorithms known as BIDE and TKS for sequential pattern discovery. Finally, the thesis utilizes the created sequence database to create user detailing workflow predictions using a CPT+ algorithm. The main contribution of the thesis outcome is to open scalable options for both software usability and product development to automatically recognize and predict usability and workflow related information, deficiencies and repetitive workflow. By doing this, more quantifiable metrics can be revealed in relation to software user interface behavior analytics.Luomiseen perustuva ohjelmisto mahdollistaa käyttäjän työskentelyn oman visionsa ja sääntöjensä mukaisesti. Ohjelmien analysoinnin kannalta tämä on haastavaa, koska ei ole varmuutta siitä, kuinka ohjelmistoa tarkalleen käytetään ja millaisia työskentelytapoja ohjelmiston eri käyttäjäryhmille voi syntyä. Opinnäytetyön tavoitteena oli tutkia ja identifioida toistuvien käyttäjätapahtumasekvenssien analyysipotentiaalia tietomallinnukseen keskittyvässä luomispoh jaisessa ohjelmistossa. Opinnäyte esittelee myös evaluointimallikonseptin, jonka avulla on mahdollista tunnistaa toistuvasta käyttäytymisestä aiheutuvia käytettävyysongelmia. Lopuksi työssä esitellään sekvenssianalyysiin perustuva ohjelmiston käyttäjän toiminta-analyysi sekä ennustava koneoppimisen sovellus. Opinnäytetyössä esitellään data-analyysisovellus, joka perustuu käytettävyystutkimuksessa esiintyvien toistuvien sekvenssien tai kokeellisesti toistuvien sekvenssien analyysiteorian tarkasteluun. Sovelluksen toteutus on tehty eritoten työssä käytetylle ohjelmistolle, jossa käyttäjän detaljointitapahtumista muodostetaan sekvenssejä sekvenssitietokannan luomiseksi. Työssä käytetään sekvenssien toistuvuusanalyysiin moderneja louhintamenetelmiä nimeltään BIDE ja TKS. Lopuksi työssä hyödynnetään luotua sekvenssitietokantaa myös käyttäjän detaljointityön ennustamista varten käyttämällä CPT+ algoritmia. Opinnäytetyön tulosten pohjalta pyritään löytämään vaihtoehtoja käytettävyyden ja tuotekehityksen päätöksenteon tietopohjaiseksi tueksi tunnistamalla ja ennusta malla käyttäjien toimintaa ohjelmistossa. Löydetyn informaation avulla on mahdollista ilmaista käytettävyyteen liittyviä ongelmia kvantitatiivisen tiedon valossa

Gaze-based teleprosthetic enables intuitive continuous control of complex robot arm use: Writing & drawing

Eye tracking is a powerful mean for assistive technologies for people with movement disorders, paralysis and amputees. We present a highly intuitive eye tracking-controlled robot arm operating in 3-dimensional space based on the user's gaze target point that enables tele-writing and drawing. The usability and intuitive usage was assessed by a “tele” writing experiment with 8 subjects that learned to operate the system within minutes of first time use. These subjects were naive to the system and the task and had to write three letters on a white board with a white board pen attached to the robot arm's endpoint. The instructions are to imagine they were writing text with the pen and look where the pen would be going, they had to write the letters as fast and as accurate as possible, given a letter size template. Subjects were able to perform the task with facility and accuracy, and movements of the arm did not interfere with subjects ability to control their visual attention so as to enable smooth writing. On the basis of five consecutive trials there was a significant decrease in the total time used and the total number of commands sent to move the robot arm from the first to the second trial but no further improvement thereafter, suggesting that within writing 6 letters subjects had mastered the ability to control the system. Our work demonstrates that eye tracking is a powerful means to control robot arms in closed-loop and real-time, outperforming other invasive and non-invasive approaches to Brain-Machine-Interfaces in terms of calibration time (<;2 minutes), training time (<;10 minutes), interface technology costs. We suggests that gaze-based decoding of action intention may well become one of the most efficient ways to interface with robotic actuators - i.e. Brain-Robot-Interfaces - and become useful beyond paralysed and amputee users also for the general teleoperation of robotic and exoskeleton in human augmentation

Brain-computer interface controlled functional electrical stimulation device for foot drop due to stroke.

Gait impairment due to foot drop is a common outcome of stroke, and current physiotherapy provides only limited restoration of gait function. Gait function can also be aided by orthoses, but these devices may be cumbersome and their benefits disappear upon removal. Hence, new neuro-rehabilitative therapies are being sought to generate permanent improvements in motor function beyond those of conventional physiotherapies through positive neural plasticity processes. Here, the authors describe an electroencephalogram (EEG) based brain-computer interface (BCI) controlled functional electrical stimulation (FES) system that enabled a stroke subject with foot drop to re-establish foot dorsiflexion. To this end, a prediction model was generated from EEG data collected as the subject alternated between periods of idling and attempted foot dorsiflexion. This prediction model was then used to classify online EEG data into either "idling" or "dorsiflexion" states, and this information was subsequently used to control an FES device to elicit effective foot dorsiflexion. The performance of the system was assessed in online sessions, where the subject was prompted by a computer to alternate between periods of idling and dorsiflexion. The subject demonstrated purposeful operation of the BCI-FES system, with an average cross-correlation between instructional cues and BCI-FES response of 0.60 over 3 sessions. In addition, analysis of the prediction model indicated that non-classical brain areas were activated in the process, suggesting post-stroke cortical re-organization. In the future, these systems may be explored as a potential therapeutic tool that can help promote positive plasticity and neural repair in chronic stroke patients