Remote Usability Testing - A New Approach Facilitated By Virtual Worlds

Synchronous remote usability testing, involves a facilitator conducting a usability test in real time, interacting with a participant who is remote. This study proposes a new methodology for conducting these studies using a three-dimensional virtual world, Wonderland, and compares it with two other commonly used synchronous usability test methods: the traditional lab approach and WebEx, a web-based conferencing and screen sharing approach. The study involved 48 participants in total, 36 test subjects and 12 test facilitators. These 36 were equally divided among the three environments with the 12 test facilitators being paired with one participant in each of the environments. The participants completed 5 tasks on an e-commerce website. The three methodologies were compared with respect to the dependent variables, the time taken to complete the tasks; the usability defects identified; the severity of these usability issues; and the subjective ratings from the NASA-TLX, the presence and post-test subjective questionnaires. Most importantly, the three methodologies agreed closely in terms of the total number defects identified, number of high severity defects identified and the time taken to complete the tasks. However, there was a significant difference in the workload experienced by the test participants and facilitators, with the traditional lab condition being the least and the Wonderland and the WebEx conditions being almost the same. It was also found that both test participants and test facilitators experienced better involvement and immersive experiences in the Wonderland condition, than the WebEx condition and almost the same for traditional lab condition. The results of this study suggest that participants were productive and enjoyed the Wonderland condition, indicating the potential of a virtual world based approach as an alternative to the conventional approaches

Enhancing the Informatics Evaluation Toolkit with Remote Usability Testing

Developing functional clinical informatics products that are also usable remains a challenge. Despite evidence that usability testing should be incorporated into the lifecycle of health information technologies, rarely does this occur. Challenges include poor standards, a lack of knowledge around usability practices, and the expense involved in rigorous testing with a large number of users. Remote usability testing may be a solution for many of these challenges. Remotely testing an application can greatly enhance the number of users who can iteratively interact with a product, and it can reduce the costs associated with usability testing. A case study presents the experiences with remote usability testing when evaluating a Web site designed for health informatics knowledge dissemination. The lessons can inform others seeking to enhance their evaluation toolkits for clinical informatics products

Evaluating Medical Devices Remotely: Current Methods and Potential Innovations

Objective: We present examples of laboratory and remote studies, with a focus on studies appropriate for medical device design and evaluation. From this review and description of extant options for remote testing, we provide methods and tools to achieve research goals remotely. Background: The FDA mandates human factors evaluation of medical devices. Studies show similarities and differences in results collected in laboratories compared to data collected remotely in non-laboratory settings. Remote studies show promise, though many of these are behavioral studies related to cognitive or experimental psychology. Remote usability studies are rare but increasing, as technologies allow for synchronous and asynchronous data collection. Method: We reviewed methods of remote evaluation of medical devices, from testing labels and instruction to usability testing and simulated use. Each method was coded for the attributes (e.g., supported media) that need consideration in usability studies. Results: We present examples of how published usability studies of medical devices could be moved to remote data collection. We also present novel systems for creating such tests, such as the use of 3D printed or virtual prototypes. Finally, we advise on targeted participant recruitment. Conclusion: Remote testing will bring opportunities and challenges to the field of medical device testing. Current methods are adequate for most purposes, excepting the validation of Class III devices. Application: The tools we provide enable the remote evaluation of medical devices. Evaluations have specific research goals, and our framework of attributes helps to select or combine tools for valid testing of medical devices

Conceitos e métodos para apoio ao desenvolvimento e avaliação de colaboração remota utilizando realidade aumentada

Remote Collaboration using Augmented Reality (AR) shows great potential to establish a common ground in physically distributed scenarios where team-members need to achieve a shared goal. However, most research efforts in this field have been devoted to experiment with the enabling technology and propose methods to support its development. As the field evolves, evaluation and characterization of the collaborative process become an essential, but difficult endeavor, to better understand the contributions of AR. In this thesis, we conducted a critical analysis to identify the main limitations and opportunities of the field, while situating its maturity and proposing a roadmap of important research actions. Next, a human-centered design methodology was adopted, involving industrial partners to probe how AR could support their needs during remote maintenance. These outcomes were combined with literature methods into an AR-prototype and its evaluation was performed through a user study. From this, it became clear the necessity to perform a deep reflection in order to better understand the dimensions that influence and must/should be considered in Collaborative AR. Hence, a conceptual model and a humancentered taxonomy were proposed to foster systematization of perspectives. Based on the model proposed, an evaluation framework for contextualized data gathering and analysis was developed, allowing support the design and performance of distributed evaluations in a more informed and complete manner. To instantiate this vision, the CAPTURE toolkit was created, providing an additional perspective based on selected dimensions of collaboration and pre-defined measurements to obtain “in situ” data about them, which can be analyzed using an integrated visualization dashboard. The toolkit successfully supported evaluations of several team-members during tasks of remote maintenance mediated by AR. Thus, showing its versatility and potential in eliciting a comprehensive characterization of the added value of AR in real-life situations, establishing itself as a generalpurpose solution, potentially applicable to a wider range of collaborative scenarios.Colaboração Remota utilizando Realidade Aumentada (RA) apresenta um enorme potencial para estabelecer um entendimento comum em cenários onde membros de uma equipa fisicamente distribuídos precisam de atingir um objetivo comum. No entanto, a maioria dos esforços de investigação tem-se focado nos aspetos tecnológicos, em fazer experiências e propor métodos para apoiar seu desenvolvimento. À medida que a área evolui, a avaliação e caracterização do processo colaborativo tornam-se um esforço essencial, mas difícil, para compreender as contribuições da RA. Nesta dissertação, realizámos uma análise crítica para identificar as principais limitações e oportunidades da área, ao mesmo tempo em que situámos a sua maturidade e propomos um mapa com direções de investigação importantes. De seguida, foi adotada uma metodologia de Design Centrado no Humano, envolvendo parceiros industriais de forma a compreender como a RA poderia responder às suas necessidades em manutenção remota. Estes resultados foram combinados com métodos da literatura num protótipo de RA e a sua avaliação foi realizada com um caso de estudo. Ficou então clara a necessidade de realizar uma reflexão profunda para melhor compreender as dimensões que influenciam e devem ser consideradas na RA Colaborativa. Foram então propostos um modelo conceptual e uma taxonomia centrada no ser humano para promover a sistematização de perspetivas. Com base no modelo proposto, foi desenvolvido um framework de avaliação para recolha e análise de dados contextualizados, permitindo apoiar o desenho e a realização de avaliações distribuídas de forma mais informada e completa. Para instanciar esta visão, o CAPTURE toolkit foi criado, fornecendo uma perspetiva adicional com base em dimensões de colaboração e medidas predefinidas para obter dados in situ, que podem ser analisados utilizando o painel de visualização integrado. O toolkit permitiu avaliar com sucesso vários colaboradores durante a realização de tarefas de manutenção remota apoiada por RA, permitindo mostrar a sua versatilidade e potencial em obter uma caracterização abrangente do valor acrescentado da RA em situações da vida real. Sendo assim, estabelece-se como uma solução genérica, potencialmente aplicável a uma gama diversificada de cenários colaborativos.Programa Doutoral em Engenharia Informátic

Development of an Integrated Telerehabilitation Information Management System to Support Remote Wheelchair Prescription

Information technology (IT) is central in providing Telerehabilitation (TR), which enables people with disabilities access to limited number of qualified practitioners with specialty expertise, especially at rural areas. Prior to 2008, most TR utilized non-integrated IT systems to provide its basic infrastructure. Using this approach, data management has to be done manually over multiple non-integrated systems, increasing the possibility of outdated or missing data. An integrated system that is open, flexible, extensible, and cost-effective was designed and developed as a solution to mitigate this problem. The work described in this dissertation elaborates the process of developing such system, called the Versatile and Integrated System for Telerehabilitation (VISYTER). VISYTER was intended to become a platform that is capable of delivering any TR, and was first used to support Remote Wheelchair Prescription (RWP), a TR effort to support clinicians in rural Pennsylvania to prescribe wheeled mobility and seating devices. The development process of VISYTER consisted of three main phases: identification and verification of requirements, validation, and evaluation. The requirement identification and verification phase involved a group of expert clinicians from RWP with the purpose of identifying the requirement of the system to support RWP: a system that can provide real-time teleconsultation and documentation support for prescribing a wheeled mobility intervention. Validation studies were conducted with help from ten individuals, including physicians, clinicians, and suppliers participated to validate VISYTER in their workplaces. All participants agreed that VISYTER can be used to properly support both the teleconsultation and documentation phase of RWP. Afterward, the usability of VISYTER was evaluated through a comparison study with a commonly utilized videoconferencing system in TR, POLYCOM. Twenty-six clinicians participated in a counterbalanced experimental study to measure the difference in usability for completing client assessment tasks using both systems. The study found VISYTER to be more efficient and less prone to error when compared to POLYCOM. Based on these findings, the study concluded that an integrated system could improve the usability TR delivery when compared to non-integrated systems approach

How Continuous Monitoring Changes the Interaction of Patients with a Mobile Telemedicine System

The use of continuous glucose monitor changes the way patients manage their diabetes, as observed in the increased number of daily insulin bolus, the increased number of daily BG measurements, and the differences in the distribution of BG measurements throughout the day. Continuous monitoring also increases the interaction of patients with the information system and modifies their patterns of use