Teachers’ involvement in usability testing with children. In:

ABSTRACT This paper describes the involvement of teachers as facilitators in user-based evaluation of educational software. Findings from a case study that compare the behaviour of teachers and designers during the evaluation indicate that there is no strong evidence to support the current practice of teachers' tangential participation in the evaluation of educational software. Further research is needed to better understand the role teachers can play in this type of evaluation so that their knowledge and experience can positively contribute to the evaluation process and outcomes, whilst retaining the children's valuable contribution

Role of context in usability evaluations: A review

Usability is often defined as the ability of a system to carry out specific tasks by specific users in a specific context. Usability evaluation involves testing the system for its expected usability. Usability testing is performed in natural environment (field) or artificial environment (laboratory). The result of usability evaluation is affected by the environment in which it is carried out. Previous studies have focused only on the physical environment (lab and field) effect on the results but rarely focused on the effect of social environment (people present during testing). Therefore, this study aims to review how important it is to take context into account during usability evaluation. Context is explored through the theory of behaviour settings, according to which behaviour of individuals is strongly influenced by the physical as well as the social environment in which they function. The result of this review indicates that the physical and social context plays a substantial role in usability evaluations. Further, it also suggests that the usability evaluation model should encompass context as an important component in the framework.Comment: 10 pages, 2 tables, ACIJ- Vol3.No.2 March 201

ROLE OF CONTEXT IN USABILITY EVALUATIONS: A REVIEW

ABSTRACT Usability is often defined as the ability of a system to carry out specific tasks by specific users in a specific context. Usability evaluation involves testing the system for its expected usability. Usability testing is performed in natural environment (field) or artificial environment (laboratory). The result of usability evaluation is affected by the environment in which it is carried out. Previous studies have focused only on the physical environment (lab and field) effect on the results but rarely focused on the effect of social environment (people present during testing). Therefore, this study aims to review how important it is to take context into account during usability evaluation. Context is explored through the theory of behaviour settings, according to which behaviour of individuals is strongly influenced by the physical as well as the social environment in which they function. The result of this review indicates tha

Practical and ethical concerns in usability testing with children

It is common practice to evaluate interactive technology with users. In industry, usability companies typically carry out these evaluations, and the participants in the evaluation are usually adults. In research studies, researchers who do not do this sort of work on a daily basis, typically perform the evaluation. Complexity can be increased if the researcher is also the developer of the software and if the users are children. This case study explores that space, the evaluation of software with researchers / developers with children. The chapter describes the evaluation of an educational game that was designed to teach Spanish to children. The chapter outlines the planning for, and the execution of, a usability study of the game with 25 children aged 7-8 in a school in the UK. The study used two methods to try and discover usability problems; direct observation and retrospective think-aloud, and also gathered user experience data using the Fun Toolkit. The focus in this chapter is less on the results of the evaluation (although these are presented) but more on the practical and ethical concerns of conducting usability evaluations of games with children within a school setting. Those reading the chapter will gather hints and tips from the narrative and will better understand the use of the three methods included in the study. In addition, the researcher / developer role is discussed and it is shown that the methods used here enabled children to make judgments without the ownership of the product being an issue. To make the main points more concrete, the chapter closes with a set of ‘key points’ to consider when doing usability testing with children in schools

Improving Validity and Reliability in Children’s Self Reports of Technology Use

Researchers working in child computer interaction are constantly seeking new methods and new techniques that will enable them to carry out more valid and more reliable research. Much of this research typically considers the design and development of new products and of new interactive techniques and researchers seek to understand how easy such innovations are for children, how much fun they are to use and how attractive they may be for use. The impact of prior technology use on the children’s responses in those contexts is the core concern of this thesis. The thesis provides a set of tools (survey instruments and guidelines) that can be used by the CCI research community to ascertain the prior experience of children with any technology and with any task. These tools are generated using theory, experience and literature and are validated through user studies. The PETT survey tool comprises three questionnaires, CTEQ, CTUQ and CTHQ and an associated user guide that clearly articulates how to use PETT and demonstrates the flexibility of PETT to be used in many contexts. The guidelines (RWC, SWC and SRT) can be applied on three levels, for general use in research with children, in the design of surveys and in the specifics of designing self-report tools for prior technology experience

Usability evaluation of an e-learning tutorial using two evaluation methods

The use of interactive e-learning tutorials is an effective form of teaching and learning. It is therefore important that attention is paid to their usability. This research relates to the evaluation of a CD-based e-learning tutorial for learning Business English, with the aims of investigating its usability and identifying problems. Particular attention is paid to aspects that hinder the learner from achieving the learning objectives. The study uses two usability evaluation methods (UEMs), namely controlled usability testing in an HCI laboratory and a user questionnaire survey. The main aim of the study is to compare the findings and determine the impact of using two methods in combination. The first outcome of the research was a synthesized framework of evaluation criteria that was applied in the two UEMs. Secondly, findings of the evaluations indicated that two UEMs identified similar problems, thus confirming their reliability in usability evaluation. Another finding was instances where one method produced results not obtained by the other, which shows the complementary value of two different UEMs. A third benefit of the study was that it identified usability problems in the target system.Information SystemsMA (Information Systems

Metodologia híbrida de desenvolvimento centrado no utilizador: aplicada ao software educativo

Doutoramento em Multimédia em EducaçãoNo panorama atual do desenvolvimento de software educativo é importante que os processos de desenvolvimento sejam adequados e compatíveis com o contexto em que serão utilizados este tipo de recursos. Desta forma, é importante melhorar continuamente os processos de desenvolvimento bem como se proceder à avaliação de forma a garantir a sua qualidade e viabilidade económica. Este estudo propõe uma Metodologia Híbrida de Desenvolvimento Centrado no Utilizador (MHDCU) aplicada ao software educativo. Trata-se de um processo de desenvolvimento simples, iterativo e incremental que tem como “alicerces” princípios do Design Centrado no Utilizador, especificados na International Organization for Standardization - ISO 13407. Na sua base encontra-se a estrutura disciplinada de processos de desenvolvimento, bem como práticas e valores dos métodos ágeis de desenvolvimento de software. O processo é constituído por 4 fases principais: planeamento (guião didático), design (storyboard), implementação e manutenção/operação. A prototipagem e a avaliação são realizadas de modo transversal a todo o processo. A metodologia foi implementada numa Pequena e Média Empresa de desenvolvimento de recursos educacionais, com o objetivo de desenvolver recursos educacionais com qualidade reconhecida e simultaneamente viáveis do ponto de vista económico. O primeiro recurso que teve por base a utilização desta metodologia foi o Courseware Sere – “O Ser Humano e os Recursos Naturais”. O trabalho seguiu uma metodologia de investigação & desenvolvimento, de natureza mista, em que se pretendeu descrever e analisar/avaliar uma metodologia de desenvolvimento de software educativo, i.e., o processo, bem como o produto final. O estudo é fundamentalmente descritivo e exploratório. A metodologia de desenvolvimento do software (primeira questão de investigação) foi proposta, essencialmente, com base na revisão integrativa da literatura da especialidade e com base nos resultados que emergiram das Fases 2 e 3. Do ponto de vista exploratório, foi avaliado, por um lado, o potencial técnico e didático da 1ª versão do software inserido no Courseware Sere (segunda questão de investigação), e, por outro lado, analisar os pontos fortes e as fragilidades da metodologia utilizada para o seu desenvolvimento (terceira questão de investigação). Como técnicas de recolha de dados recorreu-se a dois inquéritos por questionário e à observação direta participante (mediada pela plataforma moodle). Quanto às técnicas de análise de dados optou-se pela análise estatística descritiva e pela análise de conteúdo. Os resultados indicam que o recurso desenvolvido possui qualidade técnica e didática. Relativamente a análise da Metodologia Híbrida de desenvolvimento Centrado no Utilizador foram propostas algumas melhorias relacionadas com o envolvimento do utilizador e introdução de novos métodos. Apesar de identificadas algumas limitações, este projeto permitiu que a empresa melhorasse significativamente os processos de desenvolvimento de recursos (mesmo os que não são informatizados), bem como permitiu o aumento do seu portefólio com o desenvolvimento do Courseware Sere.In the current educational software development scenario it is important that development processes are appropriate and consistent with the context in which such resources are used. Thus, it is important to continually improve development processes and to perform correct evaluation processes to ensure their quality and economic viability. This study propose the Hybrid User Centered Development Methodology (HUCDM). This methodology is a simple, iterative and incremental development process. The methodology is based on structured disciplined development processes, on principles of User Centered Design (UCD) processes, specified in the International Organization for Standardization - ISO 13407, as well as on practices and values of agile methods for software development. The process consists of 4 main phases: planning of educational guidelines, storyboard design, implementation and maintenance/operation. The prototyping and evaluation are carried out in order to cross the entire process. The HUCDM is being implemented in a Small and Medium Enterprise (SME) of educational resources development. The first resource that was based in this methodology was the Courseware Sere - The Human Being and the Natural Resources. The work followed a research & development methodology of mixed nature, where it was intended to describe and analyze/evaluate development methodology for educational software, the process and the final product. The study is primarily descriptive and exploratory. The software development methodology (the first research question) was proposed, essentially, based on a literature integrative review and based on the results that emerged from Phases 2 and 3. From the exploratory standpoint, on the one hand, the technical and didactic potential of the software first version inserted in Courseware Sere (the second research question) was evaluated. Moreover, the strengths and weaknesses of the methodology used for its development (the third research question) were analyzed. As data collection techniques two questionnaire surveys were used together with direct participant observation (mediated by moodle). Descriptive statistical analysis and content analysis were used as data analysis techniques. The results achieved indicate that the developed resource has technical and didactic quality. Concerning the Hybrid User Centered Development Methodology analysis improvements with user involvement and new methods were proposed. Although some limitations were identified, this project enabled the software company to significantly improve its resource development processes (even those that are not computerized) and allowed to increase its portfolio with the development of Courseware Sere

Designing a Value Centred Inspection Method for Children

There are currently several evaluation methods suited to children within the Child-Computer Interaction (CCI) community. However, these methods are user based leaving a gap in inspection method suited to children, that is, where children will act as the inspection method evaluators. This research focused on how to design an effective and easy to use inspection method where children will perform the evaluation based on their values. To ensure that the above is met, a user centred approach and a mixed methodology was explored and finally resulted in the creation of the Inspection Method for Children (IMCH) with an accompanying guideline. This six stages method could be used by CCI researchers as a guide to develop similar methods for children, by industries to perform inspection method evaluation with children on technologies designed for children and could be used by designers to gather design criteria for children’s technology. The process undertaken within this research to develop the new method is also novel and could be adapted by new and old researchers when adapting method to suit children. Future work will focus on carrying out evaluation with wider age range of children in the method to ensure suitability of the method for more children. Comparative studies of the method with other usability method to determine the effectiveness of the method and as a refinement process to produce a validated and refined IMCH method