Methodological development

Book description: Human-Computer Interaction draws on the fields of computer science, psychology, cognitive science, and organisational and social sciences in order to understand how people use and experience interactive technology. Until now, researchers have been forced to return to the individual subjects to learn about research methods and how to adapt them to the particular challenges of HCI. This is the first book to provide a single resource through which a range of commonly used research methods in HCI are introduced. Chapters are authored by internationally leading HCI researchers who use examples from their own work to illustrate how the methods apply in an HCI context. Each chapter also contains key references to help researchers find out more about each method as it has been used in HCI. Topics covered include experimental design, use of eyetracking, qualitative research methods, cognitive modelling, how to develop new methodologies and writing up your research

Sharing data from clinical trials: the rationale for a controlled access approach.

The move towards increased transparency around clinical trials is welcome. Much focus has been on under-reporting of trials and access to individual patient data to allow independent verification of findings. There are many other good reasons for data sharing from clinical trials. We describe some key issues in data sharing, including the challenges of open access to data. These include issues in consent and disclosure; risks in identification, including self-identification; risks in distorting data to prevent self-identification; and risks in analysis. These risks have led us to develop a controlled access policy, which safeguards the rights of patients entered in our trials, guards the intellectual property rights of the original researchers who designed the trial and collected the data, provides a barrier against unnecessary duplication, and ensures that researchers have the necessary resources and skills to analyse the data

Themes, iteration and recoverability in action research

This paper develops three concepts important to the practice of action research recoverability, research themes, and iteration by highlighting their applicability beyond single action research studies. The concepts are discussed against a program of action research, undertaken by a multidisciplinary research team, with a research focus on local, sector and national evels. This contrasts with the more usual pattern of action research in single situations. Action research is criticized on the grounds that it lacks generalizability and external validity from one-off studies. Goodness criteria have been derived to address these and other criticisms. The recoverability criterion, less strong than the repeatability of experimentation, is central to these. A second concept, that of research themes, links the recoverability criterion and iteration in action research. Iteration within and between projects and the notion of critical mass, of doing work in more than one setting, address the limitations of single setting studies

Evaluating system utility and conceptual fit using CASSM

There is a wealth of user-centred evaluation methods (UEMs) to support the analyst in assessing interactive systems. Many of these support detailed aspects of use – for example: Is the feedback helpful? Are labels appropriate? Is the task structure optimal? Few UEMs encourage the analyst to step back and consider how well a system supports users’ conceptual understandings and system utility. In this paper, we present CASSM, a method which focuses on the quality of ‘fit’ between users and an interactive system. We describe the methodology of conducting a CASSM analysis and illustrate the approach with three contrasting worked examples (a robotic arm, a digital library system and a drawing tool) that demonstrate different depths of analysis. We show how CASSM can help identify re-design possibilities to improve system utility. CASSM complements established evaluation methods by focusing on conceptual structures rather than procedures. Prototype tool support for completing a CASSM analysis is provided by Cassata, an open source development

The educational scorecard: The Start of our Journey

Against a backdrop of public sector cuts, increasing university fees and high youth unemployment, we are facing challenges in Higher Education to demonstrate the value of our courses. Assessing the value of learning, however, is not straight forward. This paper reports on a study of evaluation processes on a post-graduate, professionally accredited diploma delivered at a selection of post 1992 universities. The driver for the study was a concern that current evaluation processes do not fully demonstrate the value of the course nor take into account the needs of multiple stakeholders. The project included benchmarking University evaluation processes and conducting a dialogue with stakeholders. The study adopted a qualitative management research approach, involving: a review of current practice, comparison with a sample of equivalent courses, and consultation (in the form of focus groups and semi-structured interviews) with a sample of students and employers. The paper discusses findings and proposes recommendations for future evaluation procedures in the design of an ‘educational scorecard’ for the course. Kaplan and Norton’s ‘balanced scorecard (1996) concept was adapted to provide a mechanism to represent and balance the needs of different stakeholder groups in the education process. We argue that we cannot truly assess accountability and comparability without engaging a range of stakeholders, not only in soliciting their views on the outcome of the learning but also in the design and implementation of evaluation processes. The ‘educational scorecard’ presented in this paper is developed specifically for the post-graduate diploma in Human Resource Management (PDHRM) at Leeds Metropolitan University and the report concludes with some initial reflections on the benefits of adopting the scorecard methodology The proposed model is flexible and may be adapted for other HE institutions and courses

\u201cImprovisation is not allowed in a second language\u201d: A survey of Italian lecturers\u2019 concerns about teaching their subjects through English

English Medium Instruction (EMI) is increasingly being introduced across European universities in countries where English is not a commonly-used language, such as Italy and other central and southern European countries. However the competences and concerns of the lecturers involved are not always considered when such developments are introduced and support or training may not be offered. This paper reports on a survey on English-Medium Instruction (EMI) to which 115 lecturers in a public university in northern Italy responded. The survey was carried out by the university\u2019s Language Centre as part of the LEAP (Learning English for Academic Purposes) Project which was developed to support lecturers in EMI. The survey sought to identify what the lecturers perceived as their strengths and weakness in English, their concerns and also their evaluations of the experience of teaching through English if they had had any. The findings discussed in this paper shed light on the needs of lecturers that are involved in EMI, which relate to methodology as well as language issues. The implications of this for European Language Centres intending to support EMI at their universities are discussed in the conclusions

Learning participation as systems practice

We describe an evolving praxeology for Systems Practice for managing complexity built on 30 years of developing supported open learning opportunities in the area of Systems within the curriculum of The Open University (UK). We ground this description in two specific examples of how notions of participation are incorporated conceptually and practically into a learners programme of study by considering: (i) the postgraduate course 'Environmental Decision Making. A Systems Approach' (T860) and (ii) the undergraduate course 'Managing complexity. A systems approach' (T306)

The ethics of secondary data analysis: learning from the experience of sharing qualitative data from young people and their families in an international study of childhood poverty

This working paper focuses on secondary analysis, an aspect of research practice that is sometimes assumed to pose few ethical challenges. It draws in particular on the experience of a collaborative research project involving secondary analysis of qualitative data collected as part of an ongoing international longitudinal study, Young Lives (www.younglives.org.uk), and sets this alongside a wider review of regulatory guidance on research ethics and academic debates. Secondary analysis can take many forms, and bring many benefits. But it is more ethically complex than regulatory frameworks may imply. Whether or not data are publicly archived, ethical considerations have to be addressed, including responsibilities to participants and the original researchers, and the need to achieve a contextual understanding of the data by identifying and countering risks of misinterpretation. The considerations raised here are intended to aid ethical research practice by supporting planning and reflection – for primary researchers who are planning to archive their data, as well as for researchers embarking on a qualitative secondary analysis. Not least, our experience highlights the importance of developing and maintaining trusting relationships between primary and secondary researchers