Evaluating LAB@FUTURE, a collaborative e-learning Laboratory experiments platform

This paper presents Lab@Future, an advanced e-learning platform that uses novel Information and Communication Technologies to support and expand laboratory teaching practices. For this purpose, Lab@Future uses real and computer generated objects that are interfaced using mechatronic systems, augmented reality, mobile technologies and 3D multi user environments. The main aim is to develop and demonstrate technological support for practical experiments in the following focused disciplines namely: Fluid Dynamics - Science subject in Germany, Geometry - Mathematics subject in Austria, History and Environmental Awareness – Arts and Humanities subjects in Greece and Slovenia. In order to pedagogically enhance the design and functional aspects of this e-learning technology, we are investigating the dialogical operationalisation of learning theories so as to leverage our understanding of teaching and learning practices in the targeted context of deployment. To be able to evaluate the lab@future system in its entire complexity an evaluation methodology including several phases has been developed, performing formative as well as summative evaluations

An investigation into the perspectives of providers and learners on MOOC accessibility

An effective open eLearning environment should consider the target learner’s abilities, learning goals, where learning takes place, and which specific device(s) the learner uses. MOOC platforms struggle to take these factors into account and typically are not accessible, inhibiting access to environments that are intended to be open to all. A series of research initiatives are described that are intended to benefit MOOC providers in achieving greater accessibility and disabled learners to improve their lifelong learning and re-skilling. In this paper, we first outline the rationale, the research questions, and the methodology. The research approach includes interviews, online surveys and a MOOC accessibility audit; we also include factors such the risk management of the research programme and ethical considerations when conducting research with vulnerable learners. Preliminary results are presented from interviews with providers and experts and from analysis of surveys of learners. Finally, we outline the future research opportunities. This paper is framed within the context of the Doctoral Consortium organised at the TEEM'17 conference

Toward a document evaluation methodology: What does research tell us about the validity and reliability of evaluation methods?

Although the usefulness of evaluating documents has become generally accepted among communication professionals, the supporting research that puts evaluation practices empirically to the test is only beginning to emerge. This article presents an overview of the available research on troubleshooting evaluation methods. Four lines of research are distinguished concerning the validity of evaluation methods, sample composition, sample size, and the implementation of evaluation results during revisio

Methodological development

Book description: Human-Computer Interaction draws on the fields of computer science, psychology, cognitive science, and organisational and social sciences in order to understand how people use and experience interactive technology. Until now, researchers have been forced to return to the individual subjects to learn about research methods and how to adapt them to the particular challenges of HCI. This is the first book to provide a single resource through which a range of commonly used research methods in HCI are introduced. Chapters are authored by internationally leading HCI researchers who use examples from their own work to illustrate how the methods apply in an HCI context. Each chapter also contains key references to help researchers find out more about each method as it has been used in HCI. Topics covered include experimental design, use of eyetracking, qualitative research methods, cognitive modelling, how to develop new methodologies and writing up your research

On evaluating parallel computer systems

A workshop was held in an attempt to program real problems on the MIT Static Data Flow Machine. Most of the architecture of the machine was specified but some parts were incomplete. The main purpose for the workshop was to explore principles for the evaluation of computer systems employing new architectures. Principles explored were: (1) evaluation must be an integral, ongoing part of a project to develop a computer of radically new architecture; (2) the evaluation should seek to measure the usability of the system as well as its performance; (3) users from the application domains must be an integral part of the evaluation process; and (4) evaluation results should be fed back into the design process. It is concluded that the general organizational principles are achievable in practice from this workshop

Applying a User-centred Approach to Interactive Visualization Design

Analysing users in their context of work and finding out how and why they use different information resources is essential to provide interactive visualisation systems that match their goals and needs. Designers should actively involve the intended users throughout the whole process. This chapter presents a user-centered approach for the design of interactive visualisation systems. We describe three phases of the iterative visualisation design process: the early envisioning phase, the global specification hase, and the detailed specification phase. The whole design cycle is repeated until some criterion of success is reached. We discuss different techniques for the analysis of users, their tasks and domain. Subsequently, the design of prototypes and evaluation methods in visualisation practice are presented. Finally, we discuss the practical challenges in design and evaluation of collaborative visualisation environments. Our own case studies and those of others are used throughout the whole chapter to illustrate various approaches

Evaluating groupware support for software engineering students

Software engineering tasks, during both development and maintenance, typically involve teamwork using computers. Team members rarely work on isolated computers. An underlying assumption of our research is that software engineering teams will work more effectively if adequately supported by network-based groupware technology. Experience of working with groupware and evaluating groupware systems will also give software engineering students a direct appreciation of the requirements of engineering such systems. This research is investigating the provision of such network-based support for software engineering students and the impact these tools have on their groupwork. We will first describe our experiences gained through the introduction of an asynchronous virtual environment ­ SEGWorld to support groupwork during the Software Engineering Group (SEG) project undertaken by all second year undergraduates within the Department of Computer Science. Secondly we will describe our Computer Supported Cooperative Work (CSCW) module which has been introduced into the students' final year of study as a direct result of our experience with SEG, and in particular its role within Software Engineering. Within this CSCW module the students have had the opportunity to evaluate various groupware tools. This has enabled them to take a retrospective view of their experience of SEGWorld and its underlying system, BSCW, one year on. We report our findings for SEG in the form of a discussion of the hypotheses we formulated on how the SEGs would use SEGWorld, and present an initial qualitative assessment of student feedback from the CSCW module