Evaluating the effective use of emerging technologies in education

The aim of this tutorial is to present practical guidance for evaluating the effectiveness of educational initiatives involving social software and emerging technologies to support student learning and engagement. Examples of such initiatives are: inclusion of a blog in a course to encourage reflective learning, or having a wiki in a course for fostering team-working skills, or an activity in a 3D virtual world to enable students to learn through simulations, or the use of Delicious for bookmarking resources, or an App on a smartphone. ‘Evaluation’ implies investigating the usability, pedagogical effectiveness (does it meet the learning outcomes?), student experience, and impact on direct stakeholders such as educators and technical support staff (in terms of workload and support required). Educators, practitioners and educational researchers will find this tutorial useful for learning about evaluating initiatives in a systematic manner and yet be able to choose research methods that are not very resource-intensive for themselves and for the participants (primarily students but other direct stakeholders too such as technical support staff). Through examples of social software initiatives, we will discuss a number of data collection and data analysis methods in the tutorial ranging from traditional social science research (e.g. focus groups) to user-centred research methods (e.g. observations, diary studies) and to participatory design methods (e.g. experience sampling, student panels). We will also discuss about ethical considerations of conducting research, specifically, involving social software, where the personal and professional boundaries of user profiles (or identities) sometimes get blurred

A SURVEY OF ORGANIZATIONAL INSTANT MESSAGING

Instant Messaging software has increasingly been used as an alternative communications platform in many organizations. Although Instant Messaging (IM) began as a personal tool for online communication, the software has now been implemented in many organizations and workplaces. The usefulness of IM software has been shown in literature to be positive, increasing efficiency and productivity in the workplace. This paper explores the perceptions of IM software users in the workplace. We solicited opinions to verify the claim of IM’s effect on efficiency and productivity. We also discuss the limitations and negative effects of IM. A pilot survey and data analysis techniques provide the measurement of IM software’s worth or liability to an organization. The results show what components of IM software are most commonly used and what limitations software places on the users. We also provide recommendation of possible enhancements to IM software in this paper

Instant messaging: a novel means of facilitating the participation of hard-to-reach groups in sensitive topic research

As society becomes ever more reliant on internet technology for everyday communications, this paper explores the use of Instant Messaging (IM) in qualitative research. Discussed within the context of sensitive topic research with potentially hidden and hard-to-reach groups, the purpose of this paper is to discuss the value of adaptive and contemporary research approaches which facilitate participation on the terms of the participant.Reflecting on the data collection process from the primary authors PhD research, this paper critically considers some of the issues raised by IM facilitated semi-structured interviews, using a telephone interview completed as part of the same date collection process as a point of comparison.This paper raises a number of issues, including how the perceived depth of participant response is influenced by their brevity, resultant of the space between parties which allows for considered and concise communication. This disconnect, created by the use of technology, also has implications for the power relations between researcher and participant, and the ability to identify the non-verbal cues which communicate emotion and sentiment.This paper highlights that whilst limited in some respects, an IM facilitated interview provides a unique platform through which hidden and hard to reach groups may be empowered to participate in research, which they may usually avoid

Qualitative data collection technologies: A comparison of instant messaging, email, and phone.

ABSTRACT With the growing body of qualitative research on HCI and social computing, it is natural that researchers may choose to conduct that research in a mediated fashion-over telephone or computer networks. In this paper we compare three different qualitative data collection technologies: phone, instant message (IM), and email. We use quantitative analysis techniques to examine the differences between the methods specifically concerning word count and qualitative codes. We find that there are differences between the methods, and that each technology has affordances that impact the data. Although phone interviews contain four times as many words on average as email and IM, we were surprised to discover that there is no significant difference in number of unique qualitative codes expressed between phone and IM

Conducting empirical research in virtual worlds: experiences from two projects in Second Life

At The Open University in the UK, we have conducted empirical qualitative research in Second Life, a virtual world. The first project is in the education domain in which we are investigating the designs of learning spaces in Second Life, while the second project is related to shopping and consumer behavior of users in Second Life. In this paper, we discuss our experiences of conducting empirical research in virtual worlds with a specific focus on the following aspects: ethical norms, real-world and virtual identities, privacy of the participants, communication modalities (voice, text and use of gestures), logistics of conducting user-based studies, and skills and training needs of researchers. Although our experiences are based on conducting empirical research in Second Life, we hope that our experiences and discussions in this paper will also be useful for researchers who aim to conduct research in avatar-based virtual worlds other than Second Life

Galaxy Zoo: Exploring the Motivations of Citizen Science Volunteers

The Galaxy Zoo citizen science website invites anyone with an Internet connection to participate in research by classifying galaxies from the Sloan Digital Sky Survey. As of April 2009, more than 200,000 volunteers had made more than 100 million galaxy classifications. In this paper, we present results of a pilot study into the motivations and demographics of Galaxy Zoo volunteers, and define a technique to determine motivations from free responses that can be used in larger multiple-choice surveys with similar populations. Our categories form the basis for a future survey, with the goal of determining the prevalence of each motivation.Comment: 15 pages, 3 figure

From “listen and repeat” to “listen and revise”: how to transcribe interviews offline quickly and for free using voice recognition software

Transcribing interviews is one of the most time-consuming and alienating tasks in qualitative research. Some have tried to bypass the problem by hiring external transcribers, which however can be very expensive. Whether due to the time and energy or the financial investment that transcribing requires, researchers therefore often self-impose a limit on the number of interviews that they conduct or even refuse to conduct interviews altogether. To prevent this and help reduce transcription fatigue, some scholars have developed the so-called “listen and repeat” technique. This involves speaking into a microphone what one hears through their headset while a voice recognition software transcribes word by word what it hears. However, this technique has many limitations and still requires a considerable amount of time and effort to be put by the researcher. This article introduces an alternative transcription technique which helps overcome these problems thanks to recent advancements of Artificial Intelligence in the field of voice recognition. Although the drawbacks and unintended consequences of Artificial Intelligence are often highlighted, this article explores its use for interview transcription showing that it can improve drastically the work and life of qualitative researchers. More specifically, this article introduces a transcription technique which allows to generate transcripts fully offline (avoiding in so doing the security concerns that the rising number of cloud-based transcription platforms often raise), rapidly and at little to no cost which one only needs to revise whilst listening to interview recordings, which is why I call this the “listen and revise” technique

HUNTED BY THE CROWD: A QUALITATIVE ANALYSIS OF COLLABORATIVE INFORMATION SEARCHING IN CHINA

This study explores a particular form of cyber surveillance in China known as "human flesh search," in which unrelated Internet users collaboratively conduct surveillance on fellow citizens. Its theoretical framework draws up the notion of panoptic model, first articulated in the early 1800s by Jeremy Bentham and then developed by the French social theorist and philosopher Michel Foucault. Unlike some previous studies on human flesh searching, which focus on highly publicized search incidents, this study examines cyber surveillance in its daily practice, and probes how and why collaborative searches occur in China. It also explores structural constraints and empowerment experienced by search participants through the lens of power, in order to understand such a controversial activity. The study involved content analysis of a Chinese leading search forum--MOP Human Flesh Search Forum; an online survey with 158 search participants; and in-depth interviews with 9 search participants. The study found that Chinese human flesh search often took the forms of coveillance (peer-to-peer surveillance) and sousveillance (bottom-up surveillance). Fun-seeking was the primary motive for participants, who are mainly male youths; being helpful is the next. Privacy invasions and power abuses have complicated or even undermined search practices, limiting the potential of this activity to contribute to civil governance. In terms of empowerment, participating in the human flesh search seemed to give individual searchers a sense of empowerment, but such effects vary greatly depending on individuals' knowledge, social resources and search experiences. Privacy invasions and power abuses were consistently evident in searching practices, limiting the potential of this crowd-based searching, even when this is said to promote justice, to contribute to civil governance. The study also found that the panoptic model is still highly relevant and useful in understanding collaborative online surveillance, especially the function and effects of "gaze." Once conducted in a collective manner, the gaze of fellow citizens can be greatly extended in its reach and intensified by massive participation. The human flesh search mechanism studied here has great potential to profoundly change China's media landscape, but such potential is limited by current media censorship and the lack of accountability of search participants. Although the study examines searching phenomenon only in Chinese cyber space, the findings may shed light on similar surveillance practices which have emerged elsewhere in recent years. The whole question of citizen participation might benefit from the explication of the role of participation in this form of surveillance