Measuring internet activity: a (selective) review of methods and metrics

Two Decades after the birth of the World Wide Web, more than two billion people around the world are Internet users. The digital landscape is littered with hints that the affordances of digital communications are being leveraged to transform life in profound and important ways. The reach and influence of digitally mediated activity grow by the day and touch upon all aspects of life, from health, education, and commerce to religion and governance. This trend demands that we seek answers to the biggest questions about how digitally mediated communication changes society and the role of different policies in helping or hindering the beneficial aspects of these changes. Yet despite the profusion of data the digital age has brought upon us—we now have access to a flood of information about the movements, relationships, purchasing decisions, interests, and intimate thoughts of people around the world—the distance between the great questions of the digital age and our understanding of the impact of digital communications on society remains large. A number of ongoing policy questions have emerged that beg for better empirical data and analyses upon which to base wider and more insightful perspectives on the mechanics of social, economic, and political life online. This paper seeks to describe the conceptual and practical impediments to measuring and understanding digital activity and highlights a sample of the many efforts to fill the gap between our incomplete understanding of digital life and the formidable policy questions related to developing a vibrant and healthy Internet that serves the public interest and contributes to human wellbeing. Our primary focus is on efforts to measure Internet activity, as we believe obtaining robust, accurate data is a necessary and valuable first step that will lead us closer to answering the vitally important questions of the digital realm. Even this step is challenging: the Internet is difficult to measure and monitor, and there is no simple aggregate measure of Internet activity—no GDP, no HDI. In the following section we present a framework for assessing efforts to document digital activity. The next three sections offer a summary and description of many of the ongoing projects that document digital activity, with two final sections devoted to discussion and conclusions

“Stickiness”: Gauging students’ attention to online learning activities

Purpose: Online content developers use the term “stickiness” to refer to the ability of their online service or game to attract and hold the attention of users and create a compelling and magnetic reason for them to return repeatedly (examples include virtual pets and social media). In business circles, the same term connotes the level of consumer loyalty to a particular brand. This paper aims to extend the concept of “stickiness” not only to describe repeat return and commitment to the learning “product”, but also as a measure of the extent to which students are engaged in online learning opportunities. Design/methodology/approach: This paper explores the efficacy of several approaches to the monitoring and measuring of online learning environments, and proposes a framework for assessing the extent to which these environments are compelling, engaging and “sticky”. Findings: In particular, the exploration so far has highlighted the difference between how lecturers have monitored the engagement of students in a face-to-face setting versus the online teaching environment. Practical implications: In the higher education environment where increasingly students are being asked to access learning in the online space, it is vital for teachers to be in a position to monitor and guide students in their engagement with online materials. Originality/value: The mere presence of learning materials online is not sufficient evidence of engagement. This paper offers options for testing specific attention to online materials allowing greater assurance around engagement with relevant and effective online learning activities

An investigation into the perspectives of providers and learners on MOOC accessibility

An effective open eLearning environment should consider the target learner’s abilities, learning goals, where learning takes place, and which specific device(s) the learner uses. MOOC platforms struggle to take these factors into account and typically are not accessible, inhibiting access to environments that are intended to be open to all. A series of research initiatives are described that are intended to benefit MOOC providers in achieving greater accessibility and disabled learners to improve their lifelong learning and re-skilling. In this paper, we first outline the rationale, the research questions, and the methodology. The research approach includes interviews, online surveys and a MOOC accessibility audit; we also include factors such the risk management of the research programme and ethical considerations when conducting research with vulnerable learners. Preliminary results are presented from interviews with providers and experts and from analysis of surveys of learners. Finally, we outline the future research opportunities. This paper is framed within the context of the Doctoral Consortium organised at the TEEM'17 conference

Library performance measurement in the digital age

Book synopsis: University libraries around the world have embraced the possibilities of the digital learning environment, facilitating its use and proactively seeking to develop the provision of electronic resources and services. The digital environment offers opportunities and challenges for librarians in all aspects of their work - in information literacy, virtual reference, institutional repositories, e-learning, managing digital resources and social media. The authors in this timely book are leading experts in the field of library and information management, and are at the forefront of change in their respective institutions. University Libraries and Digital Learning Environments will be invaluable for all those involved in managing libraries or learning services, whether acquiring electronic resources or developing and delivering services in digital environments

Towards a social learning space for open educational resources

We identify a number of meanings of “Open”, as part of the motivating rationale for a social media space tuned for learning, called SocialLearn. We discuss why online social learning seems to be emerging so strongly at this point, explore features of social learning, and identify some of the dimensions that we believe characterize the social learning design space, before describing the emerging design concept and implementation

A website supporting sensitive religious and cultural advance care planning (ACPTalk): Formative and summative evaluation

Background: Advance care planning (ACP) promotes conversations about future health care needs, enacted if a person is incapable of making decisions at end-of-life that may be communicated through written documentation such as advance care directives. To meet the needs of multicultural and multifaith populations in Australia, an advance care planning website, ACPTalk, was funded to support health professionals in conducting conversations within diverse religious and cultural populations. ACPTalk aimed to provide religion-specific advance care planning content and complement existing resources. Objective: The purpose of this paper was to utilize the context, input, process, and product (CIPP) framework to conduct a formative and summative evaluation of ACPTalk. Methods: The CIPP framework was used, which revolves around 4 aspects of evaluation: context, input, process, and product. Context: health professionals’ solutions for the website were determined through thematic analysis of exploratory key stakeholder interviews. Included religions were determined through an environmental scan, Australian population statistics, and documentary analysis of project steering committee meeting minutes. Input: Project implementation and challenges were examined through documentary analysis of project protocols and meeting minutes. Process: To ensure religion-specific content was accurate and appropriate, a website prototype was built with content review and functionality testing by representatives from religious and cultural organizations and other interested health care organizations who completed a Web-based survey. Product: Website analytics were used to report utilization, and stakeholder perceptions were captured through interviews and a website survey. Results: Context: A total of 16 key stakeholder health professional (7 general practitioners, 2 primary health nurses, and 7 palliative care nurses) interviews were analyzed. Website solutions included religious and cultural information, communication ideas, legal information, downloadable content, and Web-based accessibility. Christian and non-Christian faiths were to be included in the religion-specific content. Input: Difficulties gaining consensus on religion-specific content were overcome by further state and national religious organizations providing feedback. Process: A total of 37 content reviewers included representatives of religious and cultural organizations (n=29), health care (n=5), and community organizations (n=3). The majority strongly agree or agree that the content used appropriate language and tone (92%, 34/37), would support health professionals (89%, 33/37), and was accurate (83%, 24/29). Product: Resource usage within the first 9 months was 12,957 page views in 4260 sessions; majority were (83.45%, 3555/4260) from Australia. A total of 107 Australian-based users completed the website survey; most felt information was accurate (77.6%, 83/107), easy to understand (82.2%, 88/107), useful (86.0%, 92/107), and appropriate (86.0%, 92/107). A total of 20 nurses (general practice n=10, palliative care n=8, and both disciplines n=2) participated in stakeholder interviews. Qualitative findings indicated overall positivity in relation to accessibility, functionality, usefulness, design, and increased knowledge of advance care planning. Recommended improvements included shortened content, a comparable website for patients and families, and multilingual translations. Conclusions: The CIPP framework was effectively applied to evaluate the development and end product of an advance care planning website. Although overall findings were positive, further advance care planning website development should consider the recommendations derived from this study