Literacy for digital futures : Mind, body, text

The unprecedented rate of global, technological, and societal change calls for a radical, new understanding of literacy. This book offers a nuanced framework for making sense of literacy by addressing knowledge as contextualised, embodied, multimodal, and digitally mediated. In today’s world of technological breakthroughs, social shifts, and rapid changes to the educational landscape, literacy can no longer be understood through established curriculum and static text structures. To prepare teachers, scholars, and researchers for the digital future, the book is organised around three themes – Mind and Materiality; Body and Senses; and Texts and Digital Semiotics – to shape readers’ understanding of literacy. Opening up new interdisciplinary themes, Mills, Unsworth, and Scholes confront emerging issues for next-generation digital literacy practices. The volume helps new and established researchers rethink dynamic changes in the materiality of texts and their implications for the mind and body, and features recommendations for educational and professional practice

A systematic review of technology-enhanced L2 listening development since 2000

Since 2000, technology-enhanced L2 listening development (TELD) has been increasingly investigated. However, systematic reviews concerning the technologies, learning tasks, and outcomes of TELD remain limited. To fill this gap, we conducted a systematic review of publications from 2000 to 2022 on TELD from the perspectives of technologies, learning tasks, and learning outcomes. Forty-six articles from Web of Science were screened by predefined criteria and analysed based on a step-by-step procedure using the PRISMA framework. The findings revealed 13 types of technology and 19 learning tasks useful for TELD. TELD was effective both in terms of building listening skills and enhancing learner emotions. The studies showed that TELD supported learner interactions, encouraged active engagement, and augmented various learning tasks. Based on the findings, we developed a TELD model consisting of two parts: “Within cognitive systems,” in which learners deal with cognitive schemata, listening strategy application, and listening practice via solid attention; “outside of cognitive systems,” in which TELD can construct and reconstruct cognitive schemata, support listening practices, encourage and guide listening strategy application, and improve learner emotions and attention by providing learning materials and activities based on listening-related knowledge, listening exercises with feedback, prompts and feedback on listening strategy application, and a sense of enjoyment and comfort

Seven HCI Grand Challenges

This article aims to investigate the Grand Challenges which arise in the current and emerging landscape of rapid technological evolution towards more intelligent interactive technologies, coupled with increased and widened societal needs, as well as individual and collective expectations that HCI, as a discipline, is called upon to address. A perspective oriented to humane and social values is adopted, formulating the challenges in terms of the impact of emerging intelligent interactive technologies on human life both at the individual and societal levels. Seven Grand Challenges are identified and presented in this article: Human-Technology Symbiosis; Human-Environment Interactions; Ethics, Privacy and Security; Well-being, Health and Eudaimonia; Accessibility and Universal Access; Learning and Creativity; and Social Organization and Democracy. Although not exhaustive, they summarize the views and research priorities of an international interdisciplinary group of experts, reflecting different scientific perspectives, methodological approaches and application domains. Each identified Grand Challenge is analyzed in terms of: concept and problem definition; main research issues involved and state of the art; and associated emerging requirements

Enhancing interaction in mixed reality

With continuous technological innovation, we observe mixed reality emerging from research labs into the mainstream. The arrival of capable mixed reality devices transforms how we are entertained, consume information, and interact with computing systems, with the most recent being able to present synthesized stimuli to any of the human senses and substantially blur the boundaries between the real and virtual worlds. In order to build expressive and practical mixed reality experiences, designers, developers, and stakeholders need to understand and meet its upcoming challenges. This research contributes a novel taxonomy for categorizing mixed reality experiences and guidelines for designing mixed reality experiences. We present the results of seven studies examining the challenges and opportunities of mixed reality experiences, the impact of modalities and interaction techniques on the user experience, and how to enhance the experiences. We begin with a study determining user attitudes towards mixed reality in domestic and educational environments, followed by six research probes that each investigate an aspect of reality or virtuality. In the first, a levitating steerable projector enables us to investigate how the real world can be enhanced without instrumenting the user. We show that the presentation of in-situ instructions for navigational tasks leads to a significantly higher ability to observe and recall real-world landmarks. With the second probe, we enhance the perception of reality by superimposing information usually not visible to the human eye. In amplifying the human vision, we enable users to perceive thermal radiation visually. Further, we examine the effect of substituting physical components with non-functional tangible proxies or entirely virtual representations. With the third research probe, we explore how to enhance virtuality to enable a user to input text on a physical keyboard while being immersed in the virtual world. Our prototype tracked the user’s hands and keyboard to enable generic text input. Our analysis of text entry performance showed the importance and effect of different hand representations. We then investigate how to touch virtuality by simulating generic haptic feedback for virtual reality and show how tactile feedback through quadcopters can significantly increase the sense of presence. Our final research probe investigates the usability and input space of smartphones within mixed reality environments, pairing the user’s smartphone as an input device with a secondary physical screen. Based on our learnings from these individual research probes, we developed a novel taxonomy for categorizing mixed reality experiences and guidelines for designing mixed reality experiences. The taxonomy is based on the human sensory system and human capabilities of articulation. We showcased its versatility and set our research probes into perspective by organizing them inside the taxonomic space. The design guidelines are divided into user-centered and technology-centered. It is our hope that these will contribute to the bright future of mixed reality systems while emphasizing the new underlining interaction paradigm.Mixed Reality (vermischte Realitäten) gehen aufgrund kontinuierlicher technologischer Innovationen langsam von der reinen Forschung in den Massenmarkt über. Mit der Einführung von leistungsfähigen Mixed-Reality-Geräten verändert sich die Art und Weise, wie wir Unterhaltungsmedien und Informationen konsumieren und wie wir mit Computersystemen interagieren. Verschiedene existierende Geräte sind in der Lage, jeden der menschlichen Sinne mit synthetischen Reizen zu stimulieren. Hierdurch verschwimmt zunehmend die Grenze zwischen der realen und der virtuellen Welt. Um eindrucksstarke und praktische Mixed-Reality-Erfahrungen zu kreieren, müssen Designer und Entwicklerinnen die künftigen Herausforderungen und neuen Möglichkeiten verstehen. In dieser Dissertation präsentieren wir eine neue Taxonomie zur Kategorisierung von Mixed-Reality-Erfahrungen sowie Richtlinien für die Gestaltung von solchen. Wir stellen die Ergebnisse von sieben Studien vor, in denen die Herausforderungen und Chancen von Mixed-Reality-Erfahrungen, die Auswirkungen von Modalitäten und Interaktionstechniken auf die Benutzererfahrung und die Möglichkeiten zur Verbesserung dieser Erfahrungen untersucht werden. Wir beginnen mit einer Studie, in der die Haltung der nutzenden Person gegenüber Mixed Reality in häuslichen und Bildungsumgebungen analysiert wird. In sechs weiteren Fallstudien wird jeweils ein Aspekt der Realität oder Virtualität untersucht. In der ersten Fallstudie wird mithilfe eines schwebenden und steuerbaren Projektors untersucht, wie die Wahrnehmung der realen Welt erweitert werden kann, ohne dabei die Person mit Technologie auszustatten. Wir zeigen, dass die Darstellung von in-situ-Anweisungen für Navigationsaufgaben zu einer deutlich höheren Fähigkeit führt, Sehenswürdigkeiten der realen Welt zu beobachten und wiederzufinden. In der zweiten Fallstudie erweitern wir die Wahrnehmung der Realität durch Überlagerung von Echtzeitinformationen, die für das menschliche Auge normalerweise unsichtbar sind. Durch die Erweiterung des menschlichen Sehvermögens ermöglichen wir den Anwender:innen, Wärmestrahlung visuell wahrzunehmen. Darüber hinaus untersuchen wir, wie sich das Ersetzen von physischen Komponenten durch nicht funktionale, aber greifbare Replikate oder durch die vollständig virtuelle Darstellung auswirkt. In der dritten Fallstudie untersuchen wir, wie virtuelle Realitäten verbessert werden können, damit eine Person, die in der virtuellen Welt verweilt, Text auf einer physischen Tastatur eingeben kann. Unser Versuchsdemonstrator detektiert die Hände und die Tastatur, zeigt diese in der vermischen Realität an und ermöglicht somit die verbesserte Texteingaben. Unsere Analyse der Texteingabequalität zeigte die Wichtigkeit und Wirkung verschiedener Handdarstellungen. Anschließend untersuchen wir, wie man Virtualität berühren kann, indem wir generisches haptisches Feedback für virtuelle Realitäten simulieren. Wir zeigen, wie Quadrokopter taktiles Feedback ermöglichen und dadurch das Präsenzgefühl deutlich steigern können. Unsere letzte Fallstudie untersucht die Benutzerfreundlichkeit und den Eingaberaum von Smartphones in Mixed-Reality-Umgebungen. Hierbei wird das Smartphone der Person als Eingabegerät mit einem sekundären physischen Bildschirm verbunden, um die Ein- und Ausgabemodalitäten zu erweitern. Basierend auf unseren Erkenntnissen aus den einzelnen Fallstudien haben wir eine neuartige Taxonomie zur Kategorisierung von Mixed-Reality-Erfahrungen sowie Richtlinien für die Gestaltung von solchen entwickelt. Die Taxonomie basiert auf dem menschlichen Sinnessystem und den Artikulationsfähigkeiten. Wir stellen die vielseitige Verwendbarkeit vor und setzen unsere Fallstudien in Kontext, indem wir sie innerhalb des taxonomischen Raums einordnen. Die Gestaltungsrichtlinien sind in nutzerzentrierte und technologiezentrierte Richtlinien unterteilt. Es ist unsere Anliegen, dass diese Gestaltungsrichtlinien zu einer erfolgreichen Zukunft von Mixed-Reality-Systemen beitragen und gleichzeitig die neuen Interaktionsparadigmen hervorheben

Wearable sensors for learning enhancement in higher education

Wearable sensors have traditionally been used to measure and monitor vital human signs for well-being and healthcare applications. However, there is a growing interest in using and deploying these technologies to facilitate teaching and learning, particularly in a higher education environment. The aim of this paper is therefore to systematically review the range of wearable devices that have been used for enhancing the teaching and delivery of engineering curricula in higher education. Moreover, we compare the advantages and disadvantages of these devices according to the location in which they are worn on the human body. According to our survey, wearable devices for enhanced learning have mainly been worn on the head (e.g., eyeglasses), wrist (e.g., watches) and chest (e.g., electrocardiogram patch). In fact, among those locations, head-worn devices enable better student engagement with the learning materials, improved student attention as well as higher spatial and visual awareness. We identify the research questions and discuss the research inclusion and exclusion criteria to present the challenges faced by researchers in implementing learning technologies for enhanced engineering education. Furthermore, we provide recommendations on using wearable devices to improve the teaching and learning of engineering courses in higher education

Confronting people's fears about bats: Combining multimodal and environmentally sensed data to promote curiosity and discovery

Bats are often disliked and feared by people. How might we enable the general public to learn more about the true nature of these creatures, and even to like them? In this paper, we introduce PlayBat, a physical public display, which combines a multi-modal interface, a constrained narrative structure and real-time IoT environmentally sensed bat call data. The aim of our research is to investigate whether promoting curiosity and discovery through enabling people to explore real-life data, answer quiz-like questions and engage with a multi-modal interface, is effective at engaging people and confronting their fears. We report on the design process and implementation of PlayBat, and the findings from an in-the-wild study. We discuss how tapping into multiple senses can draw people in, evoke curiosity and even change their views

Mapping and analysing prospective technologies for learning – Results from a consultation with European stakeholders and roadmaps for policy action

EU policies call for the strengthening of Europe’s innovative capacity and it is considered that the modernisation of Education and Training systems and technologies for learning will be a key enabler of educational innovation and change. This report brings evidence to the debate about the technologies that are expected to play a decisive role in shaping future learning strategies in the short to medium term (5-10 years from now) in three main learning domains: formal education and training; work-place and work-related learning; re-skilling and up-skilling strategies in a lifelong-learning continuum. This is the final report of the study ‘Mapping and analysing prospective technologies for learning (MATEL)' carried out by the MENON Network EEIG on behalf of the European Commission, Joint Research Centre, Institute for Prospective Technological Studies. The report synthesises the main messages gathered from the three phases of the study: online consultation, state-of-the-art analysis and a roadmapping workshop. Eight technology clusters and a set of related key technologies that can enable learning innovation and educational change were identified. A number of these technologies were analysed to highlight their current and potential use in education, the relevant market trends and ongoing policy initiatives. Three roadmaps, one for each learning domain, were developed. These identified long-term goals and specific objectives for educational change, which in turn led to recommendations on the immediate strategies and actions to be undertaken by policy and decision makers.JRC.J.3-Information Societ