Proceedings of the 5th Workshop on Awareness and Reflection in Technology Enhanced Learning

Awareness and reflection are viewed differently across the disciplines informing Technology Enhanced Learning (CSCW, psychology, educational sciences, computer science and others). The ARTEL workshop series brings together researchers and professionals from different backgrounds to provide a forum for discussing the multi-faceted area of awareness and reflection. Through the last ARTEL workshops at EC-TEL the addressed topics are converging towards the usage of awareness and reflection in practice, its implementation in modern organisations, its impact on learners and questions of feasibility and sustainability for awareness and reflection in education and work. To reflect the growing maturity of research in ARTEL over the years the workshop particularly invited contributions that dealt with the application of awareness and reflection in practice. This is encapsulated in the workshop motto: 'Awareness and Reflection in Practice: How can awareness and reflection technology become common in work practice and how does it change work practices?

Supporting Collaborative Reflection at Work: A Socio-Technical Analysis

This study presents an analysis of a tool that supports collaborative reflection at work. So far, research has focused on individual reflection or reflection in an educational context. Therefore, little is known about designing support for collaborative reflection at work. In four studies that use an application for collaborative reflection support, built based on prior empirical work, the paper presents an analysis of the ways workers used the tool for collaborative reflection. The analysis was based on log data and material from interviews and observations. The results show that there were different ways in which people used the application and that the impact of using it differed among groups. Besides positive effects, open issues in reflection support emerged. The paper presents insights on and design challenges for collaborative reflection support and potential solutions for these challenges. The findings are related to a model of collaborative reflection support and they emphasize that such support needs to be understood as socio-technical in nature if it is to succeed in practice. Finally, the study proposes designs for further work on tools supporting collaborative reflection

Enhancing STEM Learning with ChatGPT and Bing Chat as Objects to Think With: A Case Study

This study investigates the potential of ChatGPT and Bing Chat, advanced conversational AIs, as "objects-to-think-with," resources that foster reflective and critical thinking, and concept comprehension in enhancing STEM education, using a constructionist theoretical framework. A single-case study methodology was used to analyse extensive interaction logs between students and both AI systems in simulated STEM learning experiences. The results highlight the ability of ChatGPT and Bing Chat to help learners develop reflective and critical thinking, creativity, problem-solving skills, and concept comprehension. However, integrating AIs with collaborative learning and other educational activities is crucial, as is addressing potential limitations like concerns about AI information accuracy and reliability of the AIs' information and diminished human interaction. The study concludes that ChatGPT and Bing Chat as objects-to-think-with offer promising avenues to revolutionise STEM education through a constructionist lens, fostering engagement in inclusive and accessible learning environments

Supporting undergraduate students’ acquisition of academic argumentation strategies through computer conferencing

Executive Summary Background This research grows out of work on the importance of argumentation in developingstudents’ critical abilities. It focuses attention on how students argue in computer mediated conferences as opposed to traditionalwritten assignments, investigating the way in which argumentation is realised within the relatively new context of computer conferencing which allows extended written discussions to take place overa period of weeks. Such text-based asynchronous conferencing is typically characterised by features of both spoken and written modes. Aims The main aims of the project were: • to investigate the argumentation strategies used in asynchronous text-based computer conferences; • to compare the argumentation strategies developed through conferencing with those used in the writing of academic assignments; • to examine the strategies used by tutors to encourage and facilitate argumentation in text-based computer conferences. Methods Data was collected over two years for the distance undergraduate course ‘Perspectives on Complementary and Alternative Medicine’ at the Open University.Qualitative data was obtained through interviews with the course chair, tutors and students, and through a student questionnaire. Assignments and computer-mediated tutorials were collected for textual analysis, although the timing of the assignments meant that analysis has only just begun on the essay data. To analyse the argumentation in the computer conferences and assignments a method of categorising, coding and tracking argumentative discourse was developed building on earlier work by the authors. In addition, computational searches were carried out to compare linguistic features across conference and assignment data. Results In tutorial conferences, student discussion tended to take the form of collaborative co-construction of an argument through exchanging information and experience to substantiate a position. However, students were also prepared to challenge other viewpoints. In both cases, they frequently drew on personal and professional experience to support argument claims. The use of these strategies suggests that text-based conferencing lends itself to the collective combining of diverse sources of information, experiences and ideas. Conference discussions were often personalised with fewer explicit logical links marking argument structure. They were also marked by complexity of argument strands, many of which reached no conclusion. Preliminary analysis of argumentation in assignments suggests that this did not, however, adversely affect students’ ability to create a more traditional, linear argument in their essays. Further analysis will be undertaken to compare argumentation strategies across the two sets of data. Tutors expressed concern about levels of participation in the tutorial conferences, which varied quite considerably. They also felt uncertain about their own knowledge of appropriate pedagogic strategies which would encourage students to participate in a collaborative yet critical way, and tended to rely on strategies from face-to-face teaching. Analysis of the conference discussion showed that tutors made fewer claims than students and were also less likely to provide information in support of their claims. There was, therefore, little modelling by tutors of the basic type of argumentation that would be expected in formal written assignments.Despite these concerns, student responses indicated that having a tutor and a group of peers to interact with, or just to observe, was valued as a supportive feature of this form of distance learning. No clear picture arose of how to make conferencing more interactive for more students, and this reinforces the sense gained from the tutor interviews of the difficulty of proposing a model of tutoring in computer conferences that will necessarily engage all students or raise the level of discussion and debate. Conclusions Our study suggests that text-based conferencing has an important role to play in developing students’ argumentation strategies and understanding of academic discourse and conventions. In view of its hybrid nature, somewhere between spontaneous speech and formal academic writing, course designers and tutors should aim to take advantage of both aspects – on the one hand, the informal dialogic exchange of opinions and co-construction of knowledge, and on the other,the opportunity for consolidation, reflection and re-positioning. Our findings reinforce the view that students’ willingness to exchange ideas freely and openly is partly a consequence of how personally engaged, at ease and confident students feel with one another and their tutor. In particular, it seems that there is a role for the interpersonal and, to some extent, the chat and the frivolity, which in some other studies discussed in the literature review have been regarded as negative influences. Recommendations To facilitate students’ development of argumentation and learning more generally,tutors need greater awareness of the ways in which academic argumentation operates in computer conferencing as compared to written assignments. Since pedagogic strategies developed in other contexts may not transfer well to computer conferencing, there is a need for targeted professional development, focussing in particular on: • Choosing topics for discussion and designing effective task prompts; • Supporting weaker students; • Encouraging challenging of ideas; • Finding the right tone to facilitate peer discussions. Some specific suggestions are made within the report, but our recommendations at this stage remain tentative as we still have to complete the analysis of the assignment data and draw conclusions about the impact of the computer conferencing on the quality of written argumentation within this more formal context

Re-Framing the knowledge to action challenge through NIHR knowledge mobilisation research fellows. Comment on “CIHR Health System Impact Fellows: Reflections on ‘Driving Change’ Within the Health System"

The ambition of the Canadian Institutes for Health Research (CIHR) Health Service Impact (HIS) Fellowship initiative to modernise the health system is impressive. Embedded researchers who work between academia and non-academic settings offer an opportunity to reframe the problem of evidence uptake as a product of a gap between those who produce knowledge and those who use it. As such, there has been an increasing interest in the potential of people in embedded research roles to work with stakeholders in the co-production of knowledge to address service challenges. In this commentary, we draw on research and experiential evidence of an embedded researcher initiative, which has similar intentions to the HIS Fellowships programme: the National Institute for Health Research (NIHR) Knowledge Mobilisation Research Fellowship (KMRF) scheme. We outline the similarities and differences between the two schemes, and then consider the work, characteristics and skills, and organisational arrangements evident in operationalising these types of roles

The personal created through dialogue: enhancing possibilities through the use of new media

This paper explores the relationships between a number of different developments in higher education pedagogy, which are subsumed under the broad heading of progress files. The overall concern of the paper is to explore the ways in which personal reflection and learning is enhanced through dialogue. The paper explores the ways learners engage in dialogue in two environments that use different aspects of digital technologies to support the development of portfolios. The findings from the case studies point to the ways in which different technologies facilitated personal reflection mediated through sharing and dialogue. We develop the idea of affordances as a relationship whereby the learner is involved in a purposeful engagement with the possibilities created by their environment. The affordance of digitised technologies in supporting dialogue is, therefore, conceptualised in relation to the characteristics of the learner, not as a simple technology relation

Improving Requirements Generation Thoroughness in User-Centered Workshops: The Role of Prompting and Shared User Stories

The rise of stakeholder centered software development has led to organizations engaging users early in the development process to help define system requirements. To facilitate user involvement in the requirements elicitation process, companies can use Group Support Systems (GSS) to conduct requirements elicitation workshops. The effectiveness of these workshops for generating a valuable set of requirements for system developers has been previously demonstrated. However, a more representative measure of progress towards a system that will meet users’ needs-- the completeness of the requirements generated by such groups has not been explored. We explore two process design considerations for increasing the completeness of requirements generated by these users: increased sharing of user stories (individual electronic brainstorming groups vs. shared user stories electronic brainstorming groups), and the use of reflective inducement prompts (unprompted vs. prompted groups). Using the Search for Ideas in Active Memory model, we predict that prompted electronic brainstorming groups will outperform any other group, including prompted, shared user stories groups at generating a more thorough set of requirements. To test the hypotheses an experiment with 56 groups consisting of 197 users was conducted. The users were asked to generate requirements for a fictitious online textbook exchange website. All hypotheses received support. The study has implications for GSS-Supported workshop design and for future research on collaborative performance in requirements elicitation

Designing Interactive Graphics for Validating and Interpreting Storm Track Model Outputs

We report on some initial work in which we designed interactive graphics to help climate scientists identify and extract good examples of simulated storm-tracks from a large dataset to help disseminate information to various audiences. A side-effect of this work was that the exploratory potential offered by the interactive graphics helped our climate scientist coauthors validate and interpret their data in a way that was not previously possible for them. We are extending this work to provide support for a wider range of validation and interpretative tasks, with a focus on answering questions of relevance to the insurance industry. We describe our collaborative approach, that draws on ideas from ’patchwork prototyping’ [2, 5] in which a rapid iterative process of design, implementation and testing, is used to help provide the functionality to support a set of ‘user stories’

Helping design educators foster collaborative learning amongst design students

This paper discusses the development of online teaching resources that enable design educators to foster collaborative learning amongst students in the design disciplines. These online teaching resources will be made available through the Design Collaboration website. This website was recently set up by Northumbria University, a UK based institution, to provide an online resource for design educators wishing to develop collaborative pedagogies in design education. It currently contains case studies of collaborative student projects but lacks practical teaching resources. As a result, a research project was set up to compliment the current case studies by creating a suite of design-specific tools and resources that will help foster team management and development. Although various institutions have addressed the subject of group work and collaborative learning, there has been no online resource dedicated to the development of practical teaching tools to help design students work and learn together. This paper focuses on showcasing the range of teaching tools and resources developed through classroom-based trials. These resources have been developed specifically in consultation with Northumbria University's design educators and trialled with undergraduate and postgraduate students from different design disciplines. In addition, issues surrounding the translation of these tools into a practical, easy to use and accessible in an online format is discussed. The Icograda World Design Congress 2009 Education Conference is the ideal international platform to share these tools with the wider design education community. More importantly, we hope to grow the website by encouraging other design educators to submit case studies to the website, using it not only as a means of sharing good practice but also as a tool for reflection. The research value is two-fold (a) translating implicit knowledge of collaborative learning into a practical teaching resource and, (b) helping tutors improve their teaching practice, by linking the teaching resource to real experiences through case studies and interviews

Human-AI Collaboration for Smart Education: Reframing Applied Learning to Support Metacognition

This chapter investigates the profound influence of intelligent virtual assistants (IVAs) on the educational domain, specifically in the realm of individualized learning and the instruction of writing abilities and content creation. IVAs, incorporating generative AI technologies such as ChatGPT and Stable Diffusion, hold the potential to bring about a paradigm shift in educational programs, emphasizing the enhancement of advanced metacognitive capacities rather than the fundamentals of communication. The subsequent recommendations stress the need to cultivate enduring proficiencies and ascertain tailored learning approaches for each learner, which will be indispensable for success in the evolving job market. In this context, prompt engineering is emerging as a vital competency, while continuous reskilling and lifelong learning become professional requisites. The proposed innovative method for teaching writing skills and content generation advocates for a reconfiguration of curricula to concentrate on applied learning techniques that accentuate the value of contextual judgment as a central pedagogical tenet and the mastery of sophisticated metacognitive abilities, which will be pivotal in the future of work