There is a growing interest in the potential for technology to facilitate emergency education of refugee children. But designing in this space requires knowledge of the displaced population and the contextual dynamics surrounding it. Design should therefore be informed by both existing research across relevant disciplines, and from those who are on the ground facing the problem in real life. This paper describes a process that is based on literature from emergency education, student engagement and motivation, educational technology, and participatory design. We describe how this process was implemented leading to the design of a digital learning space for children living in a refugee camp in Greece. The challenge of data analysis is critical, as the qualitative data in the process is elicited from activities of various natures and thus moving from qualitative data to designs is a critical challenge that we are looking to cover for our process to be complete and applicable. We discuss some of the challenges that can be expected in such context

Adams, Anne

Alain, George

Coughlan, Tim

Yanacopulos, Helen

English

Open Research Online

Open Research OnlineThe Open University’s repository of research publicationsand other research outputsQualitative Data Analysis Challenges in Co-DesigningEducational Technology Systems for Refugee ChildrenConference or Workshop ItemHow to cite:Alain, George; Coughlan, Tim; Adams, Anne and Yanacopulos, Helen (2018). Qualitative Data Analysis Challengesin Co-Designing Educational Technology Systems for Refugee Children. In: British HCI 2018, Workshop Title:Qualitative Data Research in HCI, 3 Jul 2018, Belfast.For guidance on citations see FAQs.c© [not recorded]Version: Accepted ManuscriptCopyright and Moral Rights for the articles on this site are retained by the individual authors and/or other copyrightowners. For more information on Open Research Online’s data policy on reuse of materials please consult the policiespage.oro.open.ac.uk                              1 © Alain et al. Published by BCS Learning and Development Ltd. Proceedings of British HCI 2018. Belfast, UK. Qualitative Data Analysis Challenges in Co-Designing Educational Technology Systems for Refugee Children  George Alain     Tim Coughlan     Anne Adams     Helen Yanacopulos         Institute of Educational        Institute of Educational        Institute of Educational                 Faculty of Arts &       Technology      Technology       Technology         Social Sciences           The Open University           The Open University           The Open University                 The Open University             Milton Keynes, UK            Milton Keynes, UK               Milton Keynes, UK                   Milton Keynes, UK        george.alain@open.ac.uk    tim.coughlan@open.ac.uk  anne.adams@open.ac.uk     h.yanacopulos@open.ac.uk  There is a growing interest in the potential for technology to facilitate emergency education of refugee children. But designing in this space requires knowledge of the displaced population and the contextual dynamics surrounding it. Design should therefore be informed by both existing research across relevant disciplines, and from those who are on the ground facing the problem in real life. This paper describes a process that is based on literature from emergency education, student engagement and motivation, educational technology, and participatory design. We describe how this process was implemented leading to the design of a digital learning space for children living in a refugee camp in Greece. The challenge of data analysis is critical, as the qualitative data in the process is elicited from activities of various natures and thus moving from qualitative data to designs is a critical challenge that we are looking to cover for our process to be complete and applicable. We discuss some of the challenges that can be expected in such context. Co-design; Participatory design; refugees; qualitative data analysis; thematic data analysis. 1. INTRODUCTION According to the United Nations High Commissioner for Refugees (UNHCR), there are at least 65 million forcibly displaced people worldwide. Around 25 million of which are refugees fleeing their homeland, mostly because of war and persecution. Moreover, 51% of the displaced population are children. Even in the most positive estimates, only 50% of refugee children are attending any sort of schooling (UNHCR, 2016b, 2016a).  There is a growing interest in the potential for technology to facilitate emergency education, but designing in this space requires knowledge of the displaced population and the contextual dynamics surrounding it. Design should therefore be informed by both existing research across relevant disciplines, and those who are on the ground facing the problem in real life. In our paper (Alain et al., 2018) we discussed literature from different disciplines such as emergency education, student engagement, and educational technology design. Emergency education stresses three points which are: the importance of understanding the context of displacement, the need to assess educational content suitability, and the involvement of the researched community by following participatory approaches. (Talbot and Muigai, 1998; Pigozzi, 1999; Sinclair, 2002; Kagawa, 2005). The literature on student engagement suggests that teachers, peers, parents, the social context, and the nature of academic work can all support engagement and provided a set of characteristics of educational content that support student engagement (Skinner and Pitzer 2012). Educational technology design literature emphasised that technology can support student engagement and learning only when it is designed to tackle specific identified challenges and educational goals (Kadiyala Madhavi, 2000). Since involving the participants and the community is essential, we reviewed and adopted a participatory design PD approach as discussed in (Scaife and Rogers, 1999; Muller, 2003) with focus on PD with children such as in (Druin, 2002; Fisher, Yefimova and Yafi, 2016). According to the literature discussed above, we devised a process of four stages. We start with understanding the conflict, population, and demography. This is followed by context and location orientation, then comes the educational problem definition from the perspective of different participant groups, then we move to designing solutions that tackle the identified problems. However, moving from the data collected at different stages to the designs is a challenging process. In this paper, we discuss the implementation of the Qualitative Data Analysis Challenges in Co-Designing Educational Technology Systems for Refugee Children Alain ● Coughlan ● Adams ● Yanacopulos 2 devised design process at Ritsona refugee camp in Greece to design an educational technology system for Syrian refugee children. We will start with the data collection stages, then move to the data analysis challenges and the techniques we followed to overcome these challenges. 3. DATA COLLECTION This section will provide a brief overview of each of the design process stages in addition to the data collected at each stage. For further details on each stage, please refer to (Alain et al., 2018). 3.1 Understanding the conflict and the population Conducting this stage allows us to have a better understanding of the conflict, the displacement, the demography, the conflict related tensions, and the possible contextual, and socio-cultural sensitive matters that may arise in the later stages which involve direct communication with the displaced population. Moreover, this is essential to assess the educational content later to avoid any clashes with the conflict and society related sensitive matters.  The data resulted from this stage was a clear understanding of the Syrian conflict, the tensions, population demography, with a list of the possible sensitive topics that could affect the interaction with the researched community and the educational content. 3.2 Location orientation This stage aims to understand the location, the demography of the population living in the location, and the available resources. Moreover, this stage aims to identify the different groups that can participate in the problem definition and design stages, and to recognise their areas of expertise and the methods that can be used to get their input. We identified three main groups of participants which are: on-site social-workers and NGO staff, parents, and children. We were unable to work with educators as there was no educational provision in the camp which caused a challenge in the data analysis which will be discussed later. Data collection in this stage followed an ethnographic approach. Thus, data included lots of observation and personal notes on events, discussions, and notes which may have a direct or indirect reflection on the design process. 3.3 Educational Problem Definition In this stage, we work on understanding the problem definition in addition to the perception of education from all different identified participant groups. This is to identify specific educational needs and challenges which technology can then be designed to tackle. This stage consists of two parts, the first part is with adult participants, and the second is with children participants. This is to conceptualise education and educational needs and challenges from the perspective of both adults and children between the ages of 8 and 15. With adults, data collection semi-structured interviews and focus groups with social-workers, NGO volunteers, and parents. The data from this part consisted of transcribed recordings. The second part of this stage was with children. We asked them about their perception of education, the purpose of education, what they want to get educated about and why, what engages or disengages them the most. The data collection was conducted in a photography workshop and a sticky note workshop. Data collected in this part involved photographs, sticky notes of factors that make education either satisfying or dissatisfying for children, and audio recordings of the workshops. 3.4 Co-Design This stage aims to design solutions in response to identified educational challenges and needs. Design work starts with children envisaging solutions and producing requirements. Children are then asked to create both the pedagogical and technological aspects of the design, including contextual elements. The children’s designs are then brought to adult workshops where they are matched with the available resources such as locations, time, human resources, equipment, and funding to insure applicability and sustainability. This produces a design where the requirements are derived from children to ensure that it is engaging and suitable for them, and at the same time is applicable and sustainable to the context in which it will be used. In the case study, the activities with children consisted of co-design workshops that involved drawing, LEGO, and modelling. We followed a similar sampling approach to the previous activities but with only 3 to 4 children at a time. Inspired by Fisher, Yefimova and Yafi (2016), children were asked to create the “Magical Learning Machine” but we empathised more focus on educational systems. Data collected was a collection of drawings, 3D LEGO models, and recordings of the workshops.  Figure 1: Co-design workshops with children Qualitative Data Analysis Challenges in Co-Designing Educational Technology Systems for Refugee Children Alain ● Coughlan ● Adams ● Yanacopulos 3  4. THE RESULTED SYSTEM The resulted designs, educational needs and challenges from previous stages, were presented to the adult participants. We worked with them to prioritise the designs by mapping them with the available resources. In this case, due to the unavailability of teachers, the chosen outcome was a digital self-learning space at the camp using an available caravan and a set of tablets that were donated earlier but were not being used. Children would attend classes at the caravan where they will be able to use tablets with pre-loaded applications that support the educational needs which they had stated earlier that was mostly about literacy of a set of language.  The content was chosen to match the pedagogical designs of the children, and was assessed against aspects of engagement, motivation, and gamification literature. Evaluation sessions were conducted throughout the pilot and implementation phases to insure that the content and final design matched their designs and expectations. 5. DATA ANALYSIS AND CHALLENGES Throughout the research, data was thematically analysed to achieve the resulted design and as additionally to evaluate and improve the design process. The analysis of the qualitative data gathered in the previous stages to create designs is a challenging process for many reasons. In this section, we will discuss the challenges that we faced and how we tried to overcome these challenges. 5.1 Various data formats from different sources One of the major challenges was that the collected data is very different in both the data format and the sources from which it came. The data ranges from interviews and focus group recordings, personal observations and notes, photographs, sticky notes, drawings, and LEGO and 3D models. There is no easy way to overcome this, however, we used data analysis software to organize the data by source, sessions, and cases. This was followed by transcribing the data sources including the photographs and drawing scans into text format data accompanied by the conversation that took place when the design idea was discussed. This allowed capturing the design model, and children ideas by which it was led. 5.2 The absence of some participant groups In the instances of displacement, there are very limited resources including human resources such as teachers. The absence of teachers in the problem definition and design process has affected the system designs which would cause learning limitations and sustainability concerns. We could not involve teachers in the process. But in our workshops with other participants, we tried discussing the solutions from the perspective of the teachers since many of the volunteers have done some teaching work themselves with the refugees before. Additionally, we covered further literature on emergency education and papers that discussed the inputs of the educationalists in similar contexts at different locations. 5.3 Time constraints When working in such context, the access to the locations, the availability of the participants, and the time limitations are extremely challenging. It is very common that one day a whole camp can be relocated or closed, or access to the camp and the participants to be lost for any reasons. During the field work in Greece, we had very limited time for the analysis, so we conducted a light analysis leaving the full analysis to after coming back to the UK. This was reasonable especially that our main goal is to develop a design process from real life case studies. 5.4 Ethnography data analysis As we discussed above, we followed an ethnography data collection approach at the first two stages. On the one hand, data from observation can allow a better problem definition and system requirement (Crabtree, 1998), especially when working in the context of refugee camps where most people are not experts in education or technology. On the other hand, the analysis of such data is not free from debates and requires knowledge of how and where it can go wrong (Dourish, 2006). The system requirements that were derived from the ethnography data had to be triangulated with data from participants and at some points discussed directly with the participants to make sure that such requirements are relevant.  5.5 Thematic analysis approach In thematic analysis, there is a differentiation between an inductive approach and deductive approach. On the one hand, we would benefit from an inductive approach allowing the themes to emerge from the data to highlight topics and dynamics that affect the design process. On the other hand, at several design stages, specific questions were asked such as “what are the educational challenges and needs”. Such direct questions would force specific themes to build up and thus, a deductive approach may be more suitable when answering a specific question. Thus, the use of hybrid approach was needed in the data analysis phase.   Qualitative Data Analysis Challenges in Co-Designing Educational Technology Systems for Refugee Children Alain ● Coughlan ● Adams ● Yanacopulos 4 6 FUTURE WORK The process of data analysis is still in early stages, and we are still considering different methods and techniques. Our main goal is to analyse the data from two case studies in Greece and at the same time be able to provide a thorough data analysis guidelines in our design process for any future research work in this context. 7. REFERENCES Alain, G. et al. (2018) ‘A Process for Co-Designing Educational Technology Systems for Refugee Children’, in British HCI 2018. Belfast. Crabtree, A. (1998) ‘Ethnography in Participatory Design’, in Proceedings of the 1998 Participatory Design conference, pp. 93–105. Available at: http://www.cs.nott.ac.uk/~pszaxc/work/PDC98.pdf (Accessed: 17 June 2018). Dourish, P. (2006) ‘Implications for design’, Conference on Human Factors in Computing Systems - Proceedings, 1, pp. 541–550. doi: 10.1145/1124772.1124855. Druin, A. (2002) ‘The role of children in the design of new technology’, Behaviour & Information Technology, 21(1), pp. 1–25. doi: 10.1080/01449290110108659. Fisher, K. E., Yefimova, K. and Yafi, E. (2016) ‘“ Future ’ s Butterflies :” Co-Designing ICT Wayfaring Technology with Refugee Syrian Youth’, ACM SIGCHI Conference on Interaction Design and Children 2016, pp. 25–36. doi: http://dx.doi.org/10.1145/2930674.2930701. Kadiyala Madhavi, C. L. B. (2000) ‘A Review of Literature on Effectiveness of Use of Information Technology in Education’, Journal of Engineering Education, 89(April), pp. 177–189. Kagawa, F. (2005) ‘Emergency education: a critical review of the field’, Comparative Education, 41(4), pp. 487–503. doi: 10.1080/03050060500317620. Muller, M. J. (2003) ‘Participatory design: The third space in HCI’, Human-Computer Interaction Handbook, 4235, pp. 1051–1068. doi: 10.1145/153571.255960. Pigozzi, M. J. (1999) ‘Education in emergencies and for reconstruction: a developmental approach’, UNICEF working paper series, p. 21 p. Available at: http://www.reliefweb.int/rw/lib.nsf/db900sid/LGEL-5G8FMF/$file/unicef-education-1999.pdf CN - PIG 74 (Accessed: 30 April 2017). Scaife, M. and Rogers, Y. (1999) ‘Kids as informants: telling us what we didn’t know or confirming what we knew already?’, in The Design of Children’s Technology. Morgan Kaufmann, pp. 1–26. doi: 10.1145/258549.258789. Sinclair, M. (2002) Planning education in and after emergencies, International Journal of Educational Development. doi: 10.1016/j.ijedudev.2003.08.003. Skinner, E. and Pitzer, J. (2012) ‘Developmental Dynamics of Student Engagement, Coping, and Everyday Resilience’, in Handbook of Research on Student Engagement, pp. 515–539. doi: 10.1007/978-1-4614-2018-7. Talbot, C. and Muigai, K. (1998) ‘Environmental education for refugees: guidelines, implementation and lessons learned.’, Education as a Humanitarian Response, pp. 223–247. UNHCR (2016a) GLOBAL TRENDS: FORCED DISPLACEMENT IN 2016. Available at: http://www.unhcr.org/5943e8a34.pdf (Accessed: 14 May 2018). UNHCR (2016b) ‘Missing out: Refugee education in crisis’, p. 48.   

Qualitative Data Analysis Challenges in Co-Designing Educational Technology Systems for Refugee Children

http://oro.open.ac.uk/55755/1/BHCI-2018_Sourcefile_279%20%281%29.pdf

Qualitative Data Analysis Challenges in Co-Designing Educational Technology Systems for Refugee Children

Abstract

Similar works

Full text

Available Versions

Open Research Online