This poster sets out the background and development of the LACE Evidence Hub, a site that gathers evidence about learning analytics in an accessible form. The poster also describes the functionality of the site, summarises its quantitative and thematic content to date and the state of evidence. In addition, it encourages people to add to and make use of the Hub

Clow, Doug

Ferguson, Rebecca

English

Open Research Online (The Open University)

Open Research OnlineThe Open University’s repository of research publicationsand other research outputsLearning Analytics Community Exchange: EvidenceHubConference or Workshop ItemHow to cite:Ferguson, Rebecca and Clow, Doug (2016). Learning Analytics Community Exchange: Evidence Hub. In: 6thInternational Learning Analytics and Knowledge (LAK) Conference, 25-29 Apr 2016, Edinburgh, Scotland.For guidance on citations see FAQs.c© Copyright 2016 ACMVersion: Accepted ManuscriptLink(s) to article on publisher’s website:http://lak16.solaresearch.org/Copyright and Moral Rights for the articles on this site are retained by the individual authors and/or other copyrightowners. For more information on Open Research Online’s data policy on reuse of materials please consult the policiespage.oro.open.ac.ukLearning Analytics Community Exchange: Evidence Hub Rebecca Ferguson The Open University Walton Hall Milton Keynes +44 (0)1908 654956 Rebecca.ferguson@open.ac.uk      Doug Clow The Open University Walton Hall Milton Keynes +44 (0)1908 654861 Doug.clow@open.ac.uk   ABSTRACT This poster sets out the background and development of the LACE Evidence Hub, a site that gathers evidence about learning analytics in an accessible form. The poster also describes the functionality of the site, summarises its quantitative and thematic content to date, and assesses the state of evidence. In addition, it encourages people to add to and make use of the Hub. Categories and Subject Descriptors A.0 [General]: Reference General Terms Management, Documentation, Performance, Theory, Verification Keywords Ethics, evidence, learning, learning analytics, take-up, teaching 1. INTRODUCTION Educational institutions worldwide are interested in using data-informed planning and decision making to improve their learning and teaching. Learning analytics applications offer to provide institutions with opportunities to support learner progression, enabling rich personalized learning at scale. The use of large datasets, powerful analytics engines and clear visualisations could enable institutions to use the experience of the past to create supportive, insightful models of learning processes. However, ‘due to the limited number of broad scale strategic and policy examples available across the education sector, identifying the precise competitive advantages that analytics can bring to the education space is a multifaceted and complex undertaking’ [6].  Gathering evidence about the successes and failure of learning analytics is not easy, particularly as developers and school and workplace educators often find themselves blocked by a pay-wall when they try to access relevant research. Even for those who can examine these papers and reports, ‘with analytics and data mining experiments in education starting to proliferate, sorting out fact from fiction and identifying research possibilities and practical applications are not easy’ [1]. In 2011, the LAK Dataset was created to help address these problems. As its homepage states, the dataset ‘makes publicly available machine-readable versions of research sources from the Learning Analytics and EDM communities, where the main goal is to facilitate research, analysis and smart explorative applications.’ Since its creation, a series of ‘Data Challenges’ has inspired researchers and developers to explore it in many ways, providing important insights into the field of learning analytics research [5; 7; 8]. However, the LAK Dataset does not make it easy to obtain a coherent and consistent overview of evidence and the conditions under which learning analytics have a positive/negative impact on teaching and learning. To address this gap in the resources available to the learning analytics community, the Learning Analytics Community Exchange project (LACE) has developed an Evidence Hub. The LACE project brings together key European players in the fields of learning analytics and educational data mining who are committed to building communities of practice and sharing emerging best practice. One of the project’s main activities is the creation and curation of the Hub, a knowledge base of evidence that will enable the community to assess the effectiveness and relative desirability of outcomes resulting from the use of learning analytics tools and techniques. The development of this Hub by LACE draws on previous work carried out by one of the project partners, the UK’s Open University (OU). 2. PREVIOUS EVIDENCE HUB WORK The OU has developed several tools and approaches designed to link evidence together in a reliable and robust manner. For example, the Evidence Hub for Open Education was developed ‘to provide an environment to systematically interrogate the Open Education movement on what are the people, projects, organizations, key challenges, issues, solutions, claims and evidence that scaffold the movement’ [3]. This site was designed to address the need for better ways to pool, map and harness what a community knows. It was developed as a collaborative knowledge-building (specifically evidence-building) web platform that could highlight the importance of understanding different perspectives and support quality debates [2].  Today, the Open Education Resources Research Hub (OERRH) provides a focus for research, designed to provide answers to the overall question ‘What is the impact of OER on learning and teaching practices?’ and identify the influence of openness. Its content currently includes more than 6,000 responses to 20 surveys exploring the impact of open educational resources [4]. Many claims have been made about OER and the OERRH is therefore structured around a set of 11 hypotheses that cover performance, access, retention, support and related areas. Research and resources included within it are always related to this specific set of claims that people have made about OER and  Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author. Copyright is held by the authors LAK ’16, April 25–29, 2016, Edinburgh, United Kingdom ACM 978-1-4503-4190-5/16/04 http://dx.doi.org/10.1145/2883851.2883878 their potential, and evidence is always presented as evidence for or against one or more of these hypotheses. 3. LACE EVIDENCE HUB The LACE project has used similar principles to design its Evidence Hub. This is organised around four key propositions, that learning analytics: • Improve learning outcomes • Improve learning support and teaching, including retention, completion and progression • Are taken up and use widely, including deployment at scale • Are used in an ethical way. Evidence that is included within the hub is always given a polarity (positive, negative or neutral/mixed) in relation to one or more of these propositions. However, because the field of learning analytics is relatively new, much of the work that will lead to substantial evidence is still in its infancy. Therefore the LACE hub also includes a Projects section, which gathers together ongoing work in the area Evidence in the Hub can be visualized and interrogated in several ways. The Search facility enables visitors to search by keyword and to focus on a specific country or sector (informal, schools, universities, workplace or cross-sector). Once a search has been run, Summaries of the evidence can be viewed. Each of these contains a link to the original evidence, which is not stored in the Hub. Summaries can be accessed in a variety of ways. A Country Map shows the balance of negative and positive evidence in different countries. The Evidence Map also starts with a view of the world, but this time it can be searched by proposition, polarity and/or sector, or title keywords. Finally, the Evidence Flow diagram shows how much of the evidence relates to each proposition, how much originates in each sector of education, and how much is positive, negative and neutral/mixed. 4. USE OF THE LACE EVIDENCE HUB This year, for the first time, the submission system for the LAK conference has been aligned with the Evidence Hub, so that evidence can be directly imported to the site and added to the evidence that is already in place on the site.  As the Evidence Hub develops, a view of the state of learning analytics emerges that can be used to guide future activity and research. The site is currently being used as an evidence source by LAEP – a European project that is exploring the implications and opportunities of learning analytics for European educational policy.  An overview of evidence currently in the Hub, using the Evidence Flow diagram, revealed that most of the evidence currently available is positive or, at worst, neutral or mixed. As a result of this finding, the LACE project has set up a ‘Failathon’ workshop at LAK16 to explore the apparent lack of negative evidence generated by the learning analytics community. Visitors to the poster will be encouraged to contribute to the Evidence Hub directly at http://evidence.laceproject.eu/ 5. ACKNOWLEDGMENTS The LACE Evidence Hub is powered by WordPress and based on the OER Research Hub developed by Rob Farrow and Martin Hawksey in the Institute of Educational Technology at The Open University as part of a project funded by the Hewlett Foundation. The European Commission Seventh Framework Programme funds the LACE project: grant number 619424. 6. REFERENCES [1] Bienkowski, M., Feng, M., and Means, B. 2012. Enhancing teaching and learning through educational data mining and learning analytics: An issue brief. US Department of Education, Office of Educational Technology. [2] De Liddo, A. and Buckingham Shum, S., 2013. The Evidence Hub: harnessing the collective intelligence of communities to build evidence-based knowledge. Paper presented at Large Scale Ideation and Deliberation Workshop (Munich, Germany, 29 June-2 July 2013). [3] De Liddo, A., Buckingham Shum, S., Mcandrew, P., and Farrow, R., 2012. The Open Education Evidence Hub: a collective intelligence tool for evidence based policy. Paper presented at Global 2012 Conference (Cambridge, UK, 16-18 April 2012). [4] De Los Arcos, B., Farrow, R., Perryman, L.-A., Pitt, R., and Weller, M. 2014. OER Evidence Report 2013-2014 OER Research Hub. [5] Derntl, M., Günnemann, N., and Klamma, R., 2013. A dynamic topic model of learning analytics research. In proceedings of LAK Data Challenge: LAK13 (Leuven, Belgium, 2013), CEUR. URL: http://ceur-ws.org/Vol-974/lakdatachallenge2013_01.pdf. [6] Siemens, G., Dawson, S., and Lynch, G. 2013. Improving the quality of productivity of the higher education sector: Policy and strategy for systems-level deployment of learning analytics. SoLAR. [7] Taibi, D. and Dietze, S., 2013. Fostering analytics on learning analytics research: the LAK dataset. In proceedings of LAK Data Challenge: LAK13 (Leuven, Belgium, 2013), CEUR. URL: http://ceur-ws.org/Vol-974/lakdatachallenge2013_preface.pdf. [8] Zouaq, A., Joksimovic, S., and Gasevic, D., 2013. Ontology learning to analyze research trends in learning analytics publications. In proceedings of LAK Data Challenge: LAK13, (Leuven, Belgium, 2013), CEUR. URL:  http://bit.ly/1kN2Y9W.    

Learning Analytics Community Exchange: Evidence Hub

http://oro.open.ac.uk/45314/1/LAK16%20poster%20revised.pdf

Learning Analytics Community Exchange: Evidence Hub

Abstract

Similar works

Full text

Available Versions

Open Research Online (The Open University)